A long winter
The first neural networks in the 1950s and 1960s promised a lot, but computers were weak and there was little data. “AI winters” followed, with no money and little interest. Geoffrey Hinton, Yann LeCun and Yoshua Bengio kept working on networks even when it was not popular.
Why 2012
Three things came together: huge amounts of data from the internet (the ImageNet database with millions of images), powerful graphics cards and better training methods. AlexNet made far fewer errors than all its rivals.
Successes
- 2012AlexNet recognises images
- 2016AlphaGo beats go master Lee Sedol
- 2017The Transformer, the basis of language models
- 2020AlphaFold predicts the shape of proteins
- 2024Nobel Prizes for neural networks and for AlphaFold