Pioneer in AI seeks new kind of deep learning algorithm
Artificial intelligence pioneer Geoffrey Hinton won two major international research awards in 2022: the Royal Society’s Royal Medal and the Princess of Asturias Award for Technical & Scientific Research. The University of Toronto Professor Emeritus shares the Princess of Asturias Award with three other leaders in the field of artificial intelligence known as deep learning: Yoshua Bengio (Université de Montréal), Yann LeCun (New York University) and Demis Hassabis (DeepMind).
His groundbreaking work in deep learning – a field of artificial intelligence that mimics the way humans acquire some kinds of knowledge – has driven advances in many areas.
“I’m trying to figure out how the brain learns,” he says of his research. “And one consequence of that is that various attempts to understand how the brain might be learning turned out to be useful for engineering.
“In particular, there’s an algorithm called backpropagation. And that’s how nearly all the large neural networks are now trained.” Backpropagation started off as an idea about how the brain might acquire knowledge, he says, but now it’s not clear whether the brain actually does backpropagation.
“I suspect that the brain doesn’t do backpropagation,” he says. So I think the technology we’ve got now is working in a different way from the brain. But it’s certainly very useful for engineering. And all of these large language models and large vision models use backpropagation.”
Dr. Hinton explains that backpropagation only works if the neural network has a perfect model of itself. “I think the brain doesn’t,” he says.
“And also analog hardware wouldn’t. So if we want to learn in analog hardware, we’re going to need a different kind of algorithm.”
That answer to that problem is the focus of Dr. Hinton’s current research.
“The question is, what kind of learning algorithm can use [analog hardware]? The brain obviously has a pretty good learning algorithm, but possibly not as good as what we now have for these deep neural networks.
“So for a long time, the aim was to make artificial neural networks work as well as real ones. And I suspect that quite soon there’ll be working better than real ones.”
Dr. Hinton is also chief scientific adviser at the Vector Institute for Artificial Intelligence and a vice-president and engineering fellow at Google. He won the Turing Award, often referred to as the ‘Nobel prize of computing,’ in 2018.