Big (Computer) Brains
(Source: DeepMind PathNet)
I recently wrote a few thoughts on artificial intelligence (AI), primarily in response to Geoffrey Hinton’s excellent short interview. There was one other concept in this interview that I wanted to touch on, which is the relationship between AI and our own intelligence.
In particular, Hinton concludes:
Eventually, I think the lessons we learn by applying deep learning will give us much better insight into how real neurons learn tasks, and I anticipate that this insight will have a big impact on deep learning.
To really appreciate this I think requires a very brief history lesson on the beginnings of machine learning (ML), AI, and neural networks.
In 1943, Warren McCulloch (who, funny enough, turns out to be a cousin of mine) and Walter Pitts proposed the first artificial neuron. It was known as the Threshold Logic Unit or Linear Threshold Unit. The...