On Intelligence (2004)
by Jeff Hawkins (1957-)
I read a review of this book when it was published, but it didn’t make much impression on me. Recently, Hawkins (who was the inventor of the Palm Pilot) launched an effort to build machines based on his theory of the brain. This instance of a man putting his money where his mouth is made an impression, so I decided to look at the book.
Hawkins spends the first two chapters talking about himself, his motivations, and his opinion that GOFAI and neural networks are hopeless. When he finally does talk about the brain, the book gets better. Unfortunately, it isn’t at all clear what might be done with Hawkins’s theory, and I don’t know what his new venture hopes to accomplish.
As is well known, the brain is complicated. As with any complicated subject, an approach based on levels of abstraction is necessary to make progress toward understanding. In the case of the brain, the following levels seem to me to be appropriate.
- Molecules – Neurons interact using molecules. Though some are known, there is still a lot to learn, including the molecular basis for various forms of “psychoactive” substances. In particular, understanding the short- and long-term effects of synaptic molecules on the plasticity of the cortex requires more research to understand the formation and modification of synapses.
- Cells – The neurons and supporting cells of the brain, sensory organs, and effectors (driving muscles and glands) are the next lowest level. According to Hawkins, even this level is poorly understood. For instance, the details of creating new synapses between neurons and modifying the strength of existing synapses (presumed crucial in learning) are not known.
- Tissue anatomy – Hawkins makes much about the 6-layer structure of the neocortex, and points out that the cortex is about as thick as a stack of 6 business cards; further, if it were spread out it would be the size of a dinner napkin. He goes into more detail about the layers, but says very little about the ‘old brain’ that the cortex enwraps.
- Cortical architecture – Chapter 6 goes into detail about the nature of the cells in the cortex, and the architecture of their connections. He describes the notion of columns of cells within the cortex, as if they are a fundamental building block; but he also says 90% of the connections on cells in a ‘column’ are to/from cells outside it. A brief primer (with a diagram or two) on brain anatomy might have made some of his discussion easier to follow. An observation and conjecture by Mountcastle is the basis for Hawkins’s theory: the anatomy of the cortex is the same everywhere, and there is a single ‘algorithm’ performed throughout. (The word ‘algorithm’ is used here in a generic sense; essentially nothing is known of its required or actual character, and it certainly isn’t the type of operation that Turing machines perform.)
- Representation – At some point in the processing of sensory inputs, the cortex represents patterns in the input. The nature of the representation is unknown, and Hawkins doesn’t attempt to describe or explain it.
- Represented – One of Hawkins’s innovations is to conjecture that what is represented in the cortex are hierarchies of sequences of patterns. The cortex has no direct access to sensory input, only the afferent neuronal signals generated by the various kinds of sensory apparatus. In fact, in cases of blindness or deafness, the cortex is capable of routing signals from one sensory mode to parts of the cortex that ordinarily process another mode. This fact alone is strong evidence that the cortex applies a single algorithm broadly. In his 1986 paper (unpublished), Hawkins has more detail on the nature of the ‘cortical units’ that perform representation; he only loosely links his cortical units with Mountcastle’s cortical columns.
- Sequences – The patterns represented are not static or in fixed relations to one another. For instance, speech is a temporal sequence of sound-patterns (such as phonemes). Vision is built up from a sequence of smallish visual fields sequenced by the saccades (about three per second). However, the sequence is not strictly temporal: saccades might result in the sequence ‘eye, nose, eye, mouth’ for a second or so, and later ‘mouth, eye, mouth, eye’.
- Hierarchy – The representations in the cortex are hierarchical in the sense that they consist of multiple levels, with higher levels representing more general patterns and depending on lower levels for details. In the visual cortex, the V1, V2, V4, and IT regions form a progression of more general representations from the edge-level to the face-level. Hawkins states that the V1 region is quite large, and should be subdivided along functional lines.
- Prediction – An essential part of the algorithm Hawkins proposes is the prediction from recent patterns of the next few patterns to be expected. The formation of expectations is also represented within the cortical units, and when a predicted pattern fails to occur, or an unpredicted pattern occurs, something like surprise arouses the cortex to seek explanations. Presumably competing patterns that were suppressed by some recent correct predictions receive less inhibition and are able to contribute new predictions.
- Unity of sensory and motor activity in the cortex – Hawkins conjectures that the cortex acts to confirm its own predictions. So if it sees an eye and a nose slightly below and to the right of it, it predicts another eye above and to the right of the nose. The prediction drives a saccade to verify the prediction. Similarly, the presence of an unknown object on the skin results in motor activity such as finger movement to explore the cause and expand the pattern to fit some expectations. In a very literal sense, the ironic “I wouldn’t have seen it if I hadn’t believed it.” is quite true.
There is probably a lot of merit in Hawkins’s approach. However, it is difficult to see how it can be applied in an engineering sense. He supports research along these lines, and presumably hopes to be able to apply it to artifacts. It will be interesting to see if he succeeds.
Some aspects of the book are poorly written. Apparently Hawkins himself is not a good enough writer to carry off the entire book, and was teamed with Susan Blakeslee (an experienced science writer) as co-author. I was confused for a while about the depiction of the six layers of the cortex, and the four regions of the visual cortex. Perhaps if the visual regions had been portrayed as proceeding from right to left, while the six layers are shown vertically stacked, I wouldn’t have been confused.