What Is Thought? (2004)
by Eric B. Baum (1941-)
I probably learned of this book from a book review in Science or American Scientist. It is not easy to read, and could have used a little more editing. Nonetheless, it is interesting and thought-provoking. The title is a deliberate play on Shrödinger’s 1944 What Is Life?, which is credited with attracting ‘hard’ scientists such as Francis Crick to biological problems. Baum hopes to attract computational thinkers to the problems of mind.
He begins by stating a bold thesis: “Semantics is equivalent to capturing and exploiting the compact structure of the world, and thought is all about semantics.” A significant part of the book is devoted to describing in some detail this notion of the distinction between syntax (the superficial aspects of the world revealed by the senses, an enormous and enormously complex collection of information) and semantics (the infinitesimally smaller collection of information that allows explanations, predictions and actions in the world).
Baum discusses Church and Turing, and reminds me of the other equivalent formulation: Emil Post’s production system. These take a starting string (e.g., A B A B C B C) and a set of rewrite rules (e.g., x B A B y -> x C B y). Matching and rewriting yields new strings (e.g., A C B C B C). The declarative nature of the syntax, and the matching and variable binding operations, seem to me to offer a promising approach for a higher level of neural-circuit simulation than neural nets, an approach closer to symbol processing.
Baum has worked on neural nets, and addresses a reason they are not more widely used. (This follows an introduction to ideas of complexity and its quantification.)
The inability to understand how nets get to their conclusions is one reason they are not applied more often in exactly this kind of context. For example, a trained medical diagnosis net exists that is more accurate than the average emergency room physician at deciding whether to admit to the hospital people complaining of chest pain, and yet no one is quite willing to replace the judgment of doctors with this net.
This lack of explainability is a practical problem for applications involving people, but it is not an argument demonstrating that the net cannot understand. Indeed, lack of explainability is to be expected if the net does understand. The point is, understanding corresponds to a compact description. Compact description is not the whole story in understanding, but it is integral to it. The trained net already compresses a huge amount of data, which is the reason it understands the process well enough to classify examples it has never seen before. A further understanding of the workings of the net would then require a further compression. That should not exist, or the data are not as compressed as they could be.
In discussing the difference between reaction and reflection in thought, he invokes the impact of evolution on the innate capabilities of the brain, a recurring theme.
It should not be surprising that thought is mostly reactive. First, evolution created us to survive and reproduce, and survival and reproduction are mostly real-time processes. To reflect at the wrong time is to be eaten by a tiger while you are deciding what to do, or to miss saying something witty to a potential mate at an opportune moment. Second, it is seemingly much easier to evolve reactive systems than reflective ones. Simple neural net learning algorithms or simple hill-climbing algorithms can be seen in simulation to give reactive solutions to various toy problems. Evolving complex intermediate representations turns out to be a lot harder, and historically seems to have happened later.
Nevertheless, there is a place for reflection. The contrast between reflective humanlike thinking and reactive behavior is perhaps most deeply respected by ethologists.
Baum goes on to describe how ethologists were influenced by the “Clever Hans” episode to draw a firm line between human mental abilities and animal behaviors. He refers to The Animal Mind (Gould and Gould) for a more modern approach.
Among human abilities is cheating detection. All humans are more adept at detecting when someone is breaking a rule than in performing general logical inference. Baum infers the existence of a special module in human brains for this capability.
Another human capability is the way that metaphor pervades language, to an extent that indicates it is fundamental to our thought processes. He refers to Metaphors We Live By (Lakoff and Johnson). For Baum, metaphor comes from code reuse. “When we understand a concept, it is because we have a module in our minds that corresponds to that concept. The module knows how to do certain computations and to take certain actions, including passing the results of computations on to other modules. Metaphor is when such a module calls another module from another domain or simply reuses code from another module. Then we understand whatever the first module computes in terms of what the second module computes.” After further discussion and examples of metaphoric influence on thought he has this amusing aside.
One cogent point that Lakoff makes repeatedly in his books and essays is that we are prisoners of our metaphors. We understand the world in terms of our metaphors, but our metaphors are not exact, and as a result, we can be mistaken about the world when we apply an inappropriate metaphor. Actually, the most compelling and (to my mind, amusing) example of this phenomenon is Lakoff himself. Lakoff, who is very politically concerned and describes himself as politically liberal, wrote another book, Moral Politics: What Conservatives Know That Liberals Don’t (1996). The point of this book is that we are trapped in our metaphors, and that liberals regard the government as a nurturing mother whereas conservatives regard the government as a stern father. He tries to write about how these metaphors color the respective views, how the views can be seen as coherent from the point of these metaphors. But even as he is attempting to stand aside and analyze the thought processes, he is utterly unable to escape his own metaphors. As he debates the merit of the two positions, nowhere is he able to realize even for an instant that the government is not a parent at all, nor even a person, and that all kinds of things he believes implicitly are thus based on a hopelessly inappropriate metaphor. Since he can’t escape the metaphor, he doesn’t even appear to understand that he is confused.
He goes on:
Our political reasoning is a particularly good example of our illogicality. It can’t possibly be fully logical: half the people are on one side of any issue and half on the other, which implies that that they are not all logically correct, and in fact there is no particular reason to believe that any of them are logically correct. People are simply not evolved to reason logically about politics.
An interesting (if technical) aspect of his research has been to add the concept of property rights to collections of agents attempting to evolve solutions to problems. With that notion, long chains of agents don’t evolve, presumably because the early ones are too far from the reward associated with success. By enforcing a distribution of reward throughout a chain (by an auction-like mechanism), long chains can evolve. However, if money is created or leaks out of the system (by theft), the tendency of agents to maximize their own rewards prevents globally optimal solutions from evolving.
Contrasting toy research problems with the difficulties faced by real people and other creatures, Baum ends chapter 11 with:
The problem of reasoning about the world is thus hard. But people have made enormous progress at it. As I discuss in chapter 13, this is largely because of language. Individuals engage in computationally intensive searches, trying different ways of extending their knowledge. When someone finds a new discovery, a new sequence of thought that goes on beyond what is fully constrained by old modules and yet that usefully exploits structure in the world, he builds a new module in his mind. And, crucially, because human beings have language, he is able to guide others to construct the new module. Thus people over tens of thousands of years built vast numbers of modules that exploit the structure of the world in new ways. These provide massive numbers of new constraints that continue to allow us to extend. We have thus greatly extended the program of thought. It is our access to this huge additional program that, in my view, separates human beings from other animals.
He is talking about memes and memic processes, and this is a fit example for memetics.
In section 12.4 Baum asks, “What inductive bias did evolution start with?”, referring to the predilection of a system to learn certain kinds of things, or in certain ways. As evolution is based on the manipulation of molecules in three-dimensional space, he suspects Euclidean topology is a strong bias. He contrasts this bias with the bias of programmers attempting to construct intelligent programs to play Go or chess. Rather than incorporating the human appreciation of the two-dimensional topology of the boards, they focus on strings of bits or other symbols; there is no sense that bits representing neighboring stones or threatening pieces are connected to others. Baum doesn’t have specific proposals based on this observation, but it occurred to me that perhaps a generalization of Post’s production systems to 2-, 3- or 4-dimensional objects, rather than 1-dimensional strings of symbols might be worth investigating. Similarly, a tree-like object might provide a connection to grammar-like computation. These ideas might fit with another inductive bias he identifies: real-time performance. The more levels of abstraction that must be spanned in a computation, the longer it will take to perform. Other biases he identifies are causality, and hill-climbing (in an abstract sense) to approach locally optimal solutions. Together, these biases lead to rapid, shallow computations, but with potentially great parallelism; though low-level computations might be shallow, a hierarchical arrangement of modules might organize a broad computational into a relatively small, serial computation at the level of awareness.
In the introduction to chapter 13, he says
Section 13.2 reviews the model of mind proposed in this book and discusses language in this context. Two features are particularly relevant. First, if thought is the execution of a complex program, built as the interaction of many semantically meaningful modules, then words can naturally be seen as labels for modules, and sentences can naturally be seen as desribing how modules are put together into a given computation. I discuss in this context the question of how language interacts with computation. I suggest that the semantics is contained in the actual code and that attaching to the code thus does relatively little directly to facilitate thought. Thus, I suggest that language is descriptive rather than integral to thought. On the other hand, there is the possibility that the advent of sophisticated grammar facilitated or was made possible by a new way of combining modules, such as a new standardization of interface.
Second, I reiterate that the construction of the mental program is a cumulative process, with new computational modules built during life on top of old ones, and that the search for such useful code is computationally hard, taking place on an extremely rough fitness landscape with many local maxima. Progress is thus made in increments, with some new module or change made to the code allowing advance to the next sticking point. These facts taken together suggest that the advance in thinking of humans over monkeys could in principle be explained purely by invoking language for communication rather than ascribing to it a role in the computation itself.
In discussing internal rewards (the mechanisms driving our drives), he says (after talking about orgasm and other rewards)
It seems clear that the universal desire of children for the praise of their parents is built in. This is built in with some distinctions as well, for example, the fictional literature, the psychological literature, and general experience all concur that human sons very much want the admiration of their fathers and are often bitter when they don’t get it. This built-in goal allows the passage of complex behaviors from parents to child. It allows culture to evolve and be passed on, with massive effects on evolutionary fitness, and on our lives.
This instinct for parental approval is not exclusively a human characteristic, for example, bears can’t forage for a particular food unless they are shown how by their parents. The built-in goal of emulating parents and seeking approval of parents, combined possibly with the built-in goal of instructing children, allows complex behaviors like salmon fishing to be passed from generation to generation of bears. Alaskan brown bears and grizzly bears are genetically indistinguishable and live only miles apart but look substantially different. The brown bears are bigger and heavier with huge shoulder muscles because they have been instructed by their parents how to harvest the rich food sources in their coastal environment and thus eat better and behave differently.
Baum tells this bear story in nearly identical words in two places.
In discussing awareness, he mentions “perhaps the most interesting suggestion about awareness is that it has been carefully engineered to be ignorant of facts known to deeper recesses of our minds, for the purpose of making us capable of lying more effectively.” He explains this, then goes on:
So what, then, is awareness? Why do we sense this computation the way we do, with a sensation of consciousness, of being aware and engaged in things? Why should we not sense the rest of what is undoubtedly going on in our minds?
The most straightforward theory is simply this. There is code at the top of this hierarchy, code that controls speech and action, that makes decisions and perhaps feeds back credit assignment. This code does not directly sense all the underlying computation; all it sees is summary bits fed to it by underlying processes. But this upper-level code is what outputs through speech and action. So, when we ask questions of ourselves, when we introspect, when we describe our thought processes to others, when we talk about what we are feeling – all of this is controlled by the upper-level code, the upper-level modules. These upper-level decisions and computations are what we report because the upper-level modules are doing the talking. Indeed, “upper-level” may be a slight misnomer. Speech and action are controlled by modules specifically evolved for controlling speech and action, which may be deliberately fed disinformation by other modules, specifically to control what we say and do in a manner advantageous to our genes. What we are verbally aware of, then, is the disinformation, not the true information only known to subconscious processes that direct the flow of information. So, it is not clear in what sense we can say that our verbal awareness is at the very top of some hierarchy. (Of course, the same could be said of the President of the United States, who although nominally at the pinnacle of the hierarchy of government may be fed disinformation by his subordinates.) … Awareness is simply our ability to talk about our summary of the world and direct our computational abilities against portions of it.
In the epilogue to his chapter on consciousness, Baum says
As I write these words on my laptop, I am sitting on the Kärntnerstrasse, a walking street in the heart of the old city of Vienna. I am sitting in one of a number of small pavilions in the center of the street that serve in the summer as cafés or bars. The sides of this pavilion are open, but there is a sail suspended horizontally overhead to keep off the sun. This particular bar has served me three Caiparinhas, on which they have a special at 40 shillings. They make an excellent Caiparinha, placing the right amount of appropriately coarse sugar in the bottom of the glass before carefully mashing in the limes, adding ground ice then strong dark rum and then more ground ice; as with much food preparation in Vienna, they pay proud attention to detail. The excellent thing about my location in the middle of the Kärntnerstrasse, aside of course from the fine weather and the antique beauty of the street itself, is that a flow of perhaps ten people per minute, many of them beautiful women, passes by the pavilion, and I am enjoying the floor show. Thankfully, the fact that I can intellectually understand that my mind is nothing but an evolved computation does not in any way detract from my enjoyment of life or from my desire to live a fruitful and moral life. That enjoyment and that desire are built in, and I feel them as keenly as I was designed to.