Introduction: Ignorance and the Community of Knowledge
5 “The human mind is not like a desktop computer, designed to hold reams of information. The mind is a flexible problem solver that evolved to extract only the most useful information to guide decisions in new situations. As a consequence, individuals store very little detailed information about the world in their heads. In that sense, people are like bees and society a beehive: Our intelligence resides not in individual brains but in the collective mind. To function, individuals rely not only on knowledge stored within our skulls but also on knowledge stored elsewhere: in our bodies, in the environment, and especially in other people. When you put it all together, human thought is incredibly impressive. But it is a product of a community, not of any individual alone.” !!!
8 “We all suffer, to a greater or lesser extent, from an illusion of understanding, an illusion that we understand how things work when in fact our understanding is meager.”
11-13 “humans specialize in reasoning about how the world works, about causality. Predicting the effects of action requires reasoning about how causes produce effects, and figuring out why something happened requires reasoning about which causes are likely to have produced an effect. […]
This makes human ignorance all the more surprising. If causality is so critical to selecting the best actions, why do individuals have so little detailed knowledge about how the world works? It’s because thought is masterful at extracting only what it needs and filtering out everything else. When you hear a sentence uttered, your speech recognition system goes to work extracting the gist, the underlying meaning of the utterance, and forgetting the specific words. When you encounter a complicated causal system, you similarly extract the gist and forget the details. […]
The mind is not built to acquire details about every individual object or situation. We learn from experience so that we can generalize to new objects and situations. The ability to act in a new context requires understanding only the deep regularities in the way the world works, not the superficial details. […]
The secret to our success is that we live in a world in which knowledge is all around us. It is in the things we make, in our bodies and workspaces, and in other people. We live in a community of knowledge.” !!
14-15 “Our skulls may delimit the frontier of our brains, but they do not delimit the frontier of our knowledge. The mind stretches beyond the brain to include the body, the environment, and people other than oneself, so the study of the mind cannot be reduced to the study of the brain. Cognitive science is not the same as neuroscience.” !!!
15 “The nature of thought is to seamlessly draw on knowledge wherever it can be found, inside and outside of our own heads. We live under the knowledge illusion because we fail to draw an accurate line between what is inside and outside our heads. And we fail because there is no sharp line. So we frequently don’t know what we don’t know.” !!
ONE: What We Know
35 “One way to think about the illusion of explanatory depth is that adults forget how complex things are and decide to just stop asking questions. Because we are not conscious that we have made this decision to stop probing, we end up thinking we understand how things work more deeply than we do.” !!
FOUR: Why We Think What Isn’t So
83 “Intuition gives us a simplified, coarse, and usually good enough analysis, and this gives us the illusion that we know a fair amount. But when we deliberate, we come to appreciate how complex things actually are, and this reveals to us how little we actually know.” !!
FIVE: Thinking with Our Bodies and the World
95 “the visual environment functions as a sort of outside memory store.” !!
105 “the brain is in the mind. The mind uses the brain and other things to process information.” !!
SIX: Thinking with Other People
111 “[The mind] did not evolve in the context of individuals sitting alone solving problems. It evolved in the context of group collaboration, and our thinking evolved interdependently, to operate in conjunction with the thinking of others. Much like a beehive, when each individual is master of a domain, the group intelligence that emerges is more than the sum of its parts.” !!
115 “Humans have an ability that no other machine or animal cognitive system does: Humans can share their attention with someone else. When humans interact with one another, they do not merely experience the same event; they also know they are experiencing the same event. And this knowledge that they are sharing their attention changes more than the nature of the experience; it also changes what they do and what they’re able to accomplish in conjunction with others.
Sharing attention is a crucial step on the road to being a full collaborator in a group sharing cognitive labor, in a community of knowledge. Once we can share attention, we can do something even more impressive—we can share common ground. We know some things that we know others know, and we know that they know that we know (and of course we know that they know that we know that they know, etc.). The knowledge is not just distributed; it is shared. Once knowledge is shared in this way, we can share intentionality; we can jointly pursue a common goal. A basic human talent is to share
intentions with others so that we accomplish things collaboratively.” !!!!
117 MICHAEL TOMASELLO, et al “[C]hildren, but not chimpanzees, often seemed to collaborate just for the sake of collaborating.”
117-118 “The ability to share intentionality supports perhaps the most important human capability of all: the ability to store and transmit knowledge from one generation to the next. This leads to what anthropologists call cumulative culture. […] Human capabilities are constantly increasing, but not because individuals are getting smarter. Unlike beehives, which have operated pretty much the same way for millions of years, our shared pursuits are always growing more complex and our shared intelligence more powerful.” !!
126 “The different bits of knowledge that different members of the community have must be compatible. We may not all always agree entirely, and in many cases we don’t, but we have to at least be thinking about related things or the division of cognitive labor would fall apart. […] our knowledge of things has to be structured so that the knowledge we expect others to fill in has an appropriate place to go.” !!!
SEVEN: Thinking with Technology
141-142 “Our dependence on smart technology has led to a paradox. As the technology improves, it becomes more reliable and more efficient. And because it’s reliable and efficient, human operators start to depend on it even more. Eventually they lose focus, become distracted, and check out, leaving the system to run on its own. In the most extreme case, piloting a massive airliner could become a passive occupation, like watching TV. This is fine until something unexpected happens. The unexpected reveals the value of human beings; what we bring to the table is the flexibility to handle new situations. Machines aren’t collaborating in pursuit of a joint goal; they are merely serving as tools. So when the human operator gives up oversight, the system is more likely to have a serious accident.
The automation paradox is that the very effectiveness of automated safety systems leads to a dependence on them, and that this dependence undermines the contribution of the human operator, leading to greater danger.” !!!
EIGHT: Thinking About Science
162 “Our beliefs are not our own. They are shared with our community. And this makes them really hard to change.” !!
163 “Attempts to foster science literacy cannot be effective if they don’t either change the consensus of the community or associate the learner with a different community.” !!
170 “New information that runs counter to our causal models is hard to absorb and easy to dismiss, especially when it contradicts the positions advocated by the people we trust. But it’s harder to dismiss the discovery that one doesn’t understand the mechanisms at play. […] The first step to correcting false beliefs is opening people’s minds to the idea that they and their community might have the science wrong. No one wants to be wrong.” !!
NINE: Thinking About Politics
172 BERTRAND RUSSELL “The opinions that are held with passion are always those for which no good ground exists.” !!
177-178 “We found that attempting to explain how a policy worked not only reduced our participants’ sense of understanding, it also reduced the extremity of their position. If we consider the whole group together, the fact that people were on average less extreme means that the group as a whole was less polarized after the explanation exercise. The attempt to explain caused their positions to converge. […]
But other studies that have asked people to think about positions have actually made people more extreme, not less, presumably for the same reason that having people discuss their position in a group makes them more extreme. Usually when people think about their position on an issue, they recollect why they believe what they do and they generate arguments in favor of the position they already have. They don’t engage in causal explanations about how the policy would lead to good or bad outcomes.
These are very different forms of thinking. Usually when people think about and talk about policies, they are not engaged in causal explanation. Most discourse about policy is about why we believe what we do: who agrees with us, why we hold whatever value the policy addresses, what we heard about it on the news the other day. […]
The beauty of causal explanation is that it takes explainers outside of their own belief systems.” !!!
179-184 “Getting people to think beyond their own interests and experiences may be necessary for reducing their hubris and thereby reducing polarization. Causal explanation may be the only form of thinking that will shatter the illusion of explanatory depth and change people’s attitudes. […]
But if people’s positions are not consequentialist but based on sacred values, then shattering the illusion won’t matter. […]
So our argument that causal explanation is an easy and effective way to moderate opinion applies only to certain issues, issues that elicit opinions based on outcomes as opposed to opinions based on values. That leaves us with plenty of issues—most opinions are driven by a consideration of outcomes. Opinions about everything from whether society should support nuclear power to education and health care are, for most people, a matter of how to achieve the best outcomes.” !!
184 “Proponents of political positions often cast policies that most people see as consequentialist in values-based terms in order to hide their ignorance, prevent moderation of opinion, and block compromise.” !!
185 “So why do politicians and interest groups so often take a sacred values position rather than thinking through the causal consequences of various policies? The most obvious answer is obfuscation: The policy preference that will earn them votes or money is not what a consequentialist analysis dictates, so they avoid the consequentialist analysis. The other answer is that thinking through the consequences of a policy is hard—very hard. It’s much easier just to hide one’s ignorance in a veil of platitudes about sacred values. It’s an old politician’s ploy. The secret that people who are practiced in the art of persuasion have learned over millennia is that when an attitude is based on a sacred value, consequences don’t matter.” !!
TEN: The New Definition of Smart
196-197 “The lionization of individuals, as well as our corresponding failure to appreciate the role of the communities they represent, is more than just a ruse to simplify complex histories. The images we retain of these individuals shape how we think about the events they participated in. Each leader has become a symbol for their respective movement and—in the popular imagination—each has come to be the movement. We say things like “MLK changed the face of America when he persuaded Congress to pass civil rights legislation” or “If it weren’t for Gandhi, the queen would still reign over India.” Statements like these are more than just figures of speech. The vast majority of people know little enough about the civil rights movement and about Indian independence that their understanding of each consists of not much more than the knowledge that these individuals made a huge difference. Cognitively speaking, the individual has become the movement and gets lone credit for complicated historical events involving millions of others.” HUBABA
210 “To predict a group’s performance, you need to look at the group. Individual intelligence scores are not that helpful.” !!
210 “indicators of group cohesion, motivation, and satisfaction did not predict how well groups did. Other measures did: social sensitivity, how often groups took turns, and the proportion of females in the group. Their data suggest that having more females helps a group because it makes it more socially sensitive” !!
213-214 “Every farmer knows that the hard part is getting the field prepared. Inserting seeds and watching them grow is easy. In the case of science and industry, the community prepares the field, yet society tends to give all the credit to the individual who happens to plant a successful seed. Planting a seed does not necessarily require overwhelming intelligence; creating an environment that allows seeds to prosper does. We need to give more credit to the community in science, politics, business, and daily life.” !!! HUBABA
ELEVEN: Making People Smart
220 “A real education includes learning that you don’t know certain things (a lot of things). Instead of looking in at the knowledge you do have, you learn to look out at the knowledge you don’t have. To do this, you have to let go of some hubris; you have to accept that you don’t know what you don’t know.” !!
224 “Everyone’s understanding—that of scientists and nonscientists alike—is dependent on what others know, so it is more important for students to understand what is known and what can be justified by others than to know the facts and the justifications themselves. […] Much of what scientists hold true is a matter of faith—not faith in a supreme being, but faith that others are telling the truth. What distinguishes this faith from religious faith is that there is a higher power to appeal to: namely, the power of verification. Scientific claims can be checked. If scientists are not telling the truth about a result or if they make a mistake, eventually they are likely to be found out because, if the issue is important enough, someone will try and fail to replicate their result.” !!
TWELVE: Making Smarter Decisions
249 “The big lesson of the nudge approach is that it is easier and more effective to change the environment than it is to change the person. And once we understand what quirks of cognition drive behavior, we can design the environment so that those quirks help us instead of hurt us. […] We have to appreciate that people are explanation foes—that we usually don’t have the inclination or even the capability to master the details of all our decisions. What we can do is try to structure the environment to help ourselves make good decisions despite our lack of understanding.” !!
CONCLUSION: Appraising Ignorance and Illusion
259 “Some Eastern philosophies encourage adherents to appreciate their own ignorance: to accept that they know little and to respect what others know. Indeed, some traditions go further, encouraging people to have gratitude for the knowledge of others. We take this as a lesson of cognitive science too. We can learn and conceive only a finite amount as individuals; to achieve greater things we need a community. In the most fundamental way—in terms of how we think—we’re all in it together.
Intelligence resides in the community and not in any individual. So decision-making procedures that elicit the wisdom of the community are likely to produce better outcomes than procedures that depend on the relative ignorance of lone individuals. A strong leader is one who knows how to inspire a community and take advantage of the knowledge within it, and who can delegate responsibility to those with the most expertise.” !!!
263 “You don’t have to have an illusion to be a team player, but having the illusion is a sign that you are.” !!
263 “Many great human achievements are underwritten by false belief in one’s own understanding. In that sense, the illusion may have been necessary for the development of human civilization.” !!