The Wisdom of Crowds

It’s possible that this book doesn’t belong here. I found it immensely useful for understanding group behavior, what makes a group wise, and what makes a group dumb. I read it because I felt it might help me better gauge in what ways the LDS church is wise, and in what ways it’s dumb. I wrote up this review for my personal use, but will post it here.

This book is about the potential wisdom and potential stupidity of groups. Both large and small groups can be formed, sampled, and utilized that can produce remarkably intelligent decisions and solve many complex problems.

Surowiecki breaks down group decision making into three main categories:

1. Cognition – thinking and information processing, such as a market determining the price of a certain good.

2. Coordination – coordinating movements within groups, such as traffic flow, pedestrian activities in New York, or in a school.

3. Cooperation – formation of a decentralized group independent of any authority or control system to accomplish a task.

As stated, some groups are wise, and some are dumb. Surowiecki has identified four elements required to form a wise group:

1. Diversity of opinion – this is really cognitive diversity, not cultural. Each person in the group needs to have access to, and accurately assess that private information. Problems in this vein often occur when groupthink prevails – a condition indicative of a homogeneous group.

2. Independence – people need to have independent opinions, that is, not determinined by the opinions of others. Problems of herding become manifest when this is not followed closely.

3. Decentralization – people need to specialize and draw on local knowledge.

4. Aggregatio – there must be a mechanism in place to turning the private judgments of the few into a collective decision.

In contrast, Surowiecki goes to great lengths to indicate what causes a failure of a group, or what makes a crowd dumb:

1. Too homogeneous – groupthink prevails.

2. Too centralized – doesn’t draw on local knowledge available from those outside the central bureaucracy.

3. Too divided – lack of communication between private individuals or entities working on a similar goal.

4. Too imitative – information cascades; happen when people observe other people making a decision and make the same decision assuming that because others do it, it must be smart. The problem here is that it does not utilize private information. Usually two key elements of information cascades prevail: decisions with subsequent actors observing decisions (not information) of previous actors, and a limited action space (e.g. an adopt/reject decision).

5. Too emotional – emotional factors like peer pressure, herd instincts, group hysteria, etc. can destroy the wisdom of a group through irrational actions.

(NOTE: this first part of the review is largely taken from the Wikipedia article on this book found here: (2009) http://en.wikipedia.org/wiki/The_Wisdom_of_Crowds)

For me, this book opened my eyes to some things that I already knew, but hadn’t applied to life and group situations. In electrical engineering we are often interested in making the optimal decision in the face of great uncertainty from a multitude of various sensors. None of these sensors are accurate by themselves (the experts) and hence cannot be trusted implicitly. But when they are filtered using advanced stochastic processing techniques (encompassing essentially all the elements spoken of above, but in a mathematical sense) the decision made collectively by incorporating all the information from the sensors appropriately is superior. Some of this is incorporated into information theory. On a personal level we are often too willing to throw out an idea, thought, group, or piece of writing if some parts of it do not ring true. This results in us missing out on the useful information (even if very small) contained therein. From a mathematical standpoint there are advanced techniques to perform this form of inductive reasoning (Bayesian inference) that can be shown to mathematically sound. Unfortunately, this idea is not easily applied to non-mathematical decision making problems. I think it is a life long process to learn to do this on a personal level.

On a group level however, Surowiecki shows that groups can indeed be very intelligent, often constructing, in effect, a Bayesian inference problem and solving it. This works very well as long as the group maintains the characteristics of a wise group. From my observation, particularly in small groups, it is extremely easy to allow groupthink, herding, and other group psychology problems to creep in. They are subtle, and are usually only identifiable to those outside the group. We must be constantly vigilant to heed the observations of outsiders and consider whether or not our group is engaged in groupthink. On a larger group level, many group psychology problems are not as prevalent and the group will, on average, perform better. This is why free markets, and democracy produce stunning results.

For me personally, this book helped me identify problems I see in groups I have been in, particularly within the LDS church. It helped strengthen my testimony in the power of free markets and democracy. Possibly most importantly, it gave me confidence that my small voice, when aggregated with everyone else’s, provides an important piece of a complex decision making process. I am resolved that my experiences, thoughts, ideas, readings, and discussions, however insignificant, are nonetheless critical to the wisdom of the various groups I am in. I am less inclined to try to persuade people to my point of view, as it destroys other people’s private information and has the potential to create group psychology problems. I am more comfortable living within groups I don’t agree with as it, on the whole, creates a stronger, wiser crowd. While I highly value open-mindedness, learning, and flexibility to new, heterodox ideas, it is important to maintain some sort of firm-ish opinion that can be contributed to group’s wisdom. It is certainly a balance.

I thought the book was well-written, easy to understand, and I thoroughly enjoyed his style. At some points, during the discussions about various businesses, I confess I got a little bored. This is mostly a reflection on my lack on interest in the topic. Also, at various points along the way I wondered if it was ever possible to have all the elements of a “wise crowd.” Nevertheless, large groups particularly, are very resilient to many of the problems discussed, and often perform very well, very consistently. Several examples were given citing studies showing that even crowds with severe biases, bad information, and other problems can still make amazingly intelligent decisions, and coordinate their behavior appropriately.