Roger Kerr: Speech - "The Wisdom Of Crowds"

Saturday 2 October 2004

ACT New Zealand Upper South Regional Conference The Wisdom Of Crowds - Roger Kerr Executive Director Christchurch New Zealand Business Roundtable

THE WISDOM OF CROWDS Those of us who are engaged in public affairs may sometimes feel tempted to feel nostalgic for an age – probably a mythical golden age – when informed ‘public opinion’ reigned unchallenged. The nineteenth century sometimes seems like such an age: earnest, educated people read serious journals and wrote literate and thoughtful letters to newspapers that were read by everyone who counted. They shared an unquestioned belief in science, civilisation and progress that inspired effective movements for reform of all kinds. Wouldn’t public policy benefit if we could rediscover their self-confidence, self-discipline and commitment?

Not according to James Surowiecki, a financial columnist for the New Yorker. The disconcerting message of his recent book The Wisdom of Crowds is that consensus among the well-informed is quite likely to be wrong. In many areas, wisdom is dispersed throughout society, often very thinly, but where it can be articulated, it is usually more reliable than the wisdom of elites.

Surowiecki begins his book with an anecdote. It’s about Francis Galton, a British authority on statistics and the science of heredity who was active about a century ago. Galton was interested in heredity because he believed that only a few people had the necessary intelligence and ability to run society. It followed that only if power was concentrated in their hands could society flourish.

In 1906 Galton‘s interest in breeding led him to an agricultural show, where he came across a weight-judging competition. A fat ox was on display and members of the public could place wagers on its weight after it had been slaughtered and dressed. Assuming that the average punter would be hopelessly wrong, Galton borrowed the several hundred betting slips from the organisers and calculated the mean guess. To his surprise, it was 1,197 pounds: just one pound short of the right answer.

Galton had stumbled on what Surowiecki calls a “simple but powerful truth … under the right circumstances, groups are remarkably intelligent, and are often smarter than the smartest people in them … when our imperfect judgements are aggregated in the right way, our collective intelligence is often excellent” (pp xiii–xiv).

It’s worth mentioning that this idea is an old one. Indeed, it was advanced by the ancient Greek philosopher Aristotle in the fourth century BC. In his Politics, Aristotle defended a form of democracy by reference to the ‘wisdom of crowds’:

It is possible that the many, no one of whom taken singly may be a good man, may yet taken all together be better than the few, not individually but collectively … Each individual will be a worse judge than the experts, but when all work together, they are better, or at any rate no worse.

Note that, for Aristotle as much as for Surowiecki, the wisdom of a crowd doesn’t depend on any member of the crowd being wise; indeed, every member of the crowd could be hopelessly foolish. But in a crowd the mistakes are likely to run in all directions and cancel one another out, leaving a sound judgment in the aggregate.

One anecdote does not establish a thesis, but the reader of the entire book is likely to be convinced that Surowiecki is on to something. Here are a couple more examples.

In 1986, the American space shuttle Challenger exploded soon after it was launched, with the loss of all its crew. Within minutes, the stock market reacted: the shares of the four companies that had most to do with the construction of the shuttle had all lost value. But one of them, Thiokol, the manufacturer of the solid-fuel booster rocket, lost more ground than the others. Moreover, the next day, when the shares of the other three companies began to recover, Thiokol’s continued to fall. Six months later, the official enquiry confirmed what the stock market (but no one else) apparently knew from the start: that the seals on the booster rocket were defective and had caused the explosion. An academic study of the unique, almost immediate and – as it happened – well-founded collapse of Thiokol’s stock turned up no evidence to support likely explanations, such as media speculation or Wall Street hype. In the end, the researchers concluded that there must have been some insider dealing, although they couldn’t prove it. But Surowiecki thinks the most likely explanation is that it is another example of the wisdom of crowds.

The other anecdote I’ll mention is closer to home but perhaps more conclusive. In the television quiz show Who Wants to be a Millionaire? contestants stumped by a question can enlist the aid of friends or relatives chosen on the basis that they think they are smarter or better informed than themselves. They can also opt to ask the studio audience. The ‘experts’ turned out to be right 65 percent of the time. But the members of the audience – supplying answers anonymously and independently of one another – were collectively right 91 percent of the time. If you ever appear on that show, you may do better to rely on the crowd than on your smart friends!

Clearly, we are happy to accept the intuitions of Surowiecki’s thesis in fashioning some of our social arrangements. The most obvious in the modern world is democratic politics. We rely on a universal, unweighted one person–one vote franchise to choose our governments. The merits of that form of democracy were by no means so obvious historically; Aristotle, after all, was challenging Plato’s advocacy of rule by a philosopher king. Sometimes, as with Lee Kwan Yew, such rule can be benign, indeed inspired. But the lesson of the twentieth century is that for every Lee Kwan Yew there are a dozen or more Joseph Stalins, Idi Amins and Robert Mugabes.

In choosing a political system, as with other public policy choices, what matters is the performance of alternative arrangements on average and over time. It is unwise to bet against the odds. Democracy is not perfect but we put faith in the wisdom of crowds for choosing our governments on the basis that democracy is better than any alternative. Looking back on the choices of the New Zealand electorate in the past 25 years, I find it hard to argue that that faith has been grossly misplaced.

An immediate objection to Surowiecki’s thesis is that there is plenty of counter-evidence showing the folly of crowds. The book openly acknowledges this evidence. Witch-hunts, the rise of Adolph Hitler, and stock market bubbles could all reasonably be cited as examples of crowds, sometimes entire nations, taking collective leave of their senses. Indeed, the assumption that crowds are more likely to get things wrong than right seems to underlie many of our familiar arrangements and practices. Where knowledge is concerned, we have entrenched divisions between experts and laypeople. While we believe in democracy, we usually stick with the indirect, representative kind: the idea is that we are better to let the decisions be made by an elected elite that has some incentive to become well-informed about issues, and let the people as a whole judge the results at the next election. It follows that, if Surowiecki is right and that “under the right circumstances”, as he puts it, the amateur masses may be wiser than the professional elites, some of those practices and arrangements should be reviewed. Naturally I am interested in the implications of Surowiecki’s thesis for public policy and I consider some of these later on.

Under what conditions, then, are crowds more likely to be wise than foolish? Surowiecki’s thesis naturally depends very greatly on the soundness of this part of his analysis. He says that crowds are likely to be wise when four conditions are met: diversity of opinion (everyone has a bit of private information, however small); independence (people’s opinions are not influenced by those of others); decentralisation (people can specialise and draw on local knowledge); and aggregation (there must be a mechanism for turning individuals’ private judgments into collective decisions).

In the economic sphere we can note at once a familiar example of ‘crowd wisdom’ as Surowiecki defines it: the marketplace. The market is driven by diverse tastes and preferences; consumers normally make private judgments about those tastes and preferences and can act on them; the market, being driven by individual choice, is as decentralised as any institution can possibly be; and consumers’ choices are continually aggregated by the price mechanism.

Today it is uncontroversial to argue that, like democracy, the free market economy outperforms alternative systems. Again this was not always the case. Serious scholars in the twentieth century, not to mention political demagogues, advocated central planning of the economy. Putting resource allocation decisions in the hands of ‘experts’, it was argued, would lead to more growth and prosperity than leaving them to the spontaneous and unpredictable workings of competitive markets.

The collapse of socialist central planning has dispelled this illusion for most. Central planners have neither the information nor the incentives to make good decisions compared with the knowledge and incentives of decentralised economic actors, ie the people at large. Yet we continue to see innumerable examples of government interventions based on the premise that politicians and bureaucratic experts in Wellington can spend people’s money and run their lives better than they can themselves.

A more specific economic illustration of the wisdom of crowds is the operation of financial markets. The prices of financial assets (such as bonds, stocks and currencies) are determined through competitive trading. A great deal of research indicates that these markets are very efficient in the sense that they reflect all publicly available information about the fundamental value of these assets. Unless investors or traders have information that is not publicly available, it is very difficult for them to ‘beat the market’ – to do better than the collective assessment of the vast numbers of people dealing in financial markets every day.

Nevertheless, crowd wisdom is not infallible, and Surowiecki devotes much of his book to showing how the absence of any of his four conditions diminishes it. Diversity is especially important to correct the ‘groupthink’ that can so easily creep up on teams of experts and lead them astray. Experiments have shown that groups of people of mixed ability are often better at solving problems than groups comprising only smart people. The problem seems to be that experts tend to share, but also to exclude, a good deal of knowledge as a result of their training, whereas less smart types may bring useful information that experts have collectively forgotten or never had. (Surowiecki cites by way of examples of failures of ‘expert’ wisdom the persistently poor performance of professional financial advisers and the failure in 2003 of NASA to register fears among some NASA staff about damage to the space shuttle Columbia, which eventually caused the shuttle to break up when it re-entered the earth’s atmosphere.) The point is not that expertise is to be avoided, but that experts can often learn from lay people, so the best decisions are likely to be a joint product of both types – if that can be arranged.

Independence – each member of the crowd should form a judgment without being directly influenced by the other members – is an equally important condition for the wisdom of crowds. Its absence can explain the examples of mass folly we mentioned earlier. A stock market bubble can occur when investors stop making independent judgments and become overly influenced by what other investors think. So much is clear. But independence also tells against consensus. The wisdom of crowds is emphatically not something that emerges from negotiation among the members of a crowd. Surowiecki cites research showing that group discussion tends to make the group and its individual members more extreme in their views than they were originally (perhaps special interest lobbies bear this out).

This idea too has a good pedigree in classical democratic theory. The eighteenth-century philosopher Rousseau argued in The Social Contract that in making decisions the members of the sovereign assembly should have no communication with one another and “each citizen should think only his own thoughts”. Otherwise, Rousseau argued, the assembly would become corrupted with special interests and would lose sight of its shared interests. Indeed, we recognise the political importance of independence of judgment in the secret ballot, which protects each voter from undue pressure from other voters: we want voters to make their own judgment, not one that reflects the influence of the last person they spoke to.

Decentralisation is probably the least controversial condition for the emergence of the wisdom of crowds, since we obviously want every member of the crowd to contribute to its collective judgment, and decentralisation encourages specialisation and diversity. As for aggregation, the last condition, Surowiecki uses a topical example to illustrate its indispensability: the activities of America’s intelligence agencies. As we know, some agents were trying to warn about the possibility, even the likelihood, of an imminent terrorist attack on American soil. But as we also know, the agencies had not worked out a way of bringing together the collective view of their staff, and so the warnings did not register where they should have.

I said earlier that I was interested in the public policy implications of the ‘wisdom of the crowd’ thesis. I’ve already cited the market as an area in which we generally accept that the crowd is wise or, at the very least, less foolish than any other source of judgment. The New Zealand Business Roundtable is, of course, well-known for its advocacy of markets, and of extending market processes into certain areas presently dominated by the state, such as health and education, where evidence of the wisdom of bureaucratic elites is indeed hard to come by. These sectors, which are vital to economic and social progress, remain largely nationalised and centrally planned; we are still operating, so to speak, behind the Berlin Wall. In education, for example, school zoning, an extensive national curriculum, teacher employment arrangements and the absence of a level playing field with the private sector heavily constrain decentralised decisions by individual parents. Not surprisingly, the frustrations of parents, teachers and the community at large run high. Worldwide trends towards school autonomy, parental choice and a greater role for the private sector suggest that the days of the Berlin Wall are numbered. The shape of the system would then be determined less by education planners and more by the cumulative decisions of individual families.

But the more original public policy implications of Surowiecki’s thesis are for our democratic political system. Representative democracy, through the universal and secret ballot, is, on the analysis offered in the book, a way of eliciting the wisdom of crowds in making collective decisions. But it is not the only way. Another, which Surowiecki does not explore, is to make more use of direct democracy, that is, referenda.

One of the arguments for representative or indirect democracy, as I mentioned earlier, is that our elected representatives are in a position to become informed about issues and that the general public’s role should therefore be confined to judging a government’s overall performance. However, this argument is open to Surowiecki’s objection that groups of experts are liable to become impervious to relevant information that non-experts can supply. Governments are, of course, far from insulated from public opinion. But one of the causes of the malaise that infects Western democracy is the rise of politics as a profession, as a result of which politicians as a whole, whatever party they belong to, seem to have more in common with one another than with their electorates, and, along with their advisers and top public servants, can with some justice be said to constitute a ‘political class’ that is vulnerable to groupthink. That is to say, representative assemblies are becoming less truly representative as they lose diversity. Referenda are a way of counteracting that tendency and getting more diversity and hence more relevant information into the political system. They also enhance decentralisation by allowing members of the public to express views on particular issues, which they cannot easily do in general elections when they typically vote for a political party or coalition.

Thus, contrary to the 1986 Royal Commission on the electoral system, I see merit in greater use of referenda for making those decisions that have to be made collectively. It is surely high time, for example, that the country was given another say on whether to retain the mixed member proportional (MMP) electoral system. Even the prime minister has said that should happen, but has to date argued that it is too soon to have another referendum. That argument is becoming less and less credible. In my view the nuclear ship visit issue is one that would be well-suited to a referendum; I, for one, would be happy to accept ‘crowd wisdom’ in weighing up competing claims. And in a report which the Business Roundtable is releasing this month, the case is made for amending the Fiscal Responsibility Act 1994 to require voter approval via referenda for new taxes or tax rate increases.

An intriguing proposal that Surowiecki discusses is the possibility of a policy analysis market. This idea seems to have grown out of the Iowa Electronic Markets (IEM) project that was set up in 1988 as a competitive alternative to opinion polls. The IEM is simply a market for betting on the outcome of elections: any member of the public can “buy and sell futures ‘contracts’ based on how they think a given candidate will do in an upcoming election” (p 17). The interesting thing is that between 1988 and 2000 the IEM generally outperformed opinion polls, even though only a few hundred people participated in them. Note the difference between a betting market and an opinion poll. In the betting market, people are making bets on how other people will vote and backing their judgment with their own money. In a poll, people say how they themselves intend to vote, but there are no sanctions if they don’t give a truthful answer or if they change their minds later. The incentive to get a forecast right is obviously much greater in a betting market.

Some private corporations have tried to tap the wisdom of their employees by setting up ‘decision markets’, based on the same principle as the IEM, on the likely success of their new products. Those cited by Surowiecki – Hewlett-Packard’s on new printer sales, and Innocentive’s on which new drugs were likely to win approval from the US Food and Drug Administration – have been remarkably accurate. How much easier for humble employees to contribute their knowledge, and to gain from doing so, through an anonymous market rather than through some consultation committee, with its built-in biases, managers unwilling to register inconvenient information, and employees afraid of the consequences of speaking up.

A leading advocate of decision markets is Robin Hanson, of George Mason University in the United States. In an article written in 1999, he says: … we suffer from a serious failure to share information. The so-called Information Revolution has greatly improved our ability to find out what others have said. However, it has done much less to improve our ability to find out what other people know … Speculative markets are a neglected way to help us find out what people know. Such markets … have many advantages over standard institutions for information aggregation, such as news media, peer review, trials, and opinion polls. Speculative markets are decentralised and relatively egalitarian, and can offer direct, concise, timely, and precise estimates in answer to questions we pose. Just as self-interest motivates us to influence the market price of a good by buying it or not buying it, so the gains from betting can motivate us to contribute accurate and relevant information to a collective decision, and more reliably so than through mechanisms where no such incentives exist.

How then would a policy analysis market work? The term comes from one such market that was set up by the Pentagon in 2003 to provide intelligence on the Middle East. The general public would be invited to make bets on possible events in the region, with a view to allowing policy-makers to bypass the vested interests of intelligence agencies in their pet theories and analyses and get some new, informal but reliable information into the policy-making arena. But it was not to be. Congressional pressure forced the Pentagon to scrap the idea on the grounds that it was morally wrong to encourage people to profit from bets on terrorist outrages, even though it was not on that sort of question that the policy market was expected to focus. Yet would we not approve if a government agent profited in career terms as a result of accurately forecasting the attacks of 9/11? (Surowiecki, who is naturally a fan of policy analysis markets, passes on the good news that in March 2004 NetExchange, the company that set up the market for the Pentagon, announced that it would open a new one, without government involvement, devoted to forecasting broad economic and military trends in the Middle East. Surowiecki also cautions that the events wagered on would have to be appropriately framed to allow a reliable market to emerge. They would have to deal with the problem of a wager becoming self-defeating; for example, a forecast assassination could be successfully anticipated and forestalled. But that would only confirm the reliability and indeed the very usefulness of such markets.)

It’s interesting to speculate on how policy analysis markets could be developed to cover the entire field of policy. A private organisation – a think tank, a newspaper, a television or radio company, even a public opinion polling company – could set up a betting market on, say, whether our current Labour-led government will, on present policies, achieve its ‘top priority’ objective of returning New Zealand to the top half of the Organisation for Economic Cooperation and Development (OECD) income rankings by some specified date. The qualification ‘on present policies’ would be necessary because the government could react to a betting market forecast of failure by changing its policies to ones more likely to succeed. And that is exactly how such a market could help improve public policy. It could be expected to be more reliable, and hence more heeded, than opinion polls, which are vulnerable to politicians’ growing skill in manipulating voter apathy, ignorance, and cynicism. I’m tempted to keep speculating in this vein. What, I wonder, would a betting market have forecast on whether the Coalition that invaded Iraq would find weapons of mass destruction, and with what effect? But rather than continue along these lines, let me sum up.

We live in an age of mass higher education; every topic one can think of can be studied somewhere for a qualification, so that it has its official ‘experts’. One thing I’ve discovered is that people with educational qualifications certainly tend to believe that they know more than laypersons, and they also tend not to be backward in promoting new legislation that embodies their expertise, regardless of the cost. So whatever its benefits, expertise needs to be accountable. The best mechanism for making it accountable is the market, where the consumer is sovereign but also ensures that any benefits of expertise are recognised and rewarded.

Where markets are not workable, as with pure public goods, the elitism of ordinary politics could benefit from the additional accountability provided by the referendum. And betting markets in policy areas could amount to a sort of continuous popular referendum – a more reliable and stable one than that offered by volatile opinion polls, as well as being a lot of fun.

ENDS

© Scoop Media