2006 Woodrow Wilson Foundation Award

For the best book published in the U.S. during the previous calendar year on government, politics, or international affairs. The award is supported by the Woodrow Wilson Foundation.

Award Committee: Michael C. Dawson, University of Chicago, Chair; Miriam A. Golden, University of California, Los Angeles; Ian S. Lustick, University of Pennsylvania

Recipient: Philip Tetlock, University of California, Berkeley

Title: Expert Political Judgment: How Good Is It? How Can We Know? (Princeton University Press, 2005)

Citation: Given the enormous variety of work in our discipline, the annual task of selecting a recipient for the Woodrow Wilson Foundation Award for the best book on government, politics, or international affairs is a daunting challenge and one that demands a certain degree of intellectual hubris. We can safely say that this year the task was particularly difficult, not only because of tough competition among several terrific books, but because the best book among them makes a devastatingly effective argument against the exercise of hubristic judgment. In Expert Political Judgment Philip E. Tetlock argues compellingly that experts who think they know the right answer to a tough question in their fields of expertise, especially those who are very confident in that answer, are only slightly more likely to be correct in that judgment than dart-throwing chimpanzees.

Tetlock's book is a fascinating, powerful, sustained, and slightly unnerving study of how much of a contribution expertise makes to the quality of political predictions. Based on nearly two decades of repeated carefully designed studies of the predictive capacities of experts about important, real world political phenomena, Tetlock concludes that experts, as a group, seldom out perform statistical extrapolation, and are usually little better than mindless algorithms in predicting what will happen in the political domain. Expertise, in short, is not worth much at all as measured by the ability of highly trained and self-acknowledged experts to predict outcomes on issues as diverse as the likely direction of events in India after the victory of a Hindutva government, the future of Russia under Yeltsin and his successors, the stability and territorial integrity of Kazakhstan, the effect of economic shock therapy for Poland, or prospects for continued rule by the Saudi royal family.

Yet, Tetlock also shows that expertise is not irrelevant if we seek to understand why and how we are wrong. Exhibiting an inspiring mastery of statistical methods integrated within a strongly and often humorous narrative, Tetlock deploys Isaiah Berlin's classic distinction between "foxes" and "hedgehogs" to powerful effect.  Hedgehogs are confident in their own ideas, have strong theoretical frameworks with which to understand the world, and are resistant to updating their beliefs even in the face of discouragingly wrong predictions. Foxes, by contrast, are often uncertain and cautious in the predictions they make, think of theoretical explanations as useful guides but not infallible intellectual architectures, and are flexible in their belief systems. Tetlock shows that for most forecasting tasks and in most analytic situations fox-type thinking outperforms hedgehog thinking, and that these differences in intellectual style are more important, per se, than sheer expertise. How we think about politics, how aware we are of our proneness to error, and how impressed we are by the complexity of the political world, are together more important to our abilities to predict outcomes than having amassed specialized degrees, decades of experience, and even public notoriety.

Rarely does a book whose contribution is squarely methodological speak so directly to all of the subfields of political science. This is an exciting study, one that should cause reflection and reconsideration throughout our discipline. The issues it raises touch on the degree of attention we should give to punditry in general and to the role of the media in promoting precisely the kind of thinking about the future most likely to be wrong.  Thus the media, notes Tetlock, actively searches for and rewards talking hedgehog heads, confident they can be relied on to provide snappy and definitive sound bites that will track consistently with desired points of view. In his attention to the deeper epistemological and methodological questions associated with counterfactual thinking, statistical models, extrapolation of case studies, belief updating, scenario writing, analogical reasoning, and his own "neo-positivist" project of testing specific predictions for accuracy, Tetlock subjects his own enterprise to the self-scrutiny that comes naturally to his fox-like style.
   
Tetlock causes us to reflect in the deepest ways on how we think and on the social and collective purposes of our shared disciplinary enterprise. Indeed if there is any source of reassurance that expertise ? passionately, systematically, and self-critically applied ? can enhance our understanding of the political world, Tetlock has provided it by example in this book.  No one who reads Expert Political Judgment will come away untouched, either professionally or personally.