Part I: Individual Rethinking Tetlock first discusses arguments about whether the world is too complex for people to find the tools to understand political phenomena, let alone predict the future. Philip Tetlock - Co-Founder - Good Judgment, Inc. | LinkedIn 5 Jun. You get to pick the reasons you find most compelling, and you come away with a real sense of ownership over them.. We dont just hesitate to rethink our answers. The antidote is to complexify by showing the range of views for a given topic. Question Certainty - Harvard Business Review We dont have to stay tethered to old images of where we want to go or who we want to be. Insights and interesting reads delivered straight to your inbox. Tetlock first discusses arguments about whether the world is too complex for people to find the tools to understand political phenomena, let alone predict the future. Skeptics are those who dont believe everything they hear. Part IV: Conclusion He asked the man How can you hate me when you dont even know me? The men became friends and the KKK member eventually renounced his membership. Rather than try to see things from someone elses point of view, talk to those people and learn directly from them. American Association for the Advancement of Science, International Society of Political Psychology, Intelligence Advanced Research Projects Activity, "Forecasting tournaments: Tools for increasing transparency and the quality of debate", "Identifying and Cultivating "Superforecasters" as a Method of Improving Probabilistic Predictions", "The Psychology of Intelligence Analysis: Drivers of Prediction Accuracy in World Politics", "Accounting for the effects of accountability", "Accountability and ideology: When left looks right and right looks left", "Cognitive biases and organizational correctives: Do both disease and cure depend on the ideological beholder? Debate topic: Should preschools be subsidized by the government? or "How likely is the head of state of Venezuela to resign by a target date?" Being persuaded is defeat. The attack on Osama bin Ladens compound employed red teams and statistical risk assessments before the operation; whereas, the battle of the Bay of Pigs was undone by a failure to employ targeted questioning.5, When the scientist tells you he does not know the answer, he is an ignorant man. Tetlock has received awards from scientific societies and foundations, including the American Psychological Association, American Political Science Association, American Association for the Advancement of Science, International Society of Political Psychology, American Academy of Arts and Sciences, the National Academy of Sciences and the MacArthur, Sage, Grawemeyer, and Carnegie Foundations. Philip E. Tetlock (born 1954) is a Canadian-American political science writer, and is currently the Annenberg University Professor at the University of Pennsylvania, where he is cross-appointed at the Wharton School and the School of Arts and Sciences. July 2011: What's Wrong with Expert Predictions | Cato Unbound It's also the question that Philip Tetlock, a psychologist at the University of Pennsylvania and a co-author of " Superforecasting: The Art and Science of Prediction ," has dedicated his career. Dont try to politic a prosecutor, and be very careful if prosecuting a popular politician. Tetlock is a psychologisthe teaches at Berkeleyand his conclusions are based on a long-term study that he began twenty years ago. Focusing on results might be good for short-term performance, but it can be an obstacle to long-term learning.. caps on vehicle emissions). Its not a matter of having low self-confidence. We dont know what might motivate someone else to change, but were generally eager to find out., Gentle recommendations that allow the other person to maintain agency are offered like: Here are a few things that have helped medo you think any of them might work for you?. Performance accountability evaluates projects, individuals and teams based on outcomes. the usefulness of hypothetical-society experiments in disentangling fact and value judgments of the impact of competing policy proposals. The Expert Political Judgements study was run over 20 years in which Tetlock asked a group of pundits to rate three possible outcomes for a political or . The fundamental message: think. He argues that most political psychologists tacitly assume that, relative to political science, psychology is the more basic discipline in their hybrid field. Being aware of these can dramatically change the approach we take for ourselves and our audience. Philip E. Tetlock | Penn Integrates Knowledge Professorships Among the more surprising findings from the tournament were: These and other findings are laid out in particularly accessible form in the Tetlock and Gardner (2015) book on "Superforecasting." In real-world debates over distributive justice, however, Tetlock argues it is virtually impossible to disentangle the factual assumptions that people are making about human beings from the value judgments people are making about end-state goals, such as equality and efficiency. By identifying the attributes shared by successful forecasters and the methodologies that allow for accurate forecasting, Tetlock and his team at Good Judgment are able to help companies promote these skills among their employees. The spotlight here is on a fundamental question in political theory: who should get what from whom, when, how, and why? They give examples of successful and unsuccessful decision-making processes, none more diametrically opposed as two US Army missions. Rethinking is not only an individual skill, its also an organizational one. Here, Philip E. Tetlock explores what constitutes good judgment in predicting future events, and looks at why experts are often wrong in their forecasts. The book mentions how experts are often no better at making predictions than most other people, and how when they are wrong, they are rarely held accountable. Harish uses a powerful combination of techniquescommon understandings, non-judgmental questions, flexible thinkingto win over some in the audience. Psychologically unsafe settings hide errors to avoid penalties. He has published over 200 articles in peer-reviewed journals and has edited or written ten books.[1]. They argue that tournaments are ways of signaling that an organization is committed to playing a pure accuracy game and generating probability estimates that are as accurate as possible (and not tilting estimates to avoid the most recent "mistake"). Staw & A. Once I'd gotten that framework into my head, I couldn't let it go. 1988-1995 Director, Institute of Personality and Social Research, University of California, Berkeley. [14] In a 2009 essay, Tetlock argues that much is still unknown about how psychologically deep the effects of accountability runfor instance, whether it is or is not possible to check automatic or implicit association-based biases,[15] a topic with legal implications for companies in employment discrimination class actions. . Make a list of conditions under which you would change your mind. Expert Political Judgment: How Good Is It? Great listeners are more interested in making their audiences feel smart., Part III: Collective Rethinking 9 Since then, Tetlock has taught courses in Superforecasting by Philip Tetlock, Dan Gardner | Waterstones Moore, D., Tetlock, P.E., Tanlu, L., &Bazerman, M. (2006). How Can We Know? Are you more Preacher, Prosecutor or Politician? - Command+F 2021 Philip E. Tetlock (born 1954) is a Canadian-American political science writer, and is currently the Annenberg University Professor at the University of Pennsylvania, where he is cross-appointed at the Wharton School and the School of Arts and Sciences. Tetlock, P. E. (2011). Taboo Cognition and Sacred Values BACK TO TOP Defining and Assessing Good Judgment My 2005 book, Expert Political Judgment: How Good Is It? Released in 2015, it was aNew York TimesBestseller and brought this concept into the mainstream by making it accessible to behavioral economists and the general population alike. [20][21][22][23] Real-world implications of this claim are explored largely in business-school journals such as the Journal of Consumer Research, California Management Review, and Journal of Consumer Psychology. Deniers reject anything from the other side. They revert to preacher, prosecutor, and politician modes. Since its original publication, Expert Political Judgment by New York Times bestselling author Philip Tetlock has established itself as a contemporary classic in the literature on evaluating expert opinion. We want to think of this idea when leading, when following, when making sales, when planning our marketing, and anywhere else we are dealing with the thoughts, opinions, and values of others. The author continuously refutes this idea. Tetlock, P.E. Home; About. When our 'sacred' beliefs are in jeopardy, we 'deliver sermons' to protect and promote our ideals. A vaccine whisperer is called in. He struck up a conversation with a white man who was a member of the Ku Klux Klan. Tetlock, R.N. Im a fan of Adam Granthes a good writer and fun to readbut Think Again isnt his best effort (I much preferred Give and Take and Originals). Even a single idea can curb overconfidence. Enjoyed the inclusion of visuals: humorous cartoons, diagrams, and charts. It looks like the CFO was in Prosecutor mode - calling out the flaws in your reasoning, marshalling arguments to prove you wrong and win her case. Dont persecute a preacher in front of their own congregation. Make your next conversation a better one. Academy of Management Review 31 (2006):10-29. Ted's Bio; Fact Sheet; Hoja Informativa Del Ted Fund; Ted Fund Board 2021-22; 2021 Ted Fund Donors; Ted Fund Donors Over the Years. So argues Wharton professor Adam Grant in a fascinating new interview. Task conflict: Arguments over specific ideas and opinions (e.g. Philip E. Tetlock is the Annenberg University Professor at the University of Pennsylvania and holds appointments in the psychology and political science departments and the Wharton School of Business. Philip Tetlock of the University of Pennsylvania and author of Superforecasting talks with EconTalk host Russ Roberts about his work on assessing probabilities with teams of thoughtful a Show EconTalk, Ep Philip Tetlock on Superforecasting - Dec 20, 2015 29). This results in more extreme beliefs. Fuzzy thinking can never be proven wrong. Follow Philip Tetlock to get new release emails from Audible and Amazon. Tetlock first discusses arguments about whether the world is too complex for people to find the tools to understand political phenomena, let alone predict the future. Think Again: The Power of Knowing What You Dont Know. Buy Expert Political Judgment - How Good Is It? How Can We - Amazon Structuring accountability systems in organizations: Key trade-offs and critical unknowns. We hesitate at the very idea of rethinking., When it comes to our own knowledge and opinions, we often favor. When, for instance, do liberals and conservatives diverge in the preferences for "process accountability" that holds people responsible for respecting rules versus "outcome accountability" that holds people accountable for bottom-line results? Tetlock first discusses arguments about whether the world is too complex for people to find the tools to understand political phenomena, let alone predict the future. Think Again is structured into three main parts. Designing accountability systems: How do people cope with various types of accountability pressures and demands in their social world? Tetlock has been interested in forecasting since the 1980s, he says during an interview at his home in Philadelphia. Philip E. Tetlock (born 1954) is a Canadian-American political science writer, and is currently the Annenberg University Professor at the University of Pennsylvania, where he is cross-appointed at the Wharton School and the School of Arts and Sciences. Wilbur Wright: Honest argument is merely a process of mutually picking the beams and motes out of each others eyes so both can see clearly.. How Accurate Are Prediction Markets? - JSTOR Daily The tournaments solicited roughly 28,000 predictions about the future and found the forecasters were often only slightly more accurate than chance, and usually worse than basic extrapolation algorithms, especially on longerrange forecasts three to five years out. ", "From the commercial to the communal: Reframing taboo trade-offs in religious and pharmaceutical marketing", "Detecting and punishing unconscious bias", "Tetlock, P.E., Armor, D., & Peterson, R. (1994). Tetlock, who was born in Canada, attended university in his native country, at the University of British Columbia, where he completed his undergraduate degree in 1975 and his Master's degree in 1976. You wouldn't use a hammer to try to cut down a tree, and try to use an axe to drive nails and you're likely to lose a finger. Psychological safety: The ability to take risks without fear of punishment or reprisal. This study tried to improve our ability to predict major - Vox We seek peak happiness (intensity), rather than small, steady positive happiness (frequency). Philip Tetlock | Psychology Philip Tetlock Leonore Annenberg University Professor BA, University of British Columbia; PhD, Psychology, Yale University Office Location: Solomon Labs, 3720 Walnut St, Room C8 Email: tetlock@wharton.upenn.edu Phone: 215-746-8541 Website: http://www.sas.upenn.edu/tetlock/ CV (url): Being persuaded is defeat. Implicit bias and accountability systems: What must organizations do to prevent discrimination? Practical tip: Favor content that presents many sides of an issue rather than a singular or binary view. Superforecasting is an informative, well-researched book, while remaining highly accessible. A Conversation with Adam Grant on Why We Need to Think Again, About Changing your mind is a sign of moral weakness. For millennia, great thinkers and scholars have been working to understand the quirks of the human mind. Although he too occasionally adopts this reductionist view of political psychology in his work, he has also raised the contrarian possibility in numerous articles and chapters that reductionism sometimes runs in reverseand that psychological research is often driven by ideological agenda (of which the psychologists often seem to be only partly conscious). From 1984 to 2004 Tetlock tracked political pundits' ability to predict world events, culminating in his 2006 book Expert Political Judgment. Political Psychology, 15, 567-577. Opening story: Marie-Helene Etienne-Rousseau of Quebec gives birth to a child. Values retain flexibility that opinions do not. If you dont change your mind frequently, youre going to be wrong a lot.. Jason Zweig ofThe Wall Street Journalcalls it the most important book on decision making since Daniel KahnemansThinking, Fast and Slow, which, in the area of behavioral economics, is very high praise indeed. Youre expected to doubt what you know, be curious about what you dont know, and update your views based on new data.. [3] The original aim of the tournament was to improve geo-political and geo-economic forecasting. The truth remains that for all our social science, the world manages to surprise us far more often than not. Opening story: Ursula Mercz, in the late 1800s, was diagnosed as blind but insisted she could see and was completely unaware of this fact. In the pursuit of scientific truth, working with adversaries can pay Tetlock, P.E., & Mitchell, G. (2009). How Can We Know? The lesson is that he lacked flexibility in his thinking. Tetlock P. and Mellers B. The first is the "Preacher". The stronger a persons belief, the more important the quality of the reasons or justifications. Why pundits and experts are so bad at predicting the future Here, Philip E. Tetlock explores what constitutes good judgment in predicting future events, and looks at why experts are often wrong in their forecasts. Psychological Review, 109, 451-472. "Hedgehogs" performed less well, especially on long-term forecasts within the domain of their expertise. Everybody'S an Expert | The New Yorker (2002). Students whose identities and ideologies were strongly intertwined (non-flexible thinkers) cracked. The Adversarial Collaboration Project, run by Cory Clark and Philip Tetlock, helps scientists with competing perspectives design joint research that tests both arguments. There are 4 modes of thinking: Preacher, prosecutor, politician, and Rethinking is fundamental to scientific thinking. It may inhibit further questioning and means for improvement. We would shut down., Philip Tetlock,Superforecasting: The Art and Science of Prediction. Recognize complexity as a signal of credibility., Psychologists find that people will ignore or even deny the existence of a problem if theyre not fond of the solution.. In practice, they often diverge.. This is especially troubling for people like policymakers, whose decisions affect entire populations. Use a steel man (instead of straw man) and consider your opponents strongest argument. Philip Tetlock - Management Department (2006) Ann Arbor, MI: University of Michigan Press. Grant recommends a fourth role to offset those found in Tetlocks model. Perspective-seeking is more useful than perspective-taking. As Prosecutor, we automatically attack any ideas that don't f Expert Political Judgment: How Good Is It? How Can We Know? Tetlock, P.E. Overcoming Our Aversion to Acknowledging Our Ignorance There Are 4 Modes of Thinking: Preacher, Prosecutor, Politician, and Expert Political Judgment. He evaluates predictions from experts in different fields, comparing them to predictions by well-informed laity or those based on simple extrapolation from current trends. Overcoming Our Aversion to Acknowledging Our Ignorance | WIRED Expert Political Judgment - Wikipedia How do we know what we know, and how do we know if were right? Just a few more efforts at rethinking can move the needle.. The final part looks at rethinking at the institutional or group level. How can organization structure incentives and accountability procedures to check common cognitive biases such as belief perseverance and over-confidence? Dan Gardner and Philip E. Tetlock review the not-too-promising record of expert predictions of political and social phenomena. Those with a scientific mindset search for truth by testing hypotheses, regularly run experiments, and continuously uncover new truths and revise their thinking. Experiments can inform our daily decisions.. [16], In addition to his work on the bias-attenuating versus bias-amplifying effects of accountability, Tetlock has explored the political dimensions of accountability. He is also the author of Expert Political Judgment and (with Aaron Belkin . One of Philip Tetlocks big ideas* is that we are typically operating in one of three modes when expressing or receiving an idea. Strong opinions like stereotypes and prejudice are less likely to be reconsidered. These include beliefs, assumptions, opinions, and more. They challenged each other's thinking and this allowed them to improve their ideas through a continuous feedback loop. This mindset embraces Grants idea of rethinking. 1 Department of Political Science, George Washington University, 2201 G. Street NW, Washington, DC 20052; e-mail: jimg@gwu.edu; 2 Departments of Psychology and Political Science, Ohio State University, 142 Townshend Hall, 1885 Neil Avenue, Columbus, Ohio 43210; e-mail: tetlock.1@osu.edu. Philip Tetlock carries out "forecasting tournaments" to test peoples' ability to predict complex events. Counterfactual thought experiments: Why we can't live with them and how we must learn to live with them. modern and postmodern values. Motivational interviewing: The best approach to changing someones mind is to help that person make the change on their own. the degree to which simple training exercises improved the accuracy of probabilistic judgments as measured by Brier scores; the degree to which the best forecasters could learn to distinguish many degrees of uncertainty along the zero to 1.0 probability scale (many more distinctions than the traditional 7-point verbal scale used by the National Intelligence Council); the consistency of the performance of the elite forecasters (superforecasters) across time and categories of questions; the power of a log-odds extremizing aggregation algorithm to out-perform competitors; the apparent ability of GJP to generate probability estimates that were "reportedly 30% better than intelligence officers with access to actual classified information. People can become very punitive "intuitive prosecutors" when they feel sacred values have been seriously violated, going well beyond the range of socially acceptable forms of punishment when given chances to do so covertly. Everyone carries cognitive tools that are regularly used and seldom questioned or subject to reflection or scrutiny. In the most comprehensive analysis of expert prediction ever conducted, Philip Tetlock assembled a group of some 280 anonymous volunteerseconomists, political scientists, intelligence analysts . Instead of searching for reasons why we are right, search for reasons for why we are wrong. American Political Science Review, 95, 829-843. Philip Tetlockin Superforecasting: The Art and Science of Prediction, Tetlocks career has been based on the assessment of good judgment. Psychologist Peter T. Coleman experiments to learn how to reverse-engineer successful conversations between people about polarizing issues. Philip Tetlock's Edge Bio Page [46.50 minutes] INTRODUCTION by Daniel Kahneman These habits of thought can be learned and cultivated by any intelligent, thoughtful, determined person., Tetlock, who was born in Canada, attended university in his native country, at the University of British Columbia, where he completed his undergraduate degree in 1975 and his Masters degree in 1976.8He went on to do his doctoral studies at Yale, where he obtained his Ph.D. in psychology in 1979.9Since then, Tetlock has taught courses in management, psychology, and political science at the University of California, Berkeley, the Ohio State University, and the University of Pennsylvania, where he is a current faculty member.10Broadly, his research focuses on the evaluation of good judgment and the criteria used to assess judgment, bias, and error.11, In describing how we think and decide, modern psychologists often deploy a dual-system model that partitions our mental universe into two domains.