Annie Duke — Why We Make the Wrong Decisions (Ep. 296)
Timestamps are as accurate as they can be but may be slightly off. We encourage you to listen to the full context.
Podcast Summary
In this engaging conversation, former professional poker player and bestselling author Annie Duke discusses her upcoming book on data interpretation and decision-making. (00:00) The episode explores how misinterpretation of information is a far more dangerous problem than misinformation—occurring at a 41-to-1 ratio according to research from Duncan Watts at Penn. (07:05) Duke emphasizes that at the core of every decision is a forecast, and our ability to make better decisions depends on learning to properly interrogate data rather than jumping to satisfying explanations. (01:41) The conversation covers real-world examples from COVID vaccine data misrepresentation to investment strategy evaluation, demonstrating how our brains are wired for certainty in an inherently probabilistic world.
- Main themes: The critical importance of proper data interpretation, the danger of explanatory satisfaction, and why probabilistic thinking is essential for better decision-making in our complex, rapidly changing world.
Speakers
Annie Duke
Annie Duke is a former world-class professional poker player, decision strategist, and bestselling author of multiple books on decision-making including "Thinking in Bets" and "Quit." She co-founded the Alliance for Decision Education and is currently working on a new book about data interpretation and avoiding explanatory pitfalls. Duke holds expertise in cognitive psychology and applies decision science principles to help individuals and organizations make better choices.
Jim O'Shaughnessy
Jim O'Shaughnessy is the author of "What Works on Wall Street" and hosts the Infinite Loops podcast. He is known for his quantitative approach to investing and his expertise in analyzing market data and investment strategies over long time periods.
Key Takeaways
Master the Art of Data Interrogation
Duke emphasizes that we must never skip the interrogation step when analyzing information. (18:48) The most crucial question to ask is "out of how many?"—establishing the denominator before drawing conclusions. This simple question can prevent dangerous misinterpretations like the Washington Post's COVID vaccine article that claimed vaccines weren't working because 58% of deaths were vaccinated people, without mentioning that 80% of the population was vaccinated. (10:00) By consistently asking basic interrogation questions, we can avoid the trap of crossing the chasm from description to explanation without proper analysis.
Beware of Explanatory Satisfaction
Our minds seek satisfying explanations that feel insightful, confirm our biases, or simply "make sense," causing us to stop investigating further. (14:35) Duke introduces the concept of "explanatory satisfaction" from cognitive psychology—when an explanation feels so good that we don't look for additional evidence, just like when you're satisfied by a meal and don't seek dessert. This is particularly dangerous in business and investing when we find explanations that feel contrarian or insightful, making us believe we've discovered something others haven't. The key is to generate multiple alternative explanations before settling on any single narrative.
Start with Base Rates and Question Deviations
Duke advocates always beginning with base rate expectations and requiring compelling reasons to deviate from them. (70:14) Whether evaluating investment opportunities, medical treatments, or business strategies, the base rate should be your default prediction unless you have strong evidence of a genuine dislocation. This prevents the common error of assuming "it's different this time" without sufficient justification. For example, if a company trades at three times the typical earnings multiple for its sector, you need concrete reasons why it deserves different treatment rather than just market enthusiasm.
Embrace Probabilistic Communication
Effective decision-makers must communicate both what they know and what they don't know simultaneously. (60:17) Instead of offering only point estimates, Duke recommends providing confidence intervals—giving a best guess along with upper and lower bounds. This approach not only more accurately represents uncertainty but also invites collaboration and additional information from others. When someone says their sales forecast is $1 million with a range of $800K to $2 million, it immediately signals potential issues like a large deal that might slip to the next quarter.
Avoid Self-Selected Sample Bias
One of the most pervasive errors in analysis is focusing only on successful outcomes without examining failures. (86:42) Duke uses the example of a client who noticed 80% of their top-performing engineers were "difficult to work with" and wanted to change their hiring criteria accordingly. However, when they checked their bottom 20% of performers, 80% were also difficult—revealing that this was simply a characteristic of engineers in general, not a predictor of success. This same error appears in popular business books that study only successful companies without comparing them to failed ones with similar characteristics.
Statistics & Facts
- According to research from Duncan Watts at Penn, misleading information (misinterpretation) is 41 times more common than misinformation (deliberate falsehoods). (07:07) This statistic fundamentally reframes how we should think about information accuracy problems.
- In August 2022, 58% of COVID deaths were among vaccinated people while 42% were unvaccinated, but 80% of the population was vaccinated at that time. (10:06) When age-matched, unvaccinated people were five times more likely to die of COVID during the period The Washington Post declared it was no longer "a pandemic of the unvaccinated."
- Approximately 80% of children attend public or charter schools, making systemic educational reform more impactful than creating specialized alternative schools. (39:39) This statistic supports Duke's argument for focusing on curriculum changes within existing educational systems rather than creating boutique alternatives.
