On Thinking: Recent Episodes
Maarten Van Doorn
Listen along as I have casual conversations with experts in human reasoning, cognitive science and philosophy to try and understand how our thinking really works—and how we might do better in figuring out the truth.
There’s a huge debate going on in social science right now. The question is simple, and strikes near the heart of all research: What counts as solid evidence? When it comes to proving an hypothesis, key terms are 'statistical significance' and 'p-values'. With philosopher of science Remco Heesen (University of Western Australia & University of Groningen) I discuss the ongoing discussion around these concepts, and clear up some misunderstandings along the way.
Here's what we talk about:
- How do scientists determine whether an experimental result should be considered true, i.e. wheher the experiment worked / whether some manipulation was effective / wheter there was an effect or difference between groups
- P-values and “statistical significance” play a central role here but are widely misunderstood. This is what they actually mean.
- Problems with this way of doing science
- What could be alternatives? (Briefly)
More info
- Remco Heesen: website, twitter, UWA page, RUG page
- Maarten van Doorn: website, substack, twitter, RU page
3. Across many scientific fields, roughly half of the research findings established over the past century – hundreds of thousands of results – might simply be untrue, and the rest might me much less important than previously thought. At the same time, more and more people trying to find the scientific literature themselves to interpret it. This seems like an interesting combination, which I discuss with philosoper of science Remco Heesen.
Timeline
03:00 What does a (single) scientific study reporting a "significant result" tell you?
06:00 The replication crisis
09:00 The holy grail of science: the p-value (or: how most scientists think about evidence and what you can conclude from it)
13:00 P-hacking
15:00 Why whether some study's result is (probably) Really True, depends on doing only study at a time
19:00 Salami publishing and wrong economic and reputational incentives in the academic publication and career systems
25:00 Solutions: preregistration, publishing negative results, opening up peer review
17. Disinformation. Trolling, conspiracies, social media pile-ons and cancel culture. On the surface, these recent additions to our daily vocabulary appear to have little in common. But together, they are driving an epistemic crisis: a multi-front challenge to our ability to distinguish fact from fiction and elevate truth above falsehood. Jonathan Rauch reaches back to the parallel eighteenth-century developments of liberal democracy and science to explain what he calls the Constitution of Knowledge our social system for turning disagreement into truth. By explicating the Constitution of Knowledge and probing the war on reality, Rauch arms defenders of truth with a clearer understanding of what they must protect, why they must do so and how they can do it.
Timeline
03:00 What an epistemic crisis is, and why we are in one (it's not about factual disagreements)
11:00 A perverse rationality in believing in false things
15:00 It's surprisingly dangerous if we can't know what people around us believe because they're not allowed to say
20:00 What does Rauch mean by "The constitution of knowledge"?
25:30 The similarities between producing justice and producing knowledge
31:00 Is the constitution of knowledge not just a network of gatekeepers?
35:00 How the constitution of knowledge has been responsible for the growing moral acceptance of homosexuality
40:00 The (overestimated) role of social media in our epistemic crisis
52:00 Making sure truth wins
Resources
Jonathan Rauch (2021) - The Constitution of Knowledge
Jonathan Rauch (2014) - Kindly Inquisitors
Braverangles.org
★ Support this podcast on Patreon ★
Predictable polarization is everywhere. When we make decisions about what college to attend, or what books to read, or which friends to hang out with, we can usually predict—not with certainty, but with confidence—that doing so will move our opinions in a particular direction. Could this process be (epistemically) rational? Kevin Dorst argues that it can be, if the evidence we get is ambiguous, in the sense that it’s rational to be unsure how to react to it. We explore this argument and an extension of it: by the same logic (says Kevin), it's also rational to interpret evidence in way that favors your prior beliefs. Upshot: polarization and confirmation bias are often rational.