Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Impact is very complicated, published by Justis on May 22, 2022 on The Effective Altruism Forum. Epistemic status: gestural food for thought. This is a post to aggregate a bunch of boring or obvious observations that are easy to overlook. I don't expect anything to be new. But I do think difficulties gauging impact, in aggregate, form a sort of missing mood that we should be paying more attention to. At the end of the post, I'll touch on why. Let us count the ways Here are some factors that can make assessing impact complex: Some interventions are backed by scientific studies. Scientific studies vary in quality in many ways. They can be larger or smaller, more or less numerous, clearly biased or apparently unbiased, clearly significant or only marginally significant, randomized or non-randomized, observational or experimental, etc. Even for good studies, the same interventions may become better or worse over time as conditions in the world change, or may be very particular to certain places. Cash transfers, for example, might just work way better in certain parts of the world than others, and it may be hard to predict how or why in advance. Many interventions require many simultaneous layers of involvement. Suppose I give to the Against Malaria Foundation. I can say "I estimate I am saving a life per $5,000 I spend." But I read Peter Singer when I was 13, then Scott Alexander when I was 19, and I likely wouldn't have ended up donating much without these. I also couldn't give to AMF if it didn't exist, so I owe a debt to Rob Mather. And perhaps whoever told Scott Alexander about AMF. All these steps are necessary to actually "save a life", so we run the risk of massively overcounting if we give every person in the chain "full credit". But there's no objectively rigorous way to decide who gets how much of the credit! Just using counterfactuals doesn't work; it may be the case that all of us are required and a single person "out of the chain" breaks it down. But we can't all get all the credit! Plus many interventions, like in the AMF example, mostly are just reducing probabilities across large numbers of people anyway. What does it even mean for "my money" to "save life". Once the money all goes into a pool, whose money actually funds which nets anyway? And which nets prevent cases of malaria that would have been fatal? No way to answer these questions even in principle. Some (perhaps all) interventions rely on difficult-to-impossible philosophical questions to resolve. How should we weigh insect suffering? All we can do is guess - learning more facts about insects doesn't really get us over Nagel's "What is it like to be a bat?" hurdle. Empirical information, analogies, and intuition pumps all can help, but there are fundamental judgment calls at play. How to assess well being and weigh well being against survival is another example here where it's hard to boil down to numbers: there are lots of ways to do it (QALYs, seeing how much people would pay to avert various harms, natural experiments) but none is perfect and all involve their own judgment calls. Some interventions require certain thresholds being met or they don't actually accomplish anything. Donating to a political campaign that promises credibly to do something good might help bring about that good thing. But if the campaign fails, that donation accomplished basically nothing. Existential risk mitigation efforts (as such) only do any good if they work. If the world ends anyway, that effort didn't actually accomplish anything. Plenty of interventions can also backfire. Some interventions aim to increase or decrease probabilities. There are a lot of ways to mess this up. My least favorite arguments in intro EA messaging historically were things like: "even if the chance of [EVENT] is quite low, say 1%, then ...