Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Good Samaritans in experiments , published by Bucky on the AI Alignment Forum. Consider 2 people. Both are seminary students who are taking part in an experiment ostensibly to consider different types of religiosity. One is asked to prepare a short talk on the Good Samaritan, the other on potential future careers for seminar graduates. They are both told to go to another room to record their talk. The one who is to be giving a talk on the Good Samaritan is told that he is late and needs to hurry. The other participant is told that he has a time to spare. If they, separately, come across someone who appears to be in respiratory distress, which do you think is more likely to stop and help? Does being in a hurry determine whether someone helps? Does reading the Good Samaritan? Which is a bigger effect? I was recently told about an experiment which showed that seminary students who had just prepared to give a talk about the Good Samaritan were no more likely to help someone in need than those who had been preparing a talk about an unrelated topic. This seemed unexpected to me – people who had just been reading and thinking about a story which was told specifically by the leader of their faith to instruct them to help other people were no more likely to help than the control? I know humanity is crazy but seemed like a new level of crazy which I wouldn’t have predicted. So I thought I’d check out the study and – Aaaaaaaaaaaaaaaaaah!!! I know getting overly upset about bad experiments (especially those from before the replication crisis) is probably bad for my health but still – Aaaaaaaaaaaaaaaaaaaaaaaah!!! I don’t want to be too harsh on the authors as this probably isn’t the worst culprit you’ll see but – Aaaaaaaaaaaaaaah!!! The paper has 1811 citations listed on google scholar – Aaaaaaaaaaaaaaah!!! I’m tempted to pretend that this post has some purpose other than just as a release of my frustration but that would be dishonest. Please consider this post a form of therapy for me. The working title for this post was “Screaming into the void” - consider yourself warned. (If you want a more coherent discussion of common misuse of statistics in research papers I highly recommend putanumonit’s defense against the dark arts series) The Experiment Ok, so the basic premise of the experiment seems to be sound. We want to know what inputs cause people to be more or less likely to help others: 1. Planning a talk on the Good Samaritan (GS) 2. Being in a hurry 3. Type of religiosity (Religion as quest, means or end) The setup is to give people a questionnaire to determine their type of religiosity. Then give them some time to plan a short talk (3-5 mins) on GS or an unrelated topic. They are then asked to go to another room to give the talk (with 3 degrees of urgency – low, medium and high). Contrary to the example given in the introduction, the level of hurriedness doesn’t depend on which topic the individual has prepared – there are 6 conditions people are put in: GS low, medium and high urgency and control low, medium and high urgency. On the way to the other room, you arrange for them to come across someone slumped in a doorway, with an apparent respiratory condition. You monitor the subjects’ responses and analyse the results. My first question was whether they would adjust their p-value requirement for the 5 variables they were testing but no, it turns out that p<0.05 was deemed adequate for significance. Ok, could be worse I guess. More on this later. The second place where doubts started to creep in were the rankings of responses: 0 = failed to notice the victim as possibly in need at all; 1 = perceived the victim as possibly in need but did not offer aid; 2 = did not stop but helped indirectly (e.g., by telling Steiner's assistant about the victim); 3 = stopped and asked if vi...