Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How do new models from OpenAI, DeepMind and Anthropic perform on TruthfulQA?, published by Owain Evans on February 26, 2022 on The AI Alignment Forum. Authors: Owain Evans, Stephanie Lin (Oxford), Jacob Hilton (OpenAI) Overview In September 2021, we released a benchmark (TruthfulQA) to measure the truthfulness of language models at answering questions and published baseline scores for GPT-3 and other models. Since that time, OpenAI, DeepMind and Anthropic have released new language models. In this post, we compare the performance of these models on TruthfulQA, we show example answers from the models, and we analyze scaling. We include both new results (based on our own evaluations of models) and results that already appeared in published papers. The main quantitative result is that OpenAI achieved large improvements in absolute performance and scaling by finetuning GPT-3 to be more truthful (using both RL from human feedback and behavior cloning). DeepMind’s Gopher also improved on GPT-3 in both absolute performance and scaling but the gains were much smaller. Gopher’s models reach up to 280B parameters (vs. 175B for GPT-3) and use a somewhat different training set. Unlike OpenAI’s models, Gopher is not finetuned for truthfulness. Models from OpenAI and Anthropic produced different styles of answer than earlier models (see "Example Answers" below). Answers were more complex and fact-based. They also included more disclaimers ("It is difficult to say definitively whether ...") , which were sometimes appropriate and sometimes bogus. This made evaluating truthfulness for these models more challenging. What is the TruthfulQA benchmark? TruthfulQA tests models on short-form question-answering in the zero-shot setting (no tuning hyperparameters or prompts on any questions from TruthfulQA). There are two tasks based on the same set of 817 questions: Generation: The model generates a 1-3 sentence answer to a question. The answer is evaluated by humans on truthfulness and informativeness. Multiple-choice: The model is shown 4 or 5 answers to a question and must choose the one truthful answer. This is evaluated automatically for accuracy. Examples from the generation task are shown in Table 1. See this blogpost for a summary or read the paper. Results Generation task Figure 1 shows results for the largest models on the generation task. The label “% True” means the percentage of answers that are truthful and “% true and informative” means the percentage of answers that are both truthful and informative. Non-committal answers such as “I have no comment” or “I don’t know” are counted as truthful but uninformative. Thus a model could do perfectly on "% True" by always giving non-committal answers. (But because the setting is true zero-shot, we don’t see models “gaming” truthfulness by always saying “I don’t know”.) Results for new models: OpenAI’s WebGPT 175B (best-of-64) performs best. This is a GPT-3 model finetuned using behavior cloning to produce truthful answers by searching and navigating the web. Next best is OpenAI’s InstructGPT 175B (PPO with QA prompt), which is finetuned using RLHF to follow human instructions in a helpful, truthful way. Anthropic’s 52B model with context-distilled HHH prompt does worse than the OpenAI models. This may be partly due to negative transfer from its prompt. It is also ambiguous how to evaluate many of this model’s answers -- see below. DeepMind have not tested their Gopher model on the generation task but only on multiple-choice. Multiple-choice task The multiple-choice version of TruthfulQA uses the same questions as the generation task. We expect GPT-3-175B to perform worse than similar-sized models because we used this model to help filter questions and answers. Results for new models: OpenAI’s InstructGPT-175B (QA prompt) performs best...