Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Coherent extrapolated dreaming, published by Alex Flint on December 26, 2022 on The AI Alignment Forum.This work was supported by the Monastic Academy for the Preservation of Life on Earth. You can support my work here.I will give a short presentation of this work followed by discussion on Wednesday Dec 28 at 12pm Pacific / 3pm Eastern. RSVP here.OutlineI have four questions above coherent extrapolated volition, which I present in the form of four short thought experiments:What kind of a thing can be extrapolated in the direction of wisdom? (Robot vacuum thought experiment)What kind of protocol connects with the wisdom of a person who has been extrapolated? (Dream research thought experiment)What kind of model captures that within a person that we hope to amplify through extrapolation? (Twitter imitator thought experiment)What kind of environment is sufficient to grow true wisdom? (Astrological signs thought experiment)The title of this post is based on the second thought experiment.I claim that we lack a theory about that-which-is-capable-of-becoming-wise, in a form that lets us say something about its relationship to models, extrapolation, and volitional dynamics. I argue that CEV does not actually provide this central theory.IntroductionCoherent extrapolated volition is Eliezer’s 2004 proposal for the goal we might give to a powerful AI. The basic idea is to have the AI work out what we would do or say if we were wiser versions of our present selves, and have the AI predicate its actions on that. To do this, the AI might work out what would happen if a person contemplated an issue for a long time, or was exposed to more conversations with excellent conversation partners, or spent a long time exploring the world, or just lived a long and varied life. It might be possible for us to describe the transformations that lead to wisdom even if we don’t know apriori what those transformations will lead to.CEV does not spell out exactly what those transformations are — though it does make suggestions — nor how exactly the AI’s actions would be connected to the results of such transformations. The main philosophical point that CEV makes is that the thing to have an AI attend to, if you’re trying to do something good with AI, is wisdom, and that wisdom arises from a process of maturation. At present we might be confused about both the nature of the world and about our own terminal values. If an AI asks us " how should honesty be traded off against courage?" we might give a muddled answer. Yet we do have a take on honesty and courage. Wiser versions of ourselves might be less confused about this, and yet still be us.An example: suppose you ask a person to select a governance structure for a new startup. If you ask the person to make a decision immediately, you might get a mediocre answer. If you give them a few minutes to contemplate then you might get a better answer. This "taking a few minutes to contemplate" is a kind of transformation of mind. Beginning from the state where the person was just asked the question, their mind changes in certain ways over the course of those few minutes and the response given after that transform is different to the response before it.Perhaps there are situations where the "taking a few minutes to contemplate" transform decreases the quality of the eventual answer, as in the phenomenon of "analysis paralysis" — CEV does not claim that this particular transform is the wisdom-inducing transform. CEV does claim that there exists some transformation of mind that leads to wisdom. It need not be that these transformations are about minds contemplating things in isolation. Perhaps there are certain insights that you can only come to through conversations with friends. If so, perhaps the AI can work out what would become of a person if...