Edit: I no longer think this post deserves attention.

Edit: Invoking a black-box agent is probably a very bad idea [https://www.lesswrong.com/posts/DNKTmmNZr5M2uCZLz/beware-of-black-boxes-in-ai-alignment-research] unless there’s a consensus in the AI safety community it should be done.

This post describes a setup that allows asking a super-intelligent agent a question, and getting an answer that humans with an unbounded amount of time could come up with (Edit: actually, if the question is "too easily distinguishable" from questions we already know the answer to, it's plausible we won't get a useful answer). The setup is not restricted to questions with a single or precise answer, and there is no restriction on the length of the answer. This setup might be safer than the naive one - let me know what you think (and let me know if it's all already covered by existing work).

THE SETUP First, we build a big dataset of (question, answer) pairs: {(Qi,Ai)}

The questions are of the following form: "Limited to 100 words, what might humans answer in 2018 when asked how people in 1950 could most effectively improve their wellbeing?" (Edit: this is a bad example, see first edit)

The answer in this example would be a 100 word text, that might contain the phrase "stop smoking". The questions and answers can be written by any "regular" crowd of people (e.g. MTruk workers).

We then create a super-intelligent agent, denoted X, that has the entire dataset written in memory at invocation time. The agent X is in a box and has no input channel. The goal of X is to find an output string o that is at most n characters long (where n is also written in X's memory at invocation time). The string o is the code of another agent, denoted Y.

Agent Y also has no input channel, and when invoked - some question Qi should be written in its memory. Its output is a probability distribution over all possible answers for Qi (we assume that an upper-bound on the answer length is define by the q
https://www.lesswrong.com/posts/68E8emGRvyJPqanvZ/a-safer-oracle-setup
* a very bad idea