Eliezer Yudkowsky

Quotes

1. Computer security is widely understood to be a problem so hard, so cursed, that it cannot be solved, period.

   p. 172

2. 

   The greatest and most central difficulty in aligning artificial superintelligence is navigating the gap between before and after.…

   Ideas and theories can only be tested before the gap. They need to work after the gap, on the first try.

   p. 161

3. 

   If you know the history of science, this kind of talk is recognizable as the stage of folk theory, the stage where lots of different people are inventing lots of different theories that appeal to them personally, the way people talk before science has really gotten started on something. They’re the words of an alchemist who’s decided that some complicated philosophical scheme will let them transmute lead into gold.

   p. 182

4. 

   The more complicated the game board, the more advantage goes to the player with more knowledge and more intelligence and more understanding of the game.

   p. 99

5. 

   We usually try to avoid shouting. It doesn’t help to shout, most of the time. It just makes people think you’re undisciplined. But at some point, after you’ve calmly gone through all the premises of your argument, we think it becomes unhelpful to downplay, lest people think it’s all just a game of calm words.

   When it comes to AI, the challenge humanity is facing is not surmountable with anything like humanity’s current level of knowledge and skill. It isn’t close.

   Attempting to solve a problem like that, with the lives of everyone on Earth at stake, would be an insane and stupid gamble that NOBODY SHOULD BE ALLOWED TO TRY.

   p. 176

6. 

   The problem with making AIs want – and ultimately do – the exact, complicated things that humans want is a major facet of what’s known as the “AI alignment problem.” It’s what we had in mind when we were brainstorming terminology with the AI professor Stuart Russell back in 2014, and settled on the term “alignment.”

   Most everyone who is building AIs, however, seems to be operating as if the alignment problem doesn’t exist – as if the preferences the AI winds up with will be exactly what they train into it. This assumption lurks in the background whenever someone says, “The USA needs to build superintelligence before China, because we don’t trust China,” as if the factional allegiance of whoever ran the gradient descent determined what the resulting AI wanted.

   p. 74

7. 

   Normality always ends. This is not to say that it’s inevitably replaced by something worse; sometimes it is and sometimes it isn’t, and sometimes it depends on how we act. But clinging to the hope that nothing too bad will be allowed to happen does not usually help.

   p. 11

That’s all the quotes.

7 quotes in the collection · come back when we add more