If Anyone Builds It, Everyone Dies

When it comes to thinking, quality trumps quantity.

Even in the face of superhuman machine intelligence, it can be tempting to imagine that the world will keep looking the way it has over the last few decades of our relatively short lives. It is true, but hard to remember, that there was a time is real as our own time, just a few short centuries ago, when civilization was radically different. Or millennia ago, when there was no civilization to speak of. Or a million years ago, when there were no humans. Or a billion years ago, when multicellular colonies had no specialized cells.

Adopting a historical perspective can help us appreciate what is so hard to see from the perspective of our own short lifespans: Nature permits disruption. Nature permits calamity. Nature permits the world to never be the same again.

In our view, intelligence is about two fundamental types of work: the work of predicting the world, and the work of steering it.

Some aspects of the future are predictable, with the right knowledge and effort; others are impossibly hard calls. Competent futurism is built around knowing the difference.

A human brain can learn to navigate wider-ranging paths through a larger cross-section of reality than any other animal. That is our special power.

The more complicated the game board, the more advantage goes to the player with more knowledge and more intelligence and more understanding of the game.

Normality always ends. This is not to say that it’s inevitably replaced by something worse; sometimes it is and sometimes it isn’t, and sometimes it depends on how we act. But clinging to the hope that nothing too bad will be allowed to happen does not usually help.