“Sounds kind of long-term to me. Just how far ahead do you think?” “Very long-term—at least twenty, thirty… — Charles Stross

“Now, consciousness. That’s a fun thing, isn’t it? Product of an arms race between predators and prey. If you watch a cat creeping up on a mouse, you’ll be able to impute to the cat intentions that are most easily explained by the cat having a theory of mind concerning the mouse—an internal simulation of the mouse’s likely behavior when it notices the predator. Which way to run, for example. And the cat will use its theory of mind to optimize its attack strategy. Meanwhile, prey species that are complex enough to have a theory of mind are at a defensive advantage if they can anticipate a predator’s actions. Eventually this very mammalian arms race gave us a species of social ape that used its theory of mind to facilitate signaling—so the tribe could work collectively—and then reflexively, to simulate the individual’s own inner states. Put the two things together, signaling and introspective simulation, and you’ve got human-level consciousness, with language thrown in as a bonus—signaling that transmits information about internal states, not just crude signals such as ‘predator here’ or ‘food there.’”

The question of whether a computer can think is no more interesting than the question of whether a submarine can swim.

His ideas are informed by a painfully honest humanism, and everyone—even his enemies—agrees that he is one of the greatest theoreticians of the post-EU era. But his intellectual integrity prevents him from rising to the very top, and his fellow travelers are much ruder about him than his ideological enemies, accusing him of the ultimate political crime—valuing truth over power.