If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All

A bird sorting pebbles into its nest is where the abstract alignment problem turns physical: "As soon as I saw that the nest had 91 pebbles…I knew it was wrong," one reader recalls, and suddenly a trained system chasing the wrong target has a face. The explanation underneath earns the fear readers keep reporting: "advanced AIs are not engineered so much as grown, operating with opaque and unpredictable results, untethered from human values," which means the builders themselves cannot say what their machine wants. The pace helps; readers keep finishing it in a day. Even expert scrutiny bends toward it: a reviewer with real technical standing, nobody's doomer, calls the core case "likely correct." An admirer leaves the skeptic one honest assignment: "figure out WHY you disagree."

Some of the harshest reviews come from readers who already believe AI is dangerous. One still calls the book "a pretty horrible example of rhetoric," since most of it is a science fiction what-if with no power to persuade anybody new. Another writes, "you can bullshit your way into any stance if you weave enough thought experiments," and the scenario isn't even original, traced by one reader to a Vernor Vinge prelude. A sympathetic philosopher names the load-bearing slide: "imperfect alignment entails catastrophic misalignment. But that's just not true." What remains: oversized type and wide margins inflating a short argument to 272 pages, all delivered in a "patronizing dumbed-down tone and storytelling schtick." Even the remedy, binding international control backed by force, strikes allies as unlikely and harmful if botched.