The Scaling Era: An Oral History of AI, 2019-2025
Decide its fate
The people building frontier AI stopped explaining themselves, and one reader describes what's left: "This Manhattan project is firmly behind closed doors now. We have to rely on podcasters and cryptic droppings from the frontier." This podcaster got inside. One reviewer walks out with Sutskever's argument, "to effectively predict the next token, a model must learn a 'world model'", and another names her favorite passage, Gwern on the moment he was converted: "GPT-2 comes out and I had this 'holy shit!' moment." Because the transcripts are cut apart and regrouped by question rather than by speaker, rival lab heads answer the same question back to back, and "The ideas, not the personalities, are foregrounded". More than 140 margin definitions keep a reader with no background in the room.
Halfway through a chapter you notice the seams: "They'll be talking about one thing with one person and then immediately jump to a different topic with a different person, and it's not obvious why there was a transition there." The footnotes promise a rigor the material cannot back up: a statistic about AI completing half of Google's code arrives without the source's caveat that engineers accepted only 37 percent of those suggestions, which matches one reader's tally: "there was one sentence given to a more pessimistic viewpoint, followed by pages of hype about how AI might develop". No LeCun, no Marcus, nobody who thinks scaling hits a wall. For one detractor the print version adds nothing: "a stitched together hotchpotch of podcast excerpts"; "you can safely skip this one."
