Why Yudkowsky is probably wrong

[John Clark]

Ben Goertzel has written a rebuttal to Eliezer Yudkowsky and Nate Soares's new book "If Anybody Builds It Everybody Dies", and I pretty much agree with everything he said: 

Why “Everyone Dies” Gets AGI All Wrong

I remember Ben from when he and I and Eliezer were on the same mailing list in the late 1990s and I disagreed with him on some stuff, but not on his review of Eliezer's book, I especially liked it when he said this:  

"Eliezer is right about one thing: we cannot know with certainty that AGI won’t lead to human extinction –  But the leap from uncertainty to “everybody dies” represents a tremendous failure of imagination about both the nature of intelligence and our capacity to shape its development. Their arguments in favor of this scary segue were inevitably extremely full of holes.     Stuff like “a mind randomly chosen from mindspace would have probability near zero of caring about humans.” OK sure, probably, but that would be a very bizarre and difficult thing to do, to choose a random mind from mindspace (according to any reasonably broad distribution) – obviously what people will actually do is quite different, more like creating our “mind children” drawn from the very biased distribution of AGI minds initialized with some version of human value system" 

On another, perhaps not completely unrelated, topic yesterday anthropic released Claude Sonnet 4.5 and it is the most powerful coding agent in the world. In May anthropic introduced Claude Opus 4, it was capable of working on a coding problem continuously for 7 hours and at the time it was the most powerful coding agent in the world, but Sonnet  4.5 can now work on a project for 30 hours. That's better than a four fold increase in four months. Oh and by the way, it also has the lowest rate of misaligned behavior and the lowest amount of sycophancy.

Forget GPT-5… Anthropic’s Sonnet 4.5 Just Changed Everything

 John K Clark    See what's on my new list at  Extropolis

lascuq
ns9