Dispatches from the last days of human relevance
John Clark
he entire process seems have been one-shot: my former student Lijie Chen simply gave GPT the problem, then GPT thought for a while and output a several-page argument that, on analysis by human experts, turned out to be correct. Of course there's selection bias here; we're not hearing as much about the hundreds of other problems GPT was given that it didn't solve (isn't that the case with humans too?). Clearly, too, GPT was helped by the facts that human mathematicians had wasted most of their time trying to prove Erdös right rather than looking for a counterexample, and that, even if they did look for a counterexample, they'd need to be experts in algebraic number theory to find this one, which hardly any discrete geometers are. So, maybe that suggests that AI, right now, is "merely" picking various medium-hanging fruits that human mathematicians missed for contingent reasons? With emphasis on the "right now."
In a companion paper, OpenAI helpfully included commentary from Timothy Gowers, Noga Alon, Will Sawin, Daniel Litt, and many other experts, reflecting on the breakthrough, the path that GPT took to get to it (which can actually be seen by examining its chain-of-thought), and what this might mean for the future of mathematical research.
I heard the news maybe an hour after it broke, when some UT grad students came to my office to tell me. For what it's worth: these students were morose, musing about how everything might soon be over for young scientists and mathematicians like themselves. I don't know whether they're right, but I feel like I should tell the truth about what their reaction was.
Then, a few days later, a team at DeepMind, including my UT Austin colleague Swarat Chaudhuri, announced that they were able to use a system called AlphaProof Nexus to settle nine more (!) Erdös problems, many of them in additive combinatorics, along with miscellaneous other open math problems. Notably, in this case the AI also fully formalized its proofs in Lean.
And then, just today, Jelani Nelson alerted me to a new CS theory paper, which solves a longstanding open problem about electrical flows on graphs using a proof from GPT5.5Pro.
It seems to me that we're now over the top of this particular rollercoaster, and it will keep accelerating until we reach the bottom, wherever that might be. I don't know whether to hope or dread that solutions to P versus NP and all our other great problems will be included in the ride---that our role, as human mathematicians, will be reduced to (at most) deciding which questions we find interesting and then understanding AI models' answers to those questions.
But maybe that won't happen. Maybe the new AI mathematicians will soon hit a wall, because they lack the uncomputable quantum gravity microtubules of Penrose and Hameroff, or some other magic human ingredient. The fantastical thing is that, one way or the other, we're going to find out empirically before very long.
Readers may have also seen the news that multiple prizewinning entries in a short fiction contest called the Commonwealth Prize, give overwhelming indications of having been written by AIs. As Kelsey Piper puts it:
There are, let’s say, also some noticeable similarities in the prose style between the winning stories that were flagged for AI use. AI chatbots love metaphors and similes, and they often spit out ones that sound vaguely pleasing but are logically incoherent or ascribe properties to things that don’t make sense.
“The Serpent in the Grove” gave us, “The girl smiled like sunrise over a sink.” “The Bastion’s Shadow” says, “She carried it now in her bag, heavy as a charm.” “Mehendi Nights” describes something as “swaying against plaster like a warning bell.”
The Commonwealth Foundation, whose judges chose these stories, hasn't exactly covered itself in glory---saying, on the one hand, that it strictly forbids AI use but on the other, that it will continue taking authors at their word that they didn't use AI, no matter the immensity of evidence to the contrary. As many others have pointed out, judges more familiar with AI would've ironically been better placed to notice the signs of its use.
If only there were some sort of automated way to detect AI-generated text. Someone should really get on that problem, don't you think?
But maybe we should just throw in the towel---as some of my colleagues have already done in the context of undergraduate projects? Maybe we should simply say that a good story is a good story, regardless of what manner of entity produced it?
As it happens, just last week I read my very first AI-written story that affected me as a story, to the extent that I wanted to read it more than once. This happened when I gave GPT5.5Pro the following simple prompt:
Write me a story about the most ancient Israelites that’s riveting like the stories of the Bible but that’s also consistent with all of the archeological evidence.
You can read the result here. One of my Facebook friends called it "disturbingly good," and I share that assessment. Of course, I'm well aware that GPT could easily generate a thousand stories like this one---sampled from the same probability distribution---and then I could even do statistics on which tropes were the most common. This makes it feel silly to overindex on the first story that happened to be output, and yet somehow I did.
I feel like at this point, both the prophets of AI utopia like Ray Kurzweil, and of AI doom like Eliezer Yudkowsky, could be forgiven for asking: dude, will you listen to us YET? Do you still find it prudent to call this new form of terrestrial intelligence a stochastic parrot, a laughable fraud, or a fad that's about to go away? Fear it all you want, hate it even, but at least respect it!
Which brings me to the other big AI news from the past week, namely that Pope Leo released his first encyclical, which is entitled "Safeguarding the Human Person in the Time of Artificial Intelligence." I read it and ... well, I certainly agreed with the theme that such a world-changing technology needs to be developed for the common good (as the Pope would have it, like the walls of Jerusalem), rather than for the profit or vanity of any one individual or company (in his analogy, like the Tower of Babel). I had quibbles with some of the other parts. Zvi Mowshowitz, as he often does, had a superb paragraph-by-paragraph analysis. Amusingly, there are indications that parts of the encyclical were written by AI.
To me, though, maybe the most notable part was that Chris Olah, who leads Anthropic's interpretability team, was standing next to the Pope at the ceremony, and delivered his own remarks. I felt like Chris, who I met even before Anthropic existed, was a non-obvious yet inspired choice here, one of the rare figures in frontier AI whose technical and moral authority are both completely unimpeachable by anyone.
And so, at this momentous era for the human project, and no less of an authority than that of the Vicar of Christ himself, the Supreme Pontiff and the Successor of Peter, I hereby throw myself on the wisdom and mercy of ... uhh, Chris Olah and his team at Anthropic.
Chris, if I am soon to share the earth with entities that can prove the Riemann Hypothesis and solve quantum gravity after 30 seconds of thought, then may you understand those entities well enough to cause them to be nice.
John Clark
> Regardless of how it turns out, it is going to be an interesting ride.
I am astonished that I am alive to see it happening.
Best wishes,
Keith
> --
> You received this message because you are subscribed to the Google Groups "extropolis" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to extropolis+...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/extropolis/CAJPayv0kuj6xAhCpZpWDtxKdS7G93W0%2BWEp1Tx8Vtq7ZdDPD8w%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups "extropolis" group.
To unsubscribe from this group and stop receiving emails from it, send an email to extropolis+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/extropolis/CAPiwVB5fFC8K_YC4mf5F0LC6GFeyVOmi6XhD26zSF4NTqguo2Q%40mail.gmail.com.
John Clark
2023: Pfft! AI can’t even do college-level math!
2024: Pfft! AI can’t even do PhD-level math!
2025: Pfft! AI can’t even do math research!
2026: Pfft! AI can’t even solve open research problems harder than the Erdős problems!
2027: Pfft! AI can’t even win the Abel Prize!
2028: Pfft! AI can’t even win a second Abel Prize!