DeepMind's AI system, AlphaProof, just solved unsolvable math problems — nine of them, to be exact — drawn from a legendary collection of puzzles that no human mathematician had cracked in over 50 years. The cost? A few hundred dollars per problem. The method? Letting an unreliable AI fail thousands of times inside a tournament loop until something correct finally emerges. It sounds almost too simple, and yet it worked. Here's the full breakdown of how DeepMind pulled this off, why it matters enormously, and what the limitations are that mainstream coverage conveniently skips.

How Did DeepMind's AI Solve 50-Year-Old Math Problems?

The short answer: it didn't solve them on the first try. Or the hundredth. AlphaProof works by running an AI agent repeatedly against a strict formal verifier, using a cleverly designed tournament to gradually improve the quality of its proof attempts until one finally checks out.

The key insight is that the system doesn't need the AI to be smart enough to solve the problem outright. It needs the loop around the AI to be tight enough that correct solutions bubble up naturally over many iterations. That's a fundamentally different philosophy from how we used to think about AI problem-solving.

Here's the step-by-step process: A mathematician first writes the problem in Lean, a formalized mathematical language where proof correctness can be automatically verified. The proof itself is left blank. The AI agent then attempts to fill in that proof. It fails — these are extraordinarily hard problems. But a second AI checks the attempt and, crucially, explains why it failed.

Then comes the genius part. A cheaper judge AI reads two previous solution attempts — both potentially wrong — and picks the better one. That winning attempt gets an ELO score, the same rating system used in chess, named after fellow Hungarian Arpad Elo. Every proof attempt now has a numerical ranking. The system doesn't restart from scratch; it restarts from the highest-scoring failed solution. Run this tournament over and over, and eventually the validator signs off on a correct proof.

What Is AlphaProof and How Does It Actually Work?

AlphaProof is DeepMind's AI system designed specifically for formal mathematical reasoning. Unlike a general-purpose language model that might confidently hallucinate a plausible-sounding but completely wrong proof, AlphaProof is anchored to Lean, a programming language for mathematics where every logical step can be machine-checked.

This is not entirely new on its own — many research groups use formal verification systems. What's new is the tournament architecture wrapped around it. Think of it as a reliability engine built from unreliable parts. The AI at the core still makes things up, still gets things wrong, still fails constantly. But the harness around it — the ELO-scored proof tournament with a judge that cannot fabricate a verification — filters out the garbage systematically.

The result is a system that is, in total, far more reliable than any individual component. That's a genuinely elegant engineering achievement.

What Are the Erdős Problems That Stumped Humanity?

Paul Erdős was a legendary Hungarian mathematician who, over his career, deliberately left more than 1,200 open problems for the mathematical community to solve. Some came with cash prizes attached. Many remained unsolved for decades — not because mathematicians didn't try, but because these problems were fiendishly hard, sitting at the edge of what human mathematical intuition could reach.

DeepMind tested AlphaProof on approximately 350 of these problems — a curated subset that was more amenable to formalization in Lean. Out of 350 attempts, the system solved 9. That's a 95.7% failure rate, which sounds terrible until you remember that these problems had a 100% failure rate among humans for the past 50 years. Solving even one of them would have been remarkable. Nine is stunning.

Why Do AI Systems Fail at Formal Math Proofs?

Standard large language models are pattern-matching engines trained on human text. When asked to produce a mathematical proof, they generate something that looks like a proof — it has the right structure, uses the right vocabulary, follows plausible logical steps — but the underlying reasoning is often subtly or catastrophically broken. This is hallucination in its most dangerous form: confident-sounding nonsense.

Formal mathematical languages like Lean eliminate this problem at the verification stage. You cannot fake a proof in Lean. Either every step is logically valid and machine-checkable, or the system rejects it. This gives AlphaProof an honest judge — something a free-running language model entirely lacks.

The tradeoff is that writing problems in Lean requires significant human expertise upfront, and not all mathematical problems translate cleanly into formal language. That's part of why the team started with a subset of 350 Erdős problems rather than all 1,200.

How Does an ELO Tournament Make AI Proofs Reliable?

The ELO scoring system, originally designed for chess, assigns a numerical rating to each player based on match outcomes. Better players accumulate higher scores; weaker players fall. AlphaProof applies this exact logic to proof attempts.

Every failed proof gets an ELO score based on how it performs against other failed proofs in head-to-head comparisons judged by the cheaper judge AI. This means even wrong answers aren't treated equally — some wrong answers are closer to right than others, and the system learns to prefer them.

When the next iteration begins, it doesn't start from zero. It starts from the best-scored previous attempt. Over thousands of rounds, this guided search gradually climbs toward a correct proof. It's a tournament where the prize isn't a trophy but a formally verified mathematical truth.

How Fast Is AI Actually Getting Better at Math?

The trajectory here is almost uncomfortable to look at directly. Four years ago, GPT-3 couldn't reliably add numbers. Two years ago, AI couldn't reliably solve high school competition math. One year ago, it couldn't reliably win a Mathematical Olympiad gold medal. Today, it's solving 50-year-old open problems from one of history's greatest mathematicians.

Each of these milestones was met with the same critique: yes, but it can't do the next hard thing yet. And each time, that critique became obsolete within a year or two. There's a useful mental model here — instead of evaluating where AI is right now, ask where it will be two research papers from now. The answer, consistently, is further than most people expect.

What's also changing is the nature of the progress. For years, the story was: make the model smarter. Now the story is also: make the harness around the model more rigorous. The intelligence isn't only inside the neural network anymore. It lives in the loop — in the tournament structure, the judge, the verifier, the feedback mechanism. That's a new and important frontier.

What Are the Real Limitations of AlphaProof?

Mainstream coverage tends to skip this part, so let's be honest about what AlphaProof doesn't do.

  • Selection bias in the problem set: The 350 problems tested were chosen partly because they were easier to formalize in Lean. The full 1,200 Erdős problems were not tested. This doesn't invalidate the result, but it means the 9 solved problems represent the more accessible end of an already hard problem set.
  • Smaller models solved zero problems. Not fewer problems — zero. The system only works with large, computationally expensive frontier models at its core. This matters when benchmarks show smaller models performing "almost as well" on standard tests. In practice, the gap is apparently much larger for genuinely hard tasks.
  • Cost per problem is still real. A few hundred dollars per solved problem sounds cheap for a 50-year-old open question, but scaling this to thousands of problems across all of mathematics adds up quickly, especially if many problems yield zero solutions.
  • Formalization bottleneck: Someone still has to translate problems into Lean. That requires mathematical expertise and time. The AI can't yet fully automate its own problem intake.

None of these limitations diminish what was achieved. An AI just solved nine mathematical problems that stumped humanity for half a century, for a few hundred dollars each, by building a reliable system out of an unreliable one. That's not just impressive — it's a preview of where this entire field is heading.