If you've been trying to figure out whether Gemini 2.5 Pro is better than Claude Opus or GPT-5, you're not alone — and you're not confused because you're missing something. You're confused because the answer is genuinely complicated. After hundreds of tests in the 24 hours following its release, the honest answer is: it depends entirely on what you're doing, and there's a technical reason that almost no one is explaining clearly.
Is Gemini 2.5 Pro Actually Better Than Claude and GPT-5?
In almost any domain you care to measure, Gemini 2.5 Pro is competitive with the best models available right now — Claude Opus 4.6 and GPT-5. It leads on several prestigious benchmarks: it outperforms both on scientific reasoning (GPQA Diamond), academic knowledge (Humanity's Last Exam), general pattern recognition (ARC-AGI-2 at 77.1% vs Claude Opus's ~69%), and it hit a record ELO on LiveCodeBench Pro for competitive coding problems.
But here's where it gets interesting. On GDM-Hard — a broad measure of expert-level tasks that human professionals actually perform — Gemini 2.5 Pro falls noticeably behind Claude Opus 4.6 and even GPT-5. So how can a model win in coding benchmarks yet lag on expert professional tasks? That contradiction is the whole story.
Why Do AI Benchmark Results Always Contradict Each Other?
There is a real technical explanation for why every hot take you read on X, YouTube, or TikTok seems to contradict the previous one. It comes down to a fundamental shift in how AI models are trained.
A year ago, the pre-training stage — where models are trained on internet-scale data — was essentially the whole game. If a model was clearly better in one domain back then, it was likely better across most domains. That's no longer true.
Today, pre-training accounts for only about 20% of total compute spent on training. The other 80% happens in the post-training stage, where generalist models are intensively fine-tuned against internal benchmarks in specific domains, often using proprietary industry data. Dario Amodei, CEO of Anthropic, admitted just a year ago that spending on this second reinforcement learning stage was small for all players. That has completely changed.
The consequence? If a lab has domain-specific data and post-trains heavily on it, their model can dramatically outperform others in that narrow area — while potentially underperforming elsewhere. The era of one model being broadly better than another is over. It now depends almost entirely on your specific domain.
What Is Post-Training and Why Is It Changing Everything?
Post-training is the phase after a base language model is created. Labs take that generalist foundation and sharpen it using reinforcement learning against specific benchmarks, sometimes with industry-sourced data targeting niche professional domains. The more a lab invests here in your domain, the better their model will feel to you — regardless of what general benchmarks say.
This is also why Dario Amodei's broader thesis is worth paying attention to. He argues that if you specialize in enough specialisms, you eventually generalize to all specialisms. There are only so many patterns derivable from human training data. His bet is that Anthropic may not even need your domain's proprietary data — that sufficiently broad specialization will generalize broadly enough. Any remaining gaps might be filled simply by stuffing more context into an ever-larger context window. Claude 4.6 already supports 750,000 words; that could soon reach several million.
How Accurate Are AI Benchmarks for Real-World Use?
This is where things get genuinely uncomfortable for benchmark enthusiasts. Consider the chess puzzle benchmark from Epoch AI: Claude Sonnet 4.5 scored 12% five months ago. Claude Opus 4.6 — the bigger, newer, supposedly more capable model — scored only 10%. Chess is arguably one of the purest measures of forward-thinking general reasoning. In the old paradigm, you'd expect chess performance to predict broad capability. We're clearly not in that paradigm anymore.
Even within benchmarks, how you frame the question matters enormously. Models are brilliant at finding shortcuts. On the Simple Bench common-sense reasoning test, switching from multiple-choice format to open-ended answers caused scores to drop 15 to 20 percentage points. The models weren't cheating exactly — they were using any available signal (like the presence of "zero" as an answer option) to infer that a trick question was coming. That's not failure; it's pattern matching. But it does mean that benchmark scores and real-world performance can diverge significantly.
François Chollet, creator of the ARC-AGI test, raised an analogous concern about coding benchmarks: sufficiently advanced agentic coding starts to resemble machine learning itself. The agent iterates until it reaches a goal, producing a codebase that works but whose internal logic you can't necessarily inspect or trust. Overfitting to a spec, or drifting from your original concept, is a real risk even when benchmark ELO scores are at record highs.
What Is ARC-AGI-2 and Why Does It Matter?
ARC-AGI-2 was prominently featured in Google DeepMind CEO Demis Hassabis's launch announcement, and Gemini 2.5 Pro's score of 77.1% is genuinely impressive — well ahead of Claude Opus 4.6's ~69%. The test is designed to use puzzles that shouldn't appear in training data, making it a theoretically cleaner measure of novel reasoning.
However, AI researcher Melanie Mitchell flagged an important caveat: when the encoding is changed from numbers to other symbols, accuracy drops. Researchers found that the numbers representing colors in the input can inadvertently enable LLMs to exploit arithmetic patterns, arriving at correct answers through unintended shortcuts rather than true reasoning. The model isn't cheating — it's doing what LLMs do — but it's a sharp reminder that even well-designed benchmarks can be gamed in ways their creators didn't anticipate.
Does Gemini 2.5 Pro Hallucinate More Than Claude?
Hallucinations were supposed to be a solved problem by now. They are not. On the Omniscience benchmark from Artificial Analysis — which Google cited in their own release materials — Gemini 2.5 Pro leads overall with a score of +30, compared to Claude Opus 4.6's +11 and Claude Sonnet 4.6's -4. That scoring accounts for penalties on incorrect answers as well as rewards for correct ones.
But zoom in specifically on how models get questions wrong, and the picture shifts. When Gemini 2.5 Pro answers incorrectly, 50% of those wrong answers are outright hallucinations. Claude Sonnet 4.6 is down at 38% — meaning it's more likely to say "I don't know" rather than confidently make something up. China's GLM-5 model is even better at 34%. Being optimized for best-case performance does not protect you from worst-case failures. A model can be brilliant when it's right and still hallucinate frequently when it's wrong.
Have AI Models Finally Matched Average Human Intelligence?
On Simple Bench — a test of common-sense reasoning and trick questions administered to both models and humans — Gemini 2.5 Pro scored 79.6%, beating its predecessor and landing within the margin of error of the human average baseline. That human baseline came from nine participants, so it's not a massive sample, but the result is still worth sitting with.
The claim being made here isn't about superhuman intelligence or AGI. It's a more modest but arguably more meaningful threshold: in English text alone, it now appears that you cannot design a fair written test at which the average person would clearly outperform frontier AI models. Not exploiting tokenization quirks, not visual puzzles — just a straightforward, fair text-based test in English. We appear to have crossed that line.
Predictive forecasting benchmarks are approaching similar territory. Metaculus data shows model forecasting accuracy is nearly at the level of an average Metaculus human forecaster — not yet at professional superforecaster level, but closing in. The exponential improvement in genuine general capability is real, even if it's uneven across domains and partially obscured by benchmark noise.
The takeaway from all of this isn't that Gemini 2.5 Pro is overhyped or that Claude is secretly better. It's that we are in a new era where no single benchmark captures the full picture, where domain specialization has fragmented model performance in ways that make simple comparisons misleading, and where the models themselves are sophisticated enough to exploit any test you give them. The question of what general AI intelligence even means — and how to measure it — is going to be one of the defining debates of 2026 and 2027.
