What Is ARC AGI 3 and Why Does AI Score Under 1%?
ARC AGI 3 is the most demanding AI benchmark ever released — and the numbers are humbling. Humans score 100% on these interactive puzzle environments, while the best frontier AI models currently score less than half a percent. Gemini 3.1, one of the strongest models available, managed just 0.37%. If you've been wondering whether current AI systems have truly reached human-level general intelligence, ARC AGI 3 offers the clearest answer yet: not even close.
Unlike previous benchmarks that tested static pattern recognition on grids, ARC AGI 3 is an interactive, game-like series of environments that simultaneously tests exploration, planning, memory, and goal-setting. Crucially, none of the goals are stated explicitly. There's no instruction telling you that the plus symbol rotates a shape, or that your objective is to make the bottom-left icon match the shape displayed at the top. Just like in real life, you have to infer the rules — and then remember what you've learned as the levels get progressively harder.
The benchmark's name gives away its core philosophy: the A in ARC stands for Abstraction. The puzzles deliberately avoid relying on language, memorized knowledge, or cultural cues. That's a deliberate design choice to prevent AI models from gaming the test through sheer training-data volume. As it turns out, that design choice is brutally effective.
Why Did Previous ARC AGI Benchmarks Get Saturated?
ARC AGI 1 and 2 were solved — or very nearly so — by frontier models in recent months. The paper explains two key reasons for that saturation. First, the public debut of chain-of-thought reasoning with OpenAI's o1 Preview in September 2024 gave models something resembling fluid intelligence: the ability to combine patterns on the fly rather than simply retrieve memorized answers.
But the second explanation is more unsettling. Because the public and private test sets of those earlier benchmarks were quite similar, labs could essentially generate thousands of variations of what the private test might look like and train on those. It's not direct memorization — it's a higher-level shortcut, almost like a systematic attack on the benchmark's integrity. The paper even notes that Gemini 3's chain-of-thought contained giveaways suggesting its training data included ARC AGI-like tasks, either accidentally or deliberately.
ARC AGI 3 counters this by using three separate distributions: a public set, a semi-private API-tested set, and a fully private competition set. These are intentionally different enough that brute-force task-space sampling won't help. The benchmark is, in the authors' own words, far less gameable.
How Does ARC AGI 3 Actually Score Models?
The scoring system is worth understanding because it's cleverly adversarial. Models are scored not on how many levels they complete, but on how many actions they took to complete those levels. Inefficiency is quadratically penalized — if you take 10 times the actions a human did, you don't score 10% of their performance, you score roughly 1%. If a model takes more than five times the number of actions compared to the human baseline, the attempt is scrapped entirely.
There's also a fascinating wrinkle in the human baseline itself: the benchmark uses the second-best human performance out of 10 trials as the 100% reference point. Which means even the third-best human might only score around 50% under the quadratic penalty system. That's a benchmark set up to expose deficiencies, not celebrate mediocrity.
Has AGI Actually Been Achieved? Jensen Huang Says Yes
Nvidia CEO Jensen Huang made headlines this week by claiming that artificial general intelligence has already been achieved. It's a bold statement — and one that sits in stark contrast to what ARC AGI 3 is showing us. The benchmark's authors hold a clear position: as long as a measurable gap exists between AI and human learning efficiency, AGI has not been reached.
There's a philosophical wrinkle worth noting here. Chimpanzees can track numbers flashing briefly on a screen better than most humans can. By the strictest definition of "matching human performance on all cognitive tasks," humans themselves might not qualify as AGI. But that rabbit hole aside, ARC AGI 3 makes a practical and concrete point: today's frontier models cannot adaptively learn within an environment the way humans do, even after a single play session. That's a significant and measurable gap.
What Is OpenAI's Spud Model and When Does It Release?
According to a recent report from The Information, OpenAI is preparing to release a new model internally referred to as Spud. Sam Altman has reportedly described it as "very strong," with a release expected within weeks. The ambition attached to it is significant — it's being positioned as a model that will meaningfully accelerate economic productivity.
Skepticism is warranted, of course. AI labs have a long track record of setting high expectations for upcoming releases. But what makes this particular claim more interesting is the context: OpenAI is apparently making real sacrifices to get Spud out the door.
Why Did OpenAI Shut Down Sora to Save Compute?
OpenAI has shelved its erotic chatbot project — reportedly after spending considerable resources developing it — and has dramatically scaled back Sora, its viral AI video generator. The reason, according to the Financial Times, is simple: compute scarcity. The company needs every available GPU cycle pointed at Spud and its broader AGI deployment goals.
Even internal OpenAI employees had flagged Sora as a drag on computing resources despite its cultural impact. The message from OpenAI's leadership appears to be that side quests are over. Everything is being consolidated into a single super-app focused on AGI deployment. Whether that laser focus produces the breakthrough they're promising remains to be seen, but the resource reallocation is real and significant.
Will Anthropic Revive Its Pentagon Deal for Claude?
A separate report from Axios adds another dimension to the "big AI leap" narrative. Anthropic has reportedly warned US government officials that its next major Claude release will significantly supercharge both offensive and defensive cyber capabilities. That warning appears to be reigniting Pentagon interest in a deal that broke down earlier this year.
According to the report, even after the very public fallout between Anthropic and the Department of Defense, a key negotiating official stated that the two sides remain "very close to an agreement." One interesting detail: Brad Gerstner, an adviser to Anthropic CEO Dario Amodei, is the architect of so-called "Trump accounts" — $1,000 savings accounts for every newborn whose parents enroll. The Axios piece speculates about whether Anthropic might partially fund such accounts as a goodwill gesture, representing an early, tentative step toward something resembling universal equity.
Is OpenAI Building a Fully Automated AI Researcher?
According to MIT Technology Review, OpenAI's north star for the next several years is building a fully automated AI researcher — one capable of tackling large, complex research problems independently. In the near term, the goal is an "intern-level" AI that can handle a small number of specific research problems by September.
The vision is that AI research will come to resemble software engineering today: AI does the grunt work, humans review the outputs. As one OpenAI leader put it, nobody edits code all the time anymore — they manage fleets of Codex agents instead. That framing will sound familiar to anyone who's watched the evolution of AI-assisted coding over the past two years.
But it's worth tempering the hype here. OpenAI's own GDP Value paper found that even when AI drafts first and humans edit — rather than vice versa — historical productivity speedups have been around 40%. That's meaningful, but it's not exponential takeoff. And counterintuitively, engineering job openings at tech companies have actually risen 50% over the past three years, from under 40,000 to 67,000 globally. AI first doesn't automatically mean human jobs last.
How Dangerous Are AI Agent Swarms? The Python Hack Explained
The final cautionary note from this week: a seemingly vibe-coded hack successfully co-opted a widely used open-source Python library. The result was that updating the library — or failing to catch an agent swarm doing so automatically — would export all your secrets and API keys to the dark web. It's a vivid illustration of what can go wrong when you grant AI agents broad autonomy over real systems.
As Nvidia distinguished scientist Jim Fan noted, AI agents need shells — probably many nested layers of them. Human oversight isn't just a nice-to-have right now; it's an active requirement. The irony is that many are proposing to fight this fire with fire, using agentic AI to detect agentic exploits. Meanwhile, as covered separately, some models are now smart enough to hack the benchmarks they're being tested on in real time.
That's exactly the messy middle we're in. Current AI is a better first drafter than most humans on many tasks, but its outputs are still riddled with holes. There's clear evidence of generalization on lower-level skills across coding and human languages. There's far less evidence of generalization on higher-order capabilities — academic integrity, adaptive goal-setting, or learning from a new environment on the fly. ARC AGI 3 measures precisely that gap, and right now, that gap is enormous. How quickly the incoming wave of new models closes it will define the story of 2026.
