Trending Now - New AI Benchmark Reveals System's Real-time Reasoning Limits
WHO:
ARC Prize Foundation
François Chollet, influential AI researcher and founder of the ARC Prize Foundation
Mike Knoop, cofounder of Zapier and member of the ARC Prize Foundation
Andy Konwinski, associated with the Laude Institute
WHAT:
The ARC Prize Foundation released ARC-AGI-3, an AI benchmark designed to measure on-the-fly reasoning.
ARC-AGI-3 consists of more than a thousand simple, video-game-like scenarios.
The benchmark tests AI agents' ability to reason through novel problems rather than just recall patterns.
A $1 million prize is available for creators of AI agents that achieve high scores.
The OpenAI o3 model, released in January 2025, scored between 75% and 87% on ARC-AGI-1.
The o3 model initially dropped to 3–4% on ARC-AGI-2.
Top scores on ARC-AGI-2 rose to 40–50% by December 2025.
WHERE:
The ARC Prize Foundation is associated with the AI research community, primarily located in the Bay Area.
WHEN:
The first ARC test was released in 2019.
The o1 model was released in September 2024.
The o3 model was released in January 2025.
ARC-AGI-2 was introduced in May 2025.
The release of ARC-AGI-3 occurred on Tuesday, March 24th, 2026.
WHY:
François Chollet argues that the field measures intelligence incorrectly, focusing on memorization rather than reasoning.
The benchmark aims to address the gap in measuring artificial general intelligence (AGI) and the limitations of existing benchmarks.
HOW:
AI agents must develop an understanding of the game environment and its rules without instructions.
Agents earn higher scores by reaching goals using fewer, more efficient steps.
AI labs began creating specialized software "harnesses" to improve ARC scores through repeated reasoning attempts.
(UPDATED March 26, 2026 00:40 UTC)
On Tuesday, March 24, 2026, the ARC Prize Foundation, led by influential AI researcher François Chollet, released ARC-AGI-3, a new benchmark designed to measure on-the-fly reasoning in AI agents through over a thousand simple, video-game-like scenarios. This benchmark challenges AI to solve novel problems rather than just recalling patterns, with a $1 million prize available for those who create agents that achieve high scores. The release follows previous benchmarks, with the OpenAI o3 model scoring between 75% and 87% on ARC-AGI-1, but dropping to just 3–4% on ARC-AGI-2 before top scores improved to 40–50% by December 2025.