New benchmark AhaBench tests whether AI agents actually learn from prior experience over long horizons; Claude Opus 4.6 and Gemini 3.1 Pro lead results

Researchers released AhaBench, a benchmark suite (Aha-Puzzle, Aha-Euler, Aha-Vending) designed to test whether AI agents actually retain and apply experience across long-horizon tasks, rather than just being scored on a single final trajectory. Across an eight-model panel, Claude Opus 4.6 led on aggregate Post-Experience Score (64.3) and Learning Lift (+25.8), with Gemini 3.1 Pro close behind (63.4). The paper, tasks, rubrics, validators and simulator code are all published on arXiv, making the results independently checkable.

This gives researchers and developers building long-horizon agents a more specific tool for comparing models on learning-from-experience rather than raw one-shot performance, which matters for anyone selecting a model for iterative or memory-dependent agent workloads. The more interesting finding is methodological: which model has the best "initial" score, the best post-experience score, and the best improvement (lift) are not the same model, so a single leaderboard number can mislead. That said, this affects evaluation practice, not deployed products, so the real-world impact is indirect and limited to a technical audience for now.

The "Claude Opus 4.6 leads" framing is a leaderboard hook and should not be read as a general capability verdict, since Gemini 3.1 Pro is within a point on the same metric and other models may lead on different sub-scores. There is also an unavoidable construct-validity caveat: the same team that built the tasks also built the scoring rubric, so independent replication matters before treating these numbers as settled.

1. 01Whether independent labs or other benchmark trackers reproduce the AhaBench numbers using the released tasks, rubrics and simulator code, since the authors both designed and scored the test
2. 02Whether AhaBench gets picked up by third-party leaderboards or model cards over the next few months, or joins the large pile of one-off agent benchmarks that see no repeat use
3. 03Whether the Initial/Post-Experience/Learning Lift decomposition (or something like it) starts appearing in other long-horizon agent evaluations, which would suggest the framing is catching on rather than just this one paper's branding