New benchmark ARC-Bench finds frozen JEPA world models rank candidate actions poorly, with closed-loop replanning masking the defect

Researchers released ARC-Bench, a new benchmark protocol designed to test whether frozen JEPA-style world models (used for robotic navigation and manipulation planning) correctly rank candidate actions by how good they actually are. Applying it to officially released checkpoints, including DINOv2, V-JEPA 1 and V-JEPA 2 at ViT-L and ViT-G scale, they found the top-ranked candidate action is almost always not the best one. Critically, they showed that when planners replan frequently in a closed loop, this ranking failure is hidden; cut the replanning frequency and success rates collapse.

This is a technical finding for researchers and developers building on latent-space world models, not a consumer or market event. It suggests that reported success rates for JEPA-based planners have been overstating how good the underlying action representations actually are, propped up by aggressive replanning rather than genuinely good rankings. Anyone building systems that adapt, amortize, or reduce replanning frequency around these models (for efficiency, latency, or cost reasons) should treat current benchmark numbers as less trustworthy until re-audited.

The claim that this "affects any method that adapts, amortizes, or replans around latent-space planners" is broader than what was actually tested, only specific navigation and manipulation checkpoints were audited. This is a single unreviewed paper from one team using their own new benchmark, with no independent replication yet. There is no deployment, product, or commercial impact here; this is an internal research-community correction, not an industry-shifting event.

1. 01Whether other labs (Meta FAIR, DeepMind, or academic groups) replicate the ARC-Bench results on additional JEPA checkpoints or architectures
2. 02Whether ARC-Bench or a similar action-rankability audit gets adopted as a standard evaluation alongside standard success-rate benchmarks in follow-up papers
3. 03Whether any production robotics or planning system reports a real-world performance regression traceable to reduced replanning frequency, which would validate the practical stakes beyond the lab setting