Google DeepMind and Artificial Analysis launch MKA, a 200-task benchmark for agents on visually complex knowledge work; Gemini 4 Carbon ranks #2 at about 6x lower cost than the top model

Google DeepMind and Artificial Analysis launched the Multimodal Knowledge Agent (MKA) benchmark. It has 200 private tasks across 10 verticals and 83 occupations. Each task takes a professional 3-8 hours and is scored on about 70 yes/no checks (75% factual accuracy, 25% visual design). Models run in the open-source Stirrup harness and are graded by an agentic autorater built on a non-Google model, which Google says agreed with expert graders 96% of the time. Google reports that Gemini 4 Carbon ranks #2 on quality at about 6x lower average cost than the top scorer. Ten public tasks and a technical report are available. The full ranking and the per-model costs were not in the material provided.

Teams buying or building agents for document, slide and report work get a new test aimed at visual output, which text-only benchmarks miss. The cost claim is the practical part: if it holds, Gemini 4 Carbon could be a cheaper option for this kind of work. The open harness and public sample tasks let others check the setup, though the 200 scored tasks are private. For now it is a data point to test against your own workloads, not a reason to switch models.

Google helped build a benchmark on which its own model places #2 and wins on cost, so the headline is self-reported and not independent proof. "6x lower cost" depends on the harness, the number of turns (30 to 130 per task) and token pricing, and it is compared only with the top scorer, not with every rival. The 96% autorater agreement is Google's figure and does not show the grader is unbiased on visual design, which is only 25% of the score.

1. 01Whether Artificial Analysis publishes the full MKA leaderboard with per-model cost and token use, and whether the Gemini 4 Carbon #2 rank and 6x figure hold on its independent page.
2. 02Whether outside groups run the Stirrup harness and the 10 public Hugging Face tasks and get similar rankings, especially with a different autorater.
3. 03Whether rival labs report MKA scores or dispute it in the next few weeks, and whether real customers report Gemini 4 Carbon matching the benchmark quality on their own documents at a lower bill.