Michael JohnsonCan an AI agent prepare an engineering report when the necessary information is scattered across a drawing, a spreadsheet, and a PDF? That is exactly the question the new MKA benchmark tests: it evaluates not a standalone answer about an image, but a complete workflow—from reading visually complex files to creating a standalone document. This distinction is useful for anyone choosing an agent for tasks involving presentations, diagrams, maps, or scans.
Google DeepMind introduced MKA on November 10, 2026, in collaboration with Artificial Analysis. The technical report describes 200 tasks spanning ten sectors and 83 professions. One key result for evaluating systems: when the model’s image-viewing tool was disabled, its performance on MKA fell while the number of agent turns increased. For tasks involving photographs and medical images, software workarounds did not make up for the loss of visual input.
What gap does MKA address?
Many tests of professional work assume that a model can extract text from files with a script and solve the task with little or no visual analysis. Separate image tests, on the other hand, often end with a short answer and do not require combining multiple sources or formatting the result as a working file. MKA is built around a sequence of actions: find elements across multiple files, combine them, apply the task’s rules, and submit the result in the required format.
Each task involves between two and 45 input files. These may include building plans, electrical diagrams, radiological images, maps, charts, spreadsheets, and text documents. In 88% of tasks, the agent must prepare or modify a separate file—for example, a PDF report, presentation, spreadsheet, or web application. According to the authors, converting materials into plain text can destroy important information, including scale, spatial layout, lines, and relationships between elements.
How is it evaluated?
Domain experts prepared the tasks, reference results, and evaluation rubrics. Each task has about 71 criteria on average. The authors say that for tasks requiring a separate output file, criteria are weighted 75% for factual accuracy and 25% for visual presentation. The final score takes the weights of individual criteria into account, so it evaluates a specific set of materials rather than just the plausibility of the text.
All the models being compared ran in the same Stirrup agent environment, with an identical set of tools and budget. The agent could open files, run code, and search the web. The technical report describes limits of up to 250 turns and eight hours per task. A separate evaluator agent checked the completed documents: it rendered files visually and could also inspect, for example, spreadsheet formulas programmatically.
The authors compared the automated evaluator’s scores with expert scores on a subset of 50 tasks. Agreement on criterion-level decisions was 96.2%; the study also reports a Cohen’s kappa coefficient of 0.91. This measures the evaluator’s agreement with experts on that subset.
What did disabling vision reveal?
The authors ran a controlled test with Gemini 4 Carbon: they disabled the image-viewing tool and compared its performance on MKA. The Climb-WSR score fell by about 5.9 percentage points, the number of turns increased by a factor of 1.28, and output token volume rose by a factor of 1.40. The model more often turned to software workarounds, trying to recover visual information through file processing.
The effect varied by input type. For clean drawings, workarounds partly restored performance, whereas for photographs and medical images they failed, according to the paper. This explains why “the model can read PDFs” is too crude a criterion for a working agent: extracted text can lose geometry, the relative positions of objects, and visual cues.
How were the models compared?
The study included Gemini 4 Carbon, Claude Fable 5.2, Claude Opus 5.5, and GPT-6 Astra. Claude Fable 5.2 achieved the highest Climb-WSR score, at 56.6%; Gemini 4 Carbon scored 54.3%. On the overall WSR scale, their scores were 73.3% and 71.9%, respectively. The authors report that Carbon used 5.7 times fewer tokens than Fable; the technical report also gives Carbon an API cost 6.3 times lower, with result quality within the margin of statistical noise. These comparisons apply to the specific tasks and conditions of MKA.
The authors define Climb-WSR as a measure based on difficult criteria on which previous-generation models did not reach the ceiling. The top systems scored less than 57% on this scale. This suggests there is room to improve in multimodal professional work. The numbers cannot be directly generalized to every profession or product task: results depend on the mix of files, requirements, and available tools.
What should developers and buyers take away?
MKA offers a practical template for an in-house evaluation. Instead of asking, “Which model works better with PDFs?”, assemble real tasks with input files, a required output document, and verifiable criteria. Include cases where the position of elements in a diagram matters, a spreadsheet must be reconciled with text, or the created file must be correct. Then compare systems in the same environment.
For this kind of test, it is useful to keep two separate scores: content correctness and artifact quality. A spreadsheet can contain the right numbers but the wrong formulas; a report can look polished but omit a relationship between a drawing and a specification. The MKA rubric shows how to formalize requirements in advance and make results verifiable, rather than relying on an overall impression of the answer.
The full set of 200 tasks is intended for evaluation on an independent platform; a separate set of ten demonstration tasks is available for exploring the format. This split keeps the main test closed while giving researchers material to study. For teams choosing an agent, the practical takeaway is clear: test not only text extraction, but also how the system uses visual information to produce the final work product.
