He Kaiming Team Releases VISTA: Visual-Native Harness Framework Enables Claude Opus 5.0 to Score Perfectly on ARC-AGI-3

A Visual-Native Harness for Multimodal Models Has Arrived!

By henry, from Ao Fei Si QbitAI | WeChat Official Account: QbitAI

A Visual-Native Harness for Multimodal Models Has Arrived!

Recently, Kaiming He’s team introduced a visual-native harness—VISTA—in their latest paper, VISTA: A Visual Harness for Reasoning in an Interactive World.

It enables existing multimodal models to directly observe environments, store raw frames, and revisit, zoom in, and inspect details during reasoning—all without requiring retraining from scratch.

Unlike traditional approaches that abstract the environment into text or code, VISTA preserves raw visual information. The model can retrieve previously seen frames back into its context based on the current problem, allowing them to participate in ongoing judgment and reasoning.

With this method, visual experience becomes context that the model can call upon at any time and repeatedly verify. Multimodal models now have a native scaffold built around visual memory.

In evaluations, paired with VISTA, Claude Opus 5.0 completed all 25 public games in ARC-AGI-3, achieving a perfect score of 100 relative to human action efficiency. It used 57.4% fewer game actions than the baseline of humans playing for the first time.

Switching to GPT-5.6 Sol also resulted in completing all tasks, with a score of 99.

Furthermore, the team validated this approach in browser games, mazes, and connection puzzles, listing embodied tasks closer to the physical world as future research directions.

How was this achieved?

A Visual-Native Scaffold

From ResNet and Mask R-CNN to MAE, visual perception and representation learning have always been a central thread in Kaiming He’s research.

This time, VISTA addresses a new question: How can models accumulate, preserve, and utilize visual experience during continuous interaction?

The answer is a Harness.

Last November, Anthropic used long-duration programming tasks as an example to illustrate how a Harness helps models sustain progress across multiple context windows.

Specifically, the Harness records pending items via task lists and saves completed work through progress files and code logs.

This way, even when the model enters a new context window, it can read this information to understand what has been done and what needs to be done next.

Complex tasks can thus be executed step-by-step, checked for results, and continued seamlessly across different context windows.

One could say that this Harness, which primarily preserves task state through text and code, has significantly driven the recent explosion in Agent capabilities.

However, in real-world environments, text-only memory is clearly insufficient.

Once entering a true visual environment, the model must not only remember object positions and orientations but also understand exactly what changes occurred in the environment before and after each action.

Moreover, when the model sees an image for the first time, it may not know which details will become important later.

At the same time, these visual details are difficult to fully capture in advance using text notes, making it hard to guarantee that the recorded information remains sufficient for subsequent task execution.

In benchmarks like ARC-AGI-3, one existing approach involves converting screens into text grids composed of numbers, then having the model write programs to simulate environmental rules and validate action plans.

(Note: ARC-AGI-3 is an interactive visual reasoning benchmark that does not provide game rules or goals upfront; agents must figure out how to win through observation and action.)

The problem is that as environments become more complex, object appearances, spatial relationships, and dynamic changes become increasingly difficult to fully translate into text and code. Additionally, as interaction history grows longer, early frames are gradually pushed out of the context or replaced by textual summaries.

Thus, the research question for VISTA became:

How can we enable agents to re-examine past visual information when needed during reasoning, without additional training of the base model?

Based on this, VISTA stores every frame returned by the environment outside the context window as-is—including intermediate animation frames during actions—and retrieves them when the model needs them.

In ARC-AGI-3 evaluations, it can both compare frames from different moments and zoom into local areas to inspect orientation markers on small blocks.

Here, text notes record the model’s current understanding, while raw frames preserve evidence for re-evaluation.

The study refers to this mechanism as explicit attention over interaction history, where the model selects which visual information re-enters the context based on the problem it is currently considering.

To make such selection possible, the system must store raw frames and provide tools for retrieving and inspecting them at any time.

This way, the model does not need to pre-determine which information is worth remembering. Instead, it can fully preserve visual experiences for later use.

Finally, there is active visual inspection, which allows the model to review historical frames on demand.

Using the inspect tool, the model can specify a particular historical turn to retrieve the corresponding frame, crop or zoom into local regions, and examine details within them.

If the model wants to know exactly what changed during an operation, it can retrieve multiple frames at once to compare the differences before and after the action.

The entire process is akin to equipping the model with a visual archive that can be consulted at any time. It not only sees the current environment but also actively reviews past observations to inform subsequent actions.

So, how do these three components work together?

According to VISTA’s execution flow, at the start of each turn, the model first observes the current frame and available actions. It then combines this with previously accumulated experience to assess the current state of the environment and propose a hypothesis for the next action.

If existing information is insufficient, the model can invoke tools to review historical frames, zoom into local areas, or even read specific pixel data to gather further evidence.

After finding evidence, the model does not act immediately. Instead, it first predicts what changes the operation might bring about.

Once the action is executed, it compares the actual result with its prediction to verify whether its judgment was correct, thereby refining its understanding of the game rules.

Through this iterative process, the model explores the environment and accumulates experience through continuous interaction.

Of course, a visual archive alone is not enough. To ensure coherence in long-horizon tasks, VISTA introduces two text-based notes:

• GUIDE.md: Records rules and experiences reusable across different levels.
• WORKING.md: Tracks the current level’s status, progress, and future plans.

When the context window approaches its limit, the model generates a handover summary before moving to a new context window to continue the task. Previous text notes, action history, and visual archives are all preserved.

Another noteworthy design aspect is that while VISTA fully preserves historical frames, it does not dump all images directly into the model’s context.

In the full implementation, after each action, the framework defaults to showing only the last frame to the model. Intermediate frames from the action process are stored uniformly in the visual archive, retrieved by the model only when needed.

This approach retains complete visual information while avoiding the excessive consumption of context space by large volumes of historical images.

More importantly, VISTA does not require training a new model specifically for this purpose.

Environment understanding, action planning, and reasoning are still handled by off-the-shelf multimodal models. The Harness is responsible for executing tool calls, saving visual history, and providing relevant evidence when the model needs it.

In other words, VISTA does not change the model itself, but rather how the model acquires, stores, and uses visual information.

Experimental Validation

Beyond the experiments mentioned earlier, the team tested VISTA on three benchmarks: GameWorld, AI GameStore, and BabyVision, covering tasks such as browser games, mazes, and line-connecting puzzles.

Using the same GPT-5.6 Sol model, compared to the official baseline framework, VISTA improved the success rate on 170 tasks in GameWorld from 40.0% to 63.3%.

On 10 games in AI GameStore, the composite score rose from 47.3 to 140.3, where the human median was normalized to 100.

On 39 maze and line-connecting problems selected from BabyVision, accuracy increased from 41.0% to 63.2%. Notably, this final set of tasks consisted solely of static images; yet the model could improve its judgments by zooming into local areas and inspecting pixels.

These results demonstrate that preserving raw frames and allowing the model to review them on demand can further unlock the capabilities of existing multimodal models.

The same VISTA framework requires only minor adaptations to apply to different games and puzzles, showcasing its potential for broader visual tasks.

In conclusion, the research points toward future validation directions involving more physically grounded embodied tasks: enabling models to save, review, and utilize their visual experiences in continuously changing environments to decide on next actions.

Author Introduction

Finally, let us introduce the authors of this work.

The paper has three co-first authors: Qiushi Han, Keya Hu, and Linlu Qiu. The other two authors are Cathy Wu and Kaiming He.

Qiushi Han (Josh Han) is currently a PhD student at MIT’s Center for Operations Research, advised by Cathy Wu.

Keya Hu is currently a PhD student in the Department of Electrical Engineering and Computer Science at MIT, jointly advised by Kaiming He and Jacob Andreas.

She graduated from the ACM Class at Shanghai Jiao Tong University for her undergraduate studies. Her research interests focus on the intersection of language and vision.

Kaiming He is currently a tenured Associate Professor in the Department of Electrical Engineering and Computer Science at MIT. He is the lead author of seminal works such as ResNet, Mask R-CNN, and MAE.

He received his bachelor's degree from Tsinghua University and his Ph.D. from The Chinese University of Hong Kong. Prior to joining MIT in 2024, he worked at Microsoft Research Asia and Facebook AI Research (FAIR).

References [1]https://arxiv.org/pdf/2610.02200