MetaGPT Proposes LAR Framework to Compress Action Chains and Cut Agent Inference Costs

MetaGPT Team Redesigns Agent Efficiency from the "Action Space" Perspective

Contributed by the LAR Team

QbitAI | WeChat Official Account: QbitAI

Large Language Model (LLM) Agents are rapidly evolving from simple question-answering systems to handling search, code generation, web navigation, and multi-turn tool calling.

However, as task complexity increases, a long-standing efficiency bottleneck has become increasingly prominent: Agents often need to generate lengthy action sequences to complete a single task.

Past optimizations have largely focused on model architecture, inference systems, context compression, or generation strategies, aiming to make each "step" of the Agent faster.

Yet, as long as the number of decision steps required to complete a task remains unchanged, the overall inference cost continues to grow with the length of the task chain.

In response, the MetaGPT team proposed a different approach: Instead of merely optimizing each step, directly redefine the Agent's "action unit."

They introduced Latent Action Reparameterization (LAR), which compresses frequently occurring, structurally stable multi-step text actions within an Agent's trajectory into learnable "latent actions."

In other words, instead of making decisions at the token level for every detail, the Agent can now select complete behavioral segments at a higher level, thereby shortening the effective decision chain and reducing inference costs.

This paper has been accepted by NeurIPS 2026.

Research Background: The Agent Bottleneck Is Not Just About "Generation Speed"

Current mainstream LLM Agents typically operate in a loop: the model reads environmental feedback, generates the next textual action, calls tools or interacts with the environment, and then continues generating subsequent actions.

The problem lies in the fact that this mechanism inherently uses tokens as extremely fine-grained decision units.

For example, a search Agent might repeatedly generate fixed thinking templates, tool call formats, system constraints, and structured sentences. While these contents occupy a large number of tokens, the elements that truly determine the task outcome are often just a few query terms, parameters, entities, or the final answer.

Therefore, even if single-token inference has been significantly accelerated through FlashAttention, KV Cache, quantization, or parallel systems, Agents may still need to execute very long "decision chains."

Based on this, the research team proposed a core insight:

Agent inference efficiency is determined not only by how fast the model runs, but also by how "actions are represented."

If a semantically stable, frequently recurring sequence of actions can be treated as a whole, the actual number of decisions the Agent needs to make can be significantly reduced.

This is precisely the starting point of LAR.

Core Technology: Redesigning Agents from the Action Space

Compressing a Sequence of Actions into a Single "Latent Action"

The core idea behind LAR is not complex.

A complete Agent trajectory often contains various types of content simultaneously, including thinking and planning, tool call formats, specific parameters, and interactions with the environment.

LAR identifies fragments within massive Agent trajectories that are high-frequency, low-entropy, and behaviorally stable, and re-encodes them into new latent action tokens.

In other words, fixed actions that previously required the Agent to "speak out" step-by-step can now be expressed directly through a new action symbol.

This change directly shortens the Agent's effective action horizon—the actual number of generative decisions required to complete a task. Behaviors that originally took multiple generation steps can now be reparameterized into higher-level action units.

However, there is a critical issue here: Not all content is suitable for reparameterization.

For action patterns that are low-entropy, structurally stable, and exhibit strong cross-context consistency, their behavioral semantics are usually fixed, allowing for safe abstraction. Conversely, high-entropy, strongly context-dependent information such as specific queries, entities, numerical values, and task parameters directly determines the result of the Agent's interaction with the external environment. If such content were also reparameterized into fixed actions, it could disrupt the original behavioral semantics and executability.

Therefore, LAR does not pursue "compressing as much as possible," but rather seeks an executable abstraction boundary between structural abstraction and semantic fidelity: it reparameterizes redundant structural actions as much as possible while preserving the key information that truly determines task behavior. This design ensures that both "abstraction" and "executability" are maintained.

The figure below illustrates the overall concept of LAR: reparameterizing raw, fine-grained action trajectories into more compact latent action trajectories, reducing effective decision steps while retaining key explicit information.

△ The overall framework of LAR. The system reparameterizes reusable, behaviorally stable action fragments into latent actions, shortening the effective action horizon while maintaining executability.

Automatically Learning an Action Vocabulary from Agent Trajectories

Unlike manually designed macro-instructions, LAR's latent actions are learned directly from existing Agent trajectories.

The entire process can be summarized in four steps:

First, repeated text fragments are extracted from trajectories and filtered based on frequency and next-token entropy. Low entropy indicates that the behavior following a fragment is relatively stable, suggesting it likely corresponds to a reusable action structure.

Next, the filtered fragments are added to a new latent action vocabulary, with each fragment corresponding to a new vocabulary symbol.

Then, the training data retains both the original trajectories and the compressed trajectories: the original version serves as the teacher, while the reparameterized version serves as the student input.

Finally, through trajectory-level distillation, the model is trained so that when it sees only the reparameterized trajectory, it can still recover the prediction distribution of the original model on the non-reparameterized content. This forces the latent actions to "capture" the semantics originally carried by the replaced fragments, ensuring consistent subsequent predictions and gradually learning functional replacements for the original action fragments.

It is worth noting that LAR does not require retraining the entire model. In experiments, only LoRA parameters and the newly added latent action embeddings are trained, accounting for approximately 0.1% of the model's total parameters, while all other pre-trained weights remain frozen.

During the inference phase, these latent actions undergo embedding lookup and Transformer forward passes just like ordinary tokens, requiring no additional expansion or post-processing. Consequently, they introduce almost no new inference overhead.

Experimental Results: Fewer Decisions While Maintaining Task Capabilities

Fewer Action Tokens, While Maintaining or Even Improving Performance

The research team validated LAR across different models, including the Qwen and Llama series, covering typical Agent scenarios such as multi-step reasoning, code generation, and web interaction.

The comparison included traditional Agent baselines, as well as representative methods that improve inference efficiency from various perspectives, such as generation control and context compression.

Overall results demonstrate that LAR achieves stable "performance–efficiency" gains across different models and tasks: in most settings, the model maintains or even improves task performance while reducing action tokens, with reductions in action tokens reaching 20%–27% in some scenarios.

More importantly, these benefits are not limited to a single type of task. Whether in multi-step reasoning, structured code generation, or web interaction, LAR can shorten the effective decision chain to varying degrees while maintaining strong task completion capabilities.

This means LAR is not simply about "generating less text," but rather about reducing unnecessary low-level decision steps while preserving key semantics as much as possible.

The table below presents the main experimental results of LAR across different models and various Agent tasks, comparing changes in task performance and action tokens.

Real Gains at the System Level

The ultimate value of reparameterizing the action space depends on whether it translates into tangible inference benefits at the system level.

Experimental results show that under different task and model configurations, LAR not only improves token throughput but also reduces peak GPU memory usage in certain scenarios, thereby enhancing overall inference efficiency.

This indicates that latent actions do more than just change the textual representation. As the effective action horizon shortens, shorter trajectories correspondingly reduce computational and storage overheads during prefill, KV Cache management, and subsequent inference processes, thus converting abstraction at the action space level into actual system-level benefits.

The following table further illustrates LAR's efficiency performance at the system level, including metrics such as token throughput, GPU utilization, and peak memory usage.

Can Latent Actions Generalize to Unseen Tasks?

If latent actions merely memorize surface patterns within a specific dataset, their practical value would be very limited.

Therefore, the team further tested held-out benchmark generalization: learning latent actions using existing tasks without additional training, and directly transferring them to unseen new tasks.

Experimental results indicate that LAR maintains strong transfer capabilities on new task distributions, matching or exceeding corresponding baselines in multiple settings.

This suggests that what the model learns is not just string patterns specific to a particular benchmark, but more reusable action structures and behavioral patterns. In other words, latent actions resemble a transferable high-level action representation rather than simple memorization of fixed fragments from the training data.

The table below shows LAR's transfer results on new tasks not involved in training, verifying whether latent actions have learned reusable high-level behavioral structures.

More Compression Isn't Always Better: The Team Identified an "Abstraction Boundary"

LAR yielded another interesting finding: There is a clear abstraction boundary in the Agent's action space.

After gradually increasing the reparameterization ratio, the research team observed that performance changes could roughly be divided into three stages:

1. Stage One: As low-entropy structures are compressed, both model efficiency and task performance improve simultaneously.
2. Stage Two: Performance reaches its peak, indicating that most stable structures safe for abstraction have been compressed.
3. Stage Three: If compression continues to cover high-entropy information such as specific queries, entities, numbers, and task parameters, model performance drops suddenly.

This decline does not happen gradually; it may manifest as a distinct "collapse."

The reason is intuitive: once latent actions begin to encompass parameters that truly determine environmental behavior, a single latent action can no longer represent stable, unique action semantics, and the original behavioral equivalence with the environment fails.

Thus, LAR's goal is not to maximize token compression, but to find the largest possible range of action abstraction that remains executable.

The figure below illustrates the trend of Agent performance ("improvement—peak—decline") as the degree of reparameterization increases.

△ Progressive Action Abstraction Experiment with LAR. Across different tasks and model configurations, performance consistently exhibits an "improvement—peak—decline" trend as the degree of reparameterization increases, demonstrating a clear executable abstraction boundary in the Agent's action space.

Scalability: From Larger Models to Industrial-Grade Agents

To verify whether LAR is effective only in specific models and experimental environments, the team further tested its scalability from two directions.

On one hand, when scaled up to larger models (8B → 32B), LAR still maintained stable performance gains, indicating that action reparameterization does not depend on a specific parameter scale and possesses good generalization across model sizes.

On the other hand, the team applied LAR to industrial-grade Agent runtimes like OpenClaw. Without modifying existing tool interfaces, interaction loops, or execution logic, LAR was still able to achieve benefits by reparameterizing stable, redundant structural content.

This implies that LAR functions more like a relatively independent Agent optimization mechanism: it can scale to larger models and integrate into existing Agent systems, working in conjunction with model architecture optimizations, inference engines, and system-level accelerations.

In other words, while previous research primarily focused on "how to make each forward pass faster," LAR proposes a different direction:

Can we enable Agents to avoid making unnecessary low-level decisions from the start?

Conclusion

As LLM Agents take on increasingly long-chain tasks, inference efficiency is shifting from being solely a model problem to becoming a comprehensive decision-system problem.

Proposed by the MetaGPT team, LAR re-examines Agent inference from the perspective of action representation: it reparameterizes reusable structural behaviors into latent actions, allowing the model to plan and execute directly within a higher-level action space.

Experiments demonstrate that this approach can shorten the effective decision chain across different model scales and various Agent tasks, delivering tangible throughput and memory benefits while maintaining or even improving task performance.

More importantly, this work reveals a key principle in Agent action abstraction: Effective compression is not about hiding everything, but distinguishing which structures can be abstracted and which information must be explicitly retained.

In the future, action space learning can be further combined with Agent memory, planning, reinforcement learning, and inference system optimizations. Compared to merely pursuing faster token generation, redesigning Agents starting from the "decision unit" itself may become a crucial path for improving efficiency in long-chain tasks, high-frequency tool calls, and large-scale Agent deployments.

Paper Title: Latent Action Reparameterization for Efficient Agent Inference Authors: Members from DeepWisdom (MetaGPT), Yale University, Université de Montréal, University of Sydney, Fudan University, University of Illinois Urbana-Champaign, Amazon Science, HKUST (Guangzhou), and Stanford University. Paper: https://arxiv.org/pdf/2605.18597 Code: https://github.com/EZ-hwh/LAR

Like, Share, and Favorite!

We welcome your thoughts in the comments section!

— End —

[Academic Submissions] Please send emails on weekdays to: [email protected]. Include [Submission] in the subject line and tell us: Who you are, Where you are from, The content of your submission with a link to the Project/Homepage, and your Contact information.

🎓 We will try our best to reply to you promptly :)

🌟 Star Us 🌟

Daily Updates on Cutting-Edge Tech Progress

Read Original

Open in WeChat