NeurIPS'26 Study: Poor Multi-Agent Communication Configs Drop Accuracy by 23.6%

Sufficient Information ≠ Useful Information

Submitted by Bo Liu's Research Group at the University of Arizona

QbitAI | WeChat Official Account: QbitAI

In current multi-agent systems, a primary agent typically holds the complete context and decomposes tasks by sending localized task messages to sub-agents, driving progress step-by-step.

However, in practical applications, even when the condition of "information sufficiency" is met, sub-agents may still take incorrect actions. With the same communication configuration, switching models or altering task states can lead to drastically different accuracy rates.

Fundamentally, existing systems often treat "information sufficiency" as a sufficient condition for successful collaboration, overlooking the implicit costs inherent in the communication configuration itself.

Researchers from the University of Arizona, Hong Kong Baptist University, and Amazon Web Services investigated this issue in their NeurIPS 2026 paper. They found that in multi-agent collaboration, the utility of a message depends not only on what it provides but also on how the receiving model processes that information.

The team had an 80B parameter Qwen model send messages to a 9B parameter model, with a length limit of 128 tokens. On ContextBench, which involves locating relevant files within codebases, next-step accuracy increased by 14.2 percentage points. However, when switched to ToolSandbox, a tool-calling benchmark, the same configuration caused accuracy to drop by 23.6 percentage points.

The study explains this discrepancy through both theoretical analysis and experiments, proposing a strategy that requires no model retraining: at each step, the system decides whether to communicate, how to communicate, and whether to switch the receiving model, based on the specific task and the capabilities of the receiver.

How Should the Primary Agent Inform the Sub-Agent?

In programming and tool-calling tasks, the primary agent usually possesses more comprehensive access to the code repository, execution logs, and task background. The sub-agent, relying on local information, decides which file to modify or which tool to invoke next.

The most direct approach is to send all raw materials. However, the sub-agent must still identify key points itself, and irrelevant content within long contexts may interfere with its judgment.

An alternative is to compress the information into short messages. While this clarifies the objective, two issues arise: Are critical details omitted? Can the sub-agent correctly utilize the information after it has been rephrased?

Looking solely at final task completion makes it difficult to isolate these causes: failure might stem from the current message or from error accumulation in previous steps. Therefore, the team narrowed their analysis to individual interactions, examining specifically how messages influence next-step decision-making.

Why Do Errors Persist Despite Complete Information?

The term "Bayes-Sufficient" in the paper title can be understood as follows: The compressed message, combined with the receiving model's existing knowledge, retains all information necessary to determine the target outcome.

It resembles the line from The Shawshank Redemption: "Salvation lies within." The answer is present in the message, but the receiving model may fail to find it.

From an information-theoretic perspective, such compression is sufficient. However, actual models must read the message, extract the goal, and select an action—these steps do not happen automatically.

△ Both expressions contain the information needed to decide "whether to bring an umbrella," yet they may lead the recipient to make different decisions. This is an illustrative case constructed by the authors, not real model output.

The team theoretically constructed a class of tasks where, for receiving models constrained by expression formats, raw materials support zero-error decision-making; however, compression that preserves decision-relevant information results in an error rate lower bound approaching 50%.

This example demonstrates that preserving information does not equate to preserving the model's ability to use that information. Evaluating compressed messages must also account for the receiving model's constraints regarding expression format.

How Much Does a Single Message Improve Accuracy?

The paper decomposes the actual improvement brought by communication into two parts.

The first part is the informational gain provided by the message: under given constraints (model, prompt, context, etc.), how many fewer errors could the receiving model theoretically make?

What the receiving model actually achieves is often below its best possible performance under those constraints. How communication alters this gap constitutes the second part: messages may increase comprehension difficulty or introduce noise, widening the gap; alternatively, they may clarify the objective, narrowing the gap. The paper terms this change in the gap the protocol tax.

Subtracting the protocol tax from the informational gain yields the net benefit of communication. Consequently, messages with more complete content may ultimately reduce accuracy.

The choice between sending short messages versus raw text can also be analyzed this way (the paper decomposed the impact of compression under log-loss): Short messages reduce processing burden but may omit information or use expressions that are difficult for the receiving model to interpret.

Neither "shorter is better" nor "more complete is better" serves as a standalone selection criterion.

Technical Breakdown: Diagnose Error Sources Before Deciding Communication Strategy

First, distinguish between information omission, comprehension failure, and action selection error.

The team categorized errors into three stages:

• Not Written Out: The primary agent possessed the information but failed to include it in the message.
• Not Perceived: The information was present in the message, but the sub-agent failed to extract it.
• Wrong Action Selected: The correct file or tool was identified, but the subsequent action was chosen incorrectly.

△ These three stages are referred to in the paper as externalization, absorption, and action closure, respectively.

Second, analyze when it is worthwhile to upgrade to a stronger primary agent.

A stronger primary agent may provide more effective information but might also generate more complex messages that are harder to process.

The paper presents a conditional theoretical result: If the net benefit of upgrading the primary agent varies continuously with task difficulty and transitions from negative to positive only once, there exists a unique difficulty threshold above which upgrading yields benefits.

Experiments used "the sub-agent's independent accuracy in completing the next step" as a proxy for task difficulty. However, difficulty does not explain all outcomes; the subsequent τ-bench experiments serve as an exception.

Third, select communication methods at inference time without retraining models.

The proposed strategy includes five options:

• Skip communication;
• Send a compressed message;
• Send raw context;
• Switch to a stronger receiving model;
• Update shared task state (e.g., current plan and file modification status).

The strategy estimates the informational gain, processing burden, current cost, and future risk for each option based on pre-calibrated rules before making a selection. Calibration uses independent records, ensuring ground truth answers are not accessed during testing.

Additionally, there is a method to directly save API calls: if independent calibration indicates that the sub-agent can achieve 80% accuracy on certain task types without messages, communication is skipped entirely, avoiding the need to call the primary agent to generate a message.

Six Benchmarks Show Inconsistent Communication Effects

Experiments covered six benchmarks: ContextBench (code context localization), ToolSandbox (stateful tool calling), SWE-bench Lite (code repair), RepoBench (code retrieval and completion), τ-bench (simulated customer service), and BFCL (function calling).

When comparing communication effectiveness, we use the next-step joint accuracy: a prediction is considered correct only if both the target file or tool and the action type are selected correctly.

First, the same communication configuration yields opposite effects on different tasks.

When Qwen3-Next-80B-A3B sends messages and Qwen3.5-9B receives them, ContextBench accuracy rises from 63.3% to 77.5%; however, ToolSandbox accuracy drops from 88.9% to 65.3%.

△ The two tests comprise 480 steps and 72 steps, respectively. Despite using the same model combination, the impact of communication was opposite in each case.

The choice of model combination also affects results: on ToolSandbox, switching to gpt-5.4 as the sender and gpt-4o-mini as the receiver increases accuracy from 80.3% (with no communication) to 100%.

Second, changing only the expression while keeping the information intact can still reduce accuracy.

The team re-encoded messages to preserve decision-relevant information, altering only field names and action labels before passing them to GPT-series models for processing.

Among samples that were originally completed correctly with the original format, accuracy dropped by 13.5, 6.0, and 3.5 percentage points on τ-bench, ContextBench, and ToolSandbox, respectively; BFCL, which already uses highly structured formats, showed no decline.

Third, even after identifying the correct file, different models may choose different actions.

The team passed identical messages generated by DeepSeek-V3.1 directly to eight receiving models, fixing the message content to isolate and compare individual model performance. The proportion of cases where they identified the correct file ranged between 61% and 74%, showing relatively small differences.

However, among samples where the correct file was identified, the rate of selecting the correct action varied drastically: DeepSeek-R1, Qwen3.5-9B, and Llama-70B all exceeded 98%, whereas GLM-5 achieved only 54.5%, and MiniMax-M2.5 reached just 5.3%.

Could this simply be due to incorrect response formats causing evaluation scripts to misjudge?

In another set of experiments where gpt-5.4 sent messages, the team relaxed parsing rules, resulting in an increase of at most 0.2 percentage points in valid responses. Among parseable responses where the correct file was selected, GLM-5, MiniMax-M2.5, and o3 still chose the wrong action in 40% to 90% of cases.

△ Results for target localization and action selection on ContextBench. The figure includes various experimental settings; see the paper for details.

Fourth, a stronger main Agent does not necessarily lead to better collaboration.

With the receiving model fixed at 9B, upgrading the sending model from 9B to 80B increased ContextBench and RepoBench accuracy by 21 and nearly 9 percentage points, respectively; however, ToolSandbox accuracy dropped from 91.7% to 65.3%, while BFCL and τ-bench declined by 16 and nearly 3 percentage points, respectively.

τ-bench is particularly noteworthy: sub-agents independently achieving the next step had an accuracy of only around 10%, and upgrading the main agent provided no improvement. This indicates that beyond task difficulty, one must consider whether the message expression suits the receiving model.

Note: SWE-bench Lite used different control conditions and is excluded from this comparison.

Fifth, on-demand communication saves tokens but involves trade-offs in accuracy.

On ContextBench, on-demand communication achieved 73.7% accuracy—higher than the 65.0% observed when never communicating, but lower than the 78.7% achieved when sending messages at every step—while reducing token usage by 46.4% compared to sending at every step.

This experiment utilized 300 static records, with gpt-5.4 sending messages and gpt-4o-mini receiving them. The strategy selected among three options: skipping communication, sending compressed messages, or sending raw context.

In another ToolSandbox setup (144 steps × 5 random seeds), skipping communication based on an 80% calibrated threshold resulted in 98.8% accuracy, essentially matching the 98.6% achieved by sending at every step, while saving 57.9% of tokens.

Performance on τ-bench was less ideal: in the open-source implementation, a threshold-based selection strategy achieved only 8.3% accuracy, falling short of the 13.9% obtained by always formatting into a fixed structure before sending, with similar token usage.

This study suggests a troubleshooting order: first check if necessary information is included in the message, then verify if the receiving model extracts the correct target, and finally examine action selection. Different stages require different improvements. If the correct file is found but the wrong action is chosen, adding more file descriptions may not help.

These conclusions primarily address next-step decision-making; their impact on complete tasks requires further validation considering multi-step error accumulation. The direct implication from the paper is: evaluate communication methods together with receiving models, and allow the system to skip communication when messages are unnecessary.

About the Team

Authors (in paper order): Yi Xie, Zhanke Zhou, Yi Fan, Yong Ge, Bo Han, Bo Liu

Bo Liu, the corresponding author, is currently an Associate Professor in the School of Electrical, Computing, and Software Engineering at the University of Arizona. His research interests include reinforcement learning and control, decision-making, interpretable AI, and safe AI.

Bo Han, currently an Associate Professor in the Department of Computer Science at Hong Kong Baptist University and head of the Trustworthy Machine Learning and Reasoning Group (TMLR Group). His research focuses on machine learning, foundation models, and trustworthy, efficient learning and reasoning methods. He was named to IEEE’s “AI’s 10 to Watch” list in 2024.

Yong Ge, currently an Associate Professor in the Department of Management Information Systems at the Eller College of Management, University of Arizona, and an Eller Fellow. His research areas include data mining, machine learning, recommendation systems, and business analytics.

Paper Title: Bayes-Sufficient Compression Is Not Enough: How Does Communication Help Multi-Agent Systems? Paper Link: https://arxiv.org/abs/2610.03769

Like, Share, and Bookmark!

We welcome your thoughts in the comments section!

— End —

[Academic Submissions] Please send emails to [email protected] on weekdays. Include [Submission] in the subject line and tell us: who you are, where you come from, attach the project/homepage link for your submission content, and provide your contact information.

🎓 We will try our best to reply promptly :)

🌟 Star Us 🌟

Daily Tech Frontier Updates