MASCOT Framework Tackles Homogeneity in Multi-Agent Systems

Keep each character’s behavioral traits distinct while allowing the coordination mechanism to schedule speaking turns based on dialogue progression.

• Paper: MASCOT: Multi-Agent Socio-Collaborative Companion Systems
• Authors: Yiyang Wang, Yiqiao Jin, Alex Cabral, Josiah Hester
• arXiv: https://arxiv.org/abs/2601.14230
• Website: https://hello-diana.github.io/MASCOT/
• Code: https://github.com/hello-diana/MASCOT

Imagine sharing your recent stress with several AI agents.

The first says: "I understand how you feel."

The second follows up: "It's normal to have such emotions."

The third adds: "You are not alone."

Everyone is friendly. Yet after a round of conversation, you only hear three versions of the same message.

The envisioned AI group chat should function like a small team with diverse strengths: one agent listens patiently, another helps analyze the situation, and yet another encourages you to take the next step by proposing substantive solutions. However, in actual interactions, these roles often become increasingly similar, and discussions tend to devolve into taking turns agreeing with one another.

While the number of roles increases, useful perspectives do not necessarily grow accordingly.

Figure 1: MASCOT’s design goal — enabling multiple characters to provide complementary perspectives around a single user and form coherent group dialogues.

A research team from Georgia Tech and MIT introduced MASCOT, which employs two-stage training to simultaneously optimize individual character behavior and group collaboration strategies. The team first trains each character to learn its specific role, then trains a "moderator" that selects the next speaker and determines response strategies based on dialogue progress.

How do multiple AIs form a collaborative team?

The paper provides an intuitive example. A user recalls feeling particularly lonely after moving abroad, when their husband had to travel for work, leaving them alone with a newborn child and a dog.

In baseline methods, the first two characters sequentially express understanding. The user responds that being listened to and understood made them feel more grounded. Subsequently, other characters continue offering comfort: "I'm glad you feel supported," "You are not alone," and "It's normal to feel overwhelmed."

Each sentence individually seems fine, but there is obvious repetition in wording and content across different characters within the context. Even though the user has expressed new feelings, the dialogue keeps repeatedly confirming the initial emotion the user shared.

MASCOT’s dialogue also begins with understanding and comfort. Then, one character builds upon the previous support, inviting the user to reflect: What strength helped you get through that time? Can those experiences help you rebuild connections now?

The user then says they kept reminding themselves that loneliness was temporary; perhaps they can maintain patience now and slowly build connections with others. The next character follows this thread, affirming the resilience and patience the user has already demonstrated.

The support provided by the previous character becomes the starting point for the next character’s continued interaction. The user’s responses also drive the unfolding of subsequent dialogue.

Figure 2: Dialogue comparison in the same scenario. Red indicates repetitive comfort in the baseline, while green highlights parts where MASCOT builds on prior context and introduces new perspectives.

After defining personas, models must truly learn to divide labor

Multi-agent collaboration faces a fundamental challenge: How can different agents maintain their distinct roles and continuously provide complementary contributions as dialogue progresses and user needs evolve?

This requires agents to achieve two things simultaneously: adjust responses based on the latest dialogue context while maintaining their own character-specific behavioral traits. For instance, a listener needs to track changes in the user’s emotional state, and an analyst needs to pose valuable questions incorporating new information. They also need to understand what other characters have already provided to avoid repeating existing responses.

Differences between characters must remain effective throughout dynamic interactions.

As communication proceeds, different characters may gradually revert to similar generic response patterns, all focusing on validating emotions or repeating advice. While each reply seems reasonable in isolation, the group as a whole lacks novel contributions. This phenomenon, known as persona collapse, undermines the functional division of labor essential for multi-agent collaboration.

To address this, MASCOT incorporates character consistency into the training objective, ensuring that character differences translate into learnable and optimizable response behaviors.

In emotional support experiments, the team defined three characters corresponding to the roles in the previous case study:

MASCOT sets up various characters in emotional support scenarios, such as:

• Alice (Listener): Focuses on listening and validating emotions, providing patient, non-judgmental responses to the user.
• Bob (Encourager): Highlights positive experiences and existing strengths, offering specific encouragement and affirmation.
• Cathy (Analyst & Reflection Guide): Helps users sort out problems and explore causes and potential next steps through probing questions and perspective-taking.
• …

These differences must be reflected in both the content and function of the responses. Simply changing the tone is not enough.

Therefore, the first stage of MASCOT is Persona-Aware Behavioral Alignment. The system generates multiple candidate responses for the same context and role, then leverages feedback from a strong model to filter preference pairs with clear quality gaps. These pairs are used to train a dedicated reward model. This reward model learns to judge whether a response aligns with the specified persona and fits the current context.

These comparisons address highly specific questions: For the Listener, which response better reflects the emotion just expressed by the user? For the Analyst, which question more effectively helps the user clarify their situation?

In this way, "persona alignment" transforms from a mere prompt instruction into a trainable signal that the model can learn.

Subsequently, MASCOT employs GRPO (Group Relative Policy Optimization) to fine-tune the speaker models based on scores provided by the reward model, thereby increasing the generation probability of responses that better adhere to persona requirements.

Each Role Must Speak, But Someone Must Decide When

Even if each role performs its individual function well, group chats may still lack smoothness.

For instance, immediately after a user expresses sadness, the Analyst might press for reasons; or when the topic has already progressed, another role might pull it back to initial consolation. While the utterance itself may be appropriate, it must also connect seamlessly with the ongoing dialogue.

The second stage of MASCOT specifically addresses this coordination problem.

The system freezes the trained speaker models and instead trains a Director Agent. This agent reads the existing dialogue, selects the next speaker, and determines the response strategy: Is it appropriate to continue listening, guide reflection, or discuss next steps at this moment? The actual reply is still generated by the selected role.

How Does the Director Know If Its Plan Is Good?

During training, simulated users continue to engage in the conversation. The system extends the dialogue forward and evaluates it based on coherence, diversity, and effective contribution across the entire exchange.

The value of a speaking arrangement must be judged in conjunction with the subsequent dialogue it generates.

For example, having one role continue offering comfort might make the user feel understood, or it might merely repeat previous points. Having another role ask a question might spark new insights, or it might interrupt the user’s expression. Looking only at the immediate reply makes it difficult to distinguish between these scenarios.

MASCOT samples the Director’s decisions at various dialogue positions, generates subsequent exchanges, and constructs group-level reward signals accordingly to optimize the Director. Thus, training considers: Did this arrangement introduce useful new content? Are roles repeatedly discussing things already said? Does the subsequent exchange remain focused on the user’s situation?

This stage also uses GRPO to update the Director’s parameters while keeping the speaker models and simulated user frozen. Consequently, the system can specifically learn which speaking arrangements lead to better subsequent interactions, building upon relatively stable role capabilities.

Figure 3: The two-stage training framework of MASCOT. Stage 1 trains how roles respond; Stage 2 freezes the speaker models and trains the Director to select speakers and response strategies, evaluating coordination effectiveness through subsequent dialogues.

These two steps correspond to two distinct capabilities: whether a role can consistently fulfill its function, and whether the system can invoke that function at the right time.

Can Role Differentiation Translate Into Better Dialogue?

MASCOT targets two types of multi-perspective scenarios: emotional support and work collaboration. In emotional support, different roles must coordinate to listen, guide reflection, and offer encouragement. In work discussions, roles need to divide tasks to organize information, clarify decisions, propose action items, and check for omissions.

Therefore, the team selected the emotional dialogue dataset Empathetic Dialogues (ED) and the meeting dataset QMSum to test whether this training framework could adapt to different role divisions and collaboration goals. Additionally, the study used the emotional support dataset ESConv, which was not involved in training, to further examine cross-dataset generalization capabilities.

These datasets provide real-world user contexts or meeting backgrounds, allowing systems to generate new group interactions from the same starting point. Within the same dataset, all methods use identical role settings. Comparisons include prompting methods, supervised fine-tuning, reasoning strategies, and specialized multi-agent frameworks.

Evaluation examines two levels:

• Individual Level: Focuses on each role’s empathy quality, persona consistency, and contribution to the response.
• Group Level: Focuses on whether the entire exchange is coherent, provides specific new perspectives, and ensures effective participation from all roles.

This design distinguishes between "individual roles answering well" and "the whole team collaborating well."

Automated evaluation uses model judges scoring against a unified scale, mapped to a 0–100 range. Subsequent validation involves human blind reviews and cross-verification using judges from different model families.

In positive scenarios within ED, MASCOT achieved a persona consistency score of 80.0, compared to 65.9 for the strongest baseline—a difference of 14.1 points. The overall individual performance scores were 76.9 and 67.6, respectively.

Across positive, negative, and neutral scenarios, MASCOT’s overall individual performance scores consistently exceeded those of the compared baselines.

However, negative scenarios remain more challenging. Determining the intensity and timing of responses when facing loss, anger, or anxiety still presents certain difficulties.

Figure 4: MASCOT’s role performance (top) and group interaction evaluation (bottom) on the ED dataset. MASCOT achieved a persona consistency of 80.0 in positive scenarios, an improvement of 14.1 points over the strongest baseline. In group interaction evaluations, MASCOT outperformed all compared baselines across all four metrics and the overall score.

Improvements at the group level are equally significant. In neutral scenarios within ED, MASCOT achieved the highest scores in all four metrics—role fidelity, originality, relevance, and engagement—with an overall score of 57.3, surpassing the strongest baseline, Few-Shot + CoT (51.9), by 5.4 points. This indicates that improvements in individual roles are accompanied by enhanced overall interaction quality.

Division of Labor and Coordination Matter in Work Scenarios Too

In the QMSum meeting scenario, roles shift accordingly: recording meeting content, organizing decisions, proposing action items, and checking for omissions or issues.

Across the Academic, Committee, and Product Meeting subsets, MASCOT’s overall group interaction scores exceeded those of the compared baselines. This demonstrates that role division and speaking coordination can also be applied to task-oriented discussions. Here, the metric measures the quality of discussions generated based on meeting context.

Figure 5: Group interaction evaluation on QMSum. MASCOT outperforms the compared baselines in overall scores across all three meeting subsets.

What Each Training Stage Contributes

The team conducted ablation studies by separately removing role alignment and collaborative optimization to observe changes in performance.

Removing role alignment (MASCOT-P) resulted in a more noticeable decline in individual empathy quality and role consistency. Removing collaborative optimization and switching to random speaking order (MASCOT-C) primarily affected group coherence and contribution participation.

These results explain why MASCOT employs a two-stage training process: while individual roles can maintain their distinct characteristics, the entire group must still learn how to coordinate their turns effectively.

Figure 6: Ablation study. Role alignment primarily improves individual behavior, while collaborative optimization primarily enhances group interaction; both components jointly support the full system's performance.

Can Humans Perceive the Improvements in Model Scores?

Do the improvements reflected in model scores translate to human perception?

The team invited 40 annotators aged between 19 and 73 (mean age 42.9) for blind evaluations, hiding system names and randomizing the display order.

MASCOT was compared against Few-Shot CoT (using few-shot chain-of-thought prompting) and MultiAgentESC (an emotional support multi-agent framework). For each baseline, 100 pairs were sampled, with each pair independently evaluated by two annotators.

Out of the 200 evaluations corresponding to each baseline, MASCOT was preferred in 69.0% and 68.5% of cases, respectively. In other words, approximately seven out of ten judgments favored MASCOT in these pairwise comparisons.

Figure 7: Human pairwise preference evaluation. The numbers indicate the count of evaluations, totaling 200 per baseline. Asterisks correspond to the two-sided sign test reported in the paper: * p<0.05, ** p<0.01, *** p<0.001.

In another human evaluation, 20 participants compared 100 sets of dialogues between MASCOT and Few-Shot CoT, with each set receiving two independent judgments. Out of these 200 judgments, MASCOT was preferred 151 times, accounting for 75.5%.

Participants also rated the dialogues on helpfulness, appropriateness of emotional response, and perceived safety, providing supplementary evidence closer to the actual reading experience regarding dialogue quality.

Can the Same AI Team Adapt to Users with Different Personalities?

Variations in user personality may also impact the collaborative performance of multi-agent systems. The team constructed simulated user profiles using the Big Five Traits from psychology. This model describes personality differences across five dimensions, whose English initials form the acronym OCEAN:

• Openness: The tendency to accept new ideas and explore new experiences.
• Conscientiousness: The degree to which one is organized, disciplined, and values planning.
• Extraversion: The tendency to actively engage in social interactions, expression, and engagement.
• Agreeableness: The tendency toward cooperation, trust, and consideration for others.
• Neuroticism: The tendency to easily experience negative emotions such as anxiety and worry.

In each experiment, one dimension was set to High or Low while the others remained at medium levels, creating 10 simulated user profiles. The team applied these profiles to 50 ED scenarios, generating and evaluating a total of 500 dialogues to examine how user characteristics influence system performance.

For simulated users with High Openness (more willing to explore new ideas and experiences), MASCOT achieved a role consistency score of 79.0. Compared to the corresponding Low setting, the overall score was 1.9 points higher under High Agreeableness (more inclined toward cooperation and consideration), and the social contribution score was 2.5 points higher under High Extraversion (more proactive in expression and interaction), while maintaining role consistency.

Different traits also presented different challenges. For instance, under High Conscientiousness (valuing organization, discipline, and planning more highly), the overall score was 71.5, lower than the 74.5 recorded for the corresponding Low setting.

Under the High/Low Neuroticism settings (more/less prone to experiencing negative emotions like anxiety and worry), the overall scores were similar, at 70.7 and 70.8, respectively.

These controlled simulation results suggest that multi-agent systems targeting diverse users need to further consider personalization in role combinations and coordination strategies.

What Questions Remain for Multi-Agent Companionship?

MASCOT offers a trainable design approach: allowing roles to maintain their distinct behavioral characteristics while enabling the coordination mechanism to schedule turns based on dialogue progression. This provides a reference for scenarios such as emotional support, brainstorming, and meeting assistance.

Current evidence primarily comes from English dialogues and controlled evaluations; long-term interactions, different cultures, and individual needs remain to be tested. As user goals and emotions continuously change, whether these roles can consistently provide useful new perspectives is the next question this type of system needs to answer.

© THE END

Please contact this official account for authorization before reprinting.

For submissions or media inquiries: [email protected]

Open in WeChat