Shanghai AI Lab Open-Sources 320B Agent Model Sage S2.1 Flash

Original title: 上海AI Lab开源Sage S2.1 Flash技术报告:四项智能体基准SOTA,科学多模态感知如何融入通用智能体?
78AI Score

Shanghai AI Lab has open-sourced Sage S2.1 Flash, a 320B-parameter Mixture-of-Experts agent model, releasing full training pipelines and technical details. It reports state-of-the-art results on four agent benchmarks, with its key contribution being the integration of scientific multimodal perception into a general-purpose agent that can drive professional software such as 3D Slicer to complete long-horizon tasks.

Full Report

Disclosing the complete training pipeline and technical details of a Mixture-of-Experts model with 320B total parameters.

Image

With a single instruction, an Agent can reconstruct a three-dimensional skeletal model from 12,487 CT images? Sage S2.1 Flash has done it. Its approach is to identify the slices belonging to the target scan sequence, organize them, and then operate the medical imaging software 3D Slicer to reconstruct a stereoscopically visible skeletal model.

Recently, the Shanghai AI Laboratory, together with Fudan University, the Institute of Software at the Chinese Academy of Sciences, Renmin University of China, the Institute of Automation at the Chinese Academy of Sciences, East China Normal University, Harbin Institute of Technology, Peking University, and several other research institutions and universities, officially released the Sage S2.1 Flash technical report. The report systematically discloses the complete training pipeline and technical details of this Mixture-of-Experts (MoE) model with 320B total parameters. Understanding molecules and spectra, calling tools in succession, and delivering work in professional software—these capabilities are now concentrated in a single model.

Positive Reviews in Its First Week

In its first week online, Sage S2.1 Flash received considerable positive feedback from real users in actual use. Multiple testers described the model as "fast and smart," experiencing smooth tool-calling in multi-turn task tests; in scenarios closer to real-world development, the model was also able to handle complete task execution; and in the "pelican test," the images generated by the model also won strong approval from testers.

Unified Scientific Multimodal Architecture

Enabling the Model to Directly Understand Scientific Data

Sage S2.1 Flash adopts a 320B Mixture-of-Experts (MoE) architecture, controlling inference compute through dynamic expert routing and combining sparse attention with linear attention to support long-context processing. The model backbone can process text, images, and scientific data representations within a shared context, providing a foundation for complex reasoning and multi-step task execution.

This general-purpose model backbone provides shared reasoning capabilities for scientific data understanding, code execution, complex retrieval, and professional software operation, and also lays the architectural groundwork for further integration of science-specific modalities.

Sage Sci-Perception:

Integrating Structured Scientific Data into General-Purpose Language Models

For scientific data such as molecules, proteins, nucleic acids, and spectra, the research team designed the Sage Sci-Perception scientific multimodal perception module, employing specialized encoding and representation methods including MoleculeSTM (molecular structures), ESM-2 (protein sequences), DNABERT-2 (DNA), RNA-FM (RNA), and DiffSpectra (spectral data) to extract features such as atomic connectivity, sequence dependencies, and continuous measurement signals, and mapping them into the unified representation space of the language model through trainable modality projectors. During training, supervised learning and reinforcement learning are combined to construct verifiable tasks such as molecular property classification, protein interaction site prediction, atom counting, and structural relationship queries, using metrics such as classification accuracy, set overlap, and numerical error to provide reward signals.

Scalable Verifiable Environments:

Building an Agent Post-Training System

To improve the model's autonomous execution capability in complex tasks, the research team built verifiable training environments covering tool calling, information retrieval, software engineering, machine learning experiments, cybersecurity, professional workspaces, and visual interaction, and optimized scientific multimodal understanding capabilities through an independent scientific reinforcement learning mechanism.

ToolSpace: Automatically Constructing Complex Task Data from Real Tool Calls

In tool-calling capability training, the research team proposed the ToolSpace framework, which extracts tool definitions, call parameters, and execution results from MCP tool services and real agent trajectories to construct a tool composition graph covering data passing, environment state changes, and failure recovery relationships. Based on state-aware random walks, the system automatically generates cross-tool, cross-environment call sequences, and ensures sequence executability through execution condition checks, failure rollback, and resampling. Successful trajectories are then converted into natural language tasks, generating verification programs for final states, key intermediate states, and output fields to filter reproducible training data. Ultimately, a tool collaboration space covering 2,042 tools, 77,760 call pattern nodes, and 804 environments was constructed; the generated supervised fine-tuning trajectories contain an average of 18.86 tool calls, with a maximum of 209, spanning an average of 2.29 environments.

ML Coding: Building Executable, Verifiable Machine Learning Research Tasks

In machine learning engineering, the research team built an automated training pipeline from real research material collection and task synthesis to experimental execution verification. The system integrates 4,936 Kaggle data sources, 2,960 research papers, and code and data resources from platforms such as Hugging Face and GitHub, matches datasets with algorithm implementations through data interfaces, training entry points, and evaluation metrics, and verifies environment availability in containers with fixed dependencies.

Training tasks cover predictive modeling, algorithm implementation, paper reproduction, and model post-training. Among them, the system constructs algorithm tasks that the model must autonomously complete by removing core code such as loss functions, feature transformations, or parameter updates; for open-ended experiments, the agent is required to design its own approach and execute training. Ultimately, all tasks are packaged as reproducible environments via Harbor, isolating task instructions, execution environments, and private evaluation programs, and verifying task completion through code execution results, predictive metrics, and model weights.

Visual Agents: From Interface Perception to Continuous Operation

Scientific research requires not only understanding specialized data but also involves extensive software operation through graphical interfaces. To this end, the research team adopted a staged training strategy to improve the model's task execution capability in dynamic visual environments.

  • The first stage uses image recognition and single-step click tasks to train the model to understand interface elements, locate interaction targets, and select the correct operation based on the current image.
  • The second stage extends these capabilities to complex continuous operation tasks. The model needs to read the current interface at each step, assess task progress, execute operations, and decide the next action based on new screenshots. Training tasks increase demands on environment state tracking and long-process planning by extending operation sequences and strengthening dependencies between steps.

In addition, the research team built a diverse set of verifiable training environments covering information retrieval, software engineering, cybersecurity, and productivity tools, spanning multi-hop retrieval, code development, vulnerability analysis, and complex tasks such as PPT, Excel, legal documents, and 3D engineering design. For each direction, centered on real data and professional task requirements, they established a complete pipeline from data collection and task synthesis to environment construction and result verification, and designed corresponding evaluation mechanisms according to the characteristics of each domain, incorporating fact and evidence verification, program execution, environment state checking, and final deliverable validation into the training process.

33 Benchmark Evaluations

Testing Complex Task Execution Capabilities

The research team conducted 33 benchmark evaluations across seven major domains: agentic coding, office productivity, tool use, machine learning engineering, deep research, information retrieval, and visual interaction.

Four First-Place Results: In the machine learning engineering task MLE-bench Lite, Sage S2.1 Flash achieved a HumanRank of 90.3, ranking first among the main comparison models. In OfficeQA Pro, Harvey LAB, and SpreadsheetBench 2, the model achieved 72.2, 25.0, and 37.1, respectively, also ranking first.

Four Second-Place Results: PostTrainBench and PaperBench achieved 39.2 and 88.0, respectively, while the desktop operation evaluation OSWorld-Verified and the web interaction evaluation WebArena-Verified achieved 85.2 and 67.6, respectively, all ranking second among the main comparison models.

Toward Real Scientific Research Software

Exploring End-to-End Scientific Task Execution

The research team further demonstrated the application of Sage S2.1 Flash in 3D engineering design, scientific data analysis, and scientific literature research.

In engineering modeling tasks, the model operates Blender and CAD software based on images and design requirements to construct 3D models such as buildings, turbofan engines, and truck chassis, generating engineering files that can be further edited and inspected.

In scientific software analysis tasks, the model uses professional tools such as Fiji, EEGLAB, and QGIS to complete microscopy image processing, EEG signal analysis, and remote sensing data comparison.

In a scientific deep research case, the model conducted literature retrieval, experimental parameter extraction, simulation computation, and visualization around the problem of optical quantum memory, and generated an interactive research application.

Today, the scholar who once pored over books behind closed doors has finally pushed open the door of the study, ventured into the field of science, and touched the laws that govern how the world works. The road toward general scientific agents has only just begun to unfold, and every step toward the real world is like humanity's first gaze at the stars and first measurement of the land—leading us toward that vivid and vast "reality."

Now, you too can hand your tasks to Sage S2.1 Flash and explore how it performs across different scenarios.

图片

© THE END

For reprints, please contact this official account for authorization.

Submissions or story tips: [email protected]

Open in WeChat