BAAI Releases FlagOS 2.2 for Cross-Chip Operator Reuse and Performance Boost

On October 17, 2026, the 2026 AI Open Computing Conference and FlagOS Technology Conference, themed "Full Stack Integration, Boundless Intelligent Computing," was held in Beijing. Following the release of FlagOS 2.2 on September 30, the conference showcased technical progress ranging from low-level operators and compilers to model deployment and services. It also addressed the practical challenge of rapidly evolving AI models requiring long-term support from compute infrastructure.

BAAI's FlagOS 2.2 has evolved from multi-chip adaptation to cross-chip reuse. The nine operator libraries now cover a total of 1,454 operators, an increase of 890 compared to the previous version. Within the testing scope of over 1,000 operators across 10 heterogeneous AI chips, cross-chip operator performance improved by approximately 27%. In practice with GLM-5.3-Flash, the logical service throughput of existing Ascend 910B clusters reached 2.04 times that before optimization. Under specified conditions, the compute cost (converted based on rental fees) was 19.4%–27.6% lower than the reference billing for corresponding commercial APIs. Quantifiable joint optimization results were also observed in world models and ARM edge-side inference.

This marks a shift in open intelligent computing from answering "which chips are supported" to answering "can we continuously optimize every type of compute resource and sustain every generation of models." This allows chip vendors to focus more on architectural features, gives compute operators an additional software optimization path during model upgrades, and enables model developers to help algorithmic innovations cross hardware boundaries faster.

I. Expanding Cross-Chip Shared Optimization Capabilities: Total Operators Exceed 1,400, Overall Operator Performance Improves by 27% Across Multiple Chips

FlagOS 2.2 continues to expand its general-purpose computing, inference-specific, and scientific computing operators. The release list covers 9 operator libraries. Public operators such as those in FlagGems increased from 564 to 1,454, adding 890 new ones. FlagGems-SGLang was released for the first time, containing 40 high-performance multi-chip operators. These new capabilities address actual computational needs: FlagGems expanded linear algebra, reduction, and training backward interfaces; FlagGems-SGLang covers Attention, Mamba state updates, MoE, LoRA, sampling, and quantization, with all 40 operators passing the NVIDIA test checklist this round; FlagGems-vLLM expanded MLA, KDA/GLA/GDN2, TopK, and low-precision computations. For Qwen3.8-Flash-Next, 8 fused operators were added, and full CUDA Graph integration was completed.

Jointly Optimizing Operators and Compilation Execution Around Critical Compute Paths

FlagOS 2.2 covers 9 operator libraries, with public operators like those in FlagGems increasing from 564 to 1,454 (adding 890). FlagGems-SGLang was released for the first time, including 40 high-performance multi-chip operators. By combining operator implementations with FlagTree compilation optimizations, performance is enhanced through low-bit and mixed precision handling, operator fusion, shape-based execution path selection, compilation, and auto-tuning. Achieving a 27% overall operator performance improvement across 10 chip vendors and over 1,000 operators signifies that FlagOS's optimization capability is moving from single-point performance breakthroughs to cross-chip, scalable system performance improvements.

AI-Assisted R&D: Scaling Operator Development from Expert Manual Optimization to Mass Production

The challenge of cross-platform operator coverage lies not just in "writing code," but in simultaneously meeting standards for correctness, performance, and maintainability. The KernelGen multi-chip automatic operator generation tool now supports nine AI chips, linking code generation, hardware verification, and auto-tuning into a mass production capability. This quarter, 765 new operators were generated, of which 416 passed accuracy validation on at least 5 domestic chips and achieved performance levels exceeding 80% of their native counterparts.

FlagPrism Unified Cross-Chip Debugging Tool: Effective Tuning for Developers and Agents Alike

FlagPrism links Triton source code, compiler IR, and device execution information, moving cross-chip development from "runnable" to "explainable." The Debugger can trace numerical issues down to specific statements, while the Profiler can break down time consumption and locate optimization opportunities. It currently supports Huawei Ascend, Iluvatar CoreX, Moore Threads, NVIDIA, and Enflame, helping developers and agents reuse tuning methodologies.

When developers, chip vendors, or even operator-generation agents can obtain comparable numerical and performance feedback, what is reused across chips is no longer just optimized code snippets, but also a methodology for discovering problems, validating hypotheses, and iterating continuously. The synergy between FlagGems, FlagTree, KernelGen, and FlagPrism represents the core technical advancement distinguishing FlagOS 2.2 from simple adaptation engineering.

II. Collaborative Optimization Enters New Model Adaptation: Unified Plugins and Cross-Chip Quantization Enable Day-0 Adaptation for Up to 12 Chips

FlagOS connects common operators and communication capabilities to training and inference frameworks via its cross-chip framework plugin library. FlagOS-Compressor provides cross-chip weight conversion and quantization capabilities. Combined with components for compilation, migration, evaluation, and release, collaborative optimization is translated into developer-ready model versions, enabling new models to be adapted to up to 12 chips within one day.

Framework Plugins: Integrating Low-Level Optimizations into Training and Inference

The FlagOS unified cross-chip plugin system acts like a set of "sockets," allowing developers to continue using familiar model development workflows while integrating capabilities from various chips into frameworks such as vLLM, Megatron-LM, TransformerEngine, SGLang, Torch, VeRL, and LightX2V. Taking TransformerEngine-FL as an example, models initiate computations through a unified operator interface. The OpManager organizes operator registration, selects FlagOS, vendor, or PyTorch implementations based on policies, and provides caching and failure fallback mechanisms, minimizing changes to core training code.

FlagOS-Compressor: Adapting Weights and Precision to Target Devices

When a target device’s low-precision support or VRAM capacity is incompatible with a new model, weight conversion and quantization offer an alternative deployment path. FlagOS 2.2 introduces FlagOS-Compressor for the first time. It selects processing strategies based on model architecture and target hardware, organizing weight conversion, quantization, export, and validation into a unified workflow. The tool allows module-specific quantization strategies, preserving high precision for sensitive modules while exporting weights, quantization parameters, and runtime configurations. For the Qwen3.8-2.4T-A95B adaptation, it provides processing paths from BF16 and FP8 to INT8. For the Qwen3.5-397B-A17B, adopting W8A8 quantization reduced the model file size from 751.41 GB to 382.45 GB—a 49.1% reduction—while achieving an absolute accuracy of 84.85% on GPQA-Diamond. Inference pipeline validation was completed on both NVIDIA H20 and domestic PPU devices.

Day0 Delivery: Adapting New Models to Up to 12 Chips Within One Day

Leveraging accumulated expertise in plugins, operators, compilers, and quantization tools, FlagOS reduces redundant work during post-release model adaptation. Over the past six months, FlagOS has successfully adapted new models to up to 12 different chips within a single day. After adaptation, models are re-evaluated to assess task performance on target devices, and both models and configurations are published to platforms such as Hugging Face and ModelScope. While "Day0" reflects response speed, delivery quality is determined by testing on target hardware and accompanying artifacts; actual service performance still requires validation against specific business workloads.

Case Study: China Telecom’s Xingchen Model Achieves Day0 Adaptation Across 10 Chips

On the training side, China Telecom’s Xingchen large model integrates FlagGems into its training infrastructure based on MindSpore, CANN, and Ascend NPUs. Specialized optimized operators replace certain native implementations, while leveraging MindSpore’s automatic parallelism and distributed communication capabilities. Training operators such as mHC and TopK have completed full-chain debugging and end-to-end training effectiveness verification. Additionally, the integration of FlagOS into MindFormers has been open-sourced and merged. On the inference side, Xing4.0-29B-A4B has completed adaptation, precision alignment, and deployment validation across ten chip platforms. Eight of these use standard adaptation, while Apple M5 Pro and Zhipu P1 employ W4A8 edge-side quantization. The team iteratively replaced 12 inference operators and contributed them to the main repositories of FlagGems and vLLM. In high-concurrency, long-input tests, compared to NVIDIA’s native operator implementations, Time to First Token (TTFT) latency decreased by 19.87%, and decoding speed increased by 26.32%.

Framework plugins address how upper-layer frameworks invoke capabilities across different chips, while FlagOS-Compressor solves how model weights and precision adapt to target devices. Day0 delivery connects these components with operators, compilation, evaluation, and publishing, forming a closed loop from underlying optimization to multi-chip model delivery. This drives true collaborative optimization into the realm of new model adaptation.

III. From Running Models to Efficient Token Production: Cross-Chip Optimization Enters Real-World Service Scenarios

After cross-chip delivery, models must meet real-world application requirements. Token services focus on whether existing compute resources can sustain competitive speed and cost as new models emerge. World models introduce video, spatial, and multimodal data, increasing computational complexity. As models move to edge devices, they must operate under constraints of limited memory, compute power, and energy consumption. These three demands collectively drive FlagOS from multi-chip adaptation toward continuous collaborative optimization.

DeepSeek V4.1 Flash: Cross-Platform Optimization Demonstrates Commercial API Competitive Potential

FlagOS utilizes an optimization stack comprising SGLang-Plugin-FL, FlagGems, FlagGems-SGLang, FlagTree, and FlagCX to optimize DeepSeek V4.1 Flash on NVIDIA H100 and T-Head 810E devices. On the H100, using a 1P1D configuration with W4A16/FP8, in coding tasks with 128K input, up to 8K output, and DSpark cache hits enabled, per-request decoding speed increased by 31.7 tokens/s at 32-way concurrency and 21.9 tokens/s at 64-way concurrency, representing improvements of 19.7% and 18.0%, respectively. On the T-Head 810E, using 16 cards of 96GB per node with a 1P1D configuration and W8A8, compared to the platform’s native SGLang path, per-request decoding speed increased by 57.2 tokens/s (an 83.1% improvement) at 24-way concurrency and by 40.2 tokens/s (a 91.5% improvement) at 48-way concurrency.

GLM-5.3-Flash: Throughput Doubles on Existing 910B Services, Scenario-Based Cost Reduction of 19.4%–27.6% Compared to API Pricing

Based on optimizations within the FlagOS technology stack, GLM-5.3-Flash achieves approximately 2.04x logical service throughput in Agentic replay scenarios compared to pre-optimization levels, increasing from 8.52k to 17.36k tokens/s. This metric accounts for input, output, and cache-hit logical input tokens, reflecting the service's capacity rather than pure generation speed. In a separate 512K long-context test, the generation speed reached 60.9 tokens/s, closely matching the external reference value of 61.1 tokens/s measured via official APIs (a ratio of ~99.7%). Note that this comparison focuses solely on generation speed and does not represent an identical workload benchmark.

Assuming a monthly rental cost of 16,500 CNY per server equipped with eight Ascend 910B cards, a 3P1D deployment configuration, a 93% cache hit rate, and full load conditions on the decoding side, the estimated compute cost per million output tokens (including corresponding inputs) is as follows:

• 256K Input / 8 Concurrency: 59.27 CNY, which is 19.4% lower than the commercial API billing reference of 73.55 CNY.
• 512K Input / 4 Concurrency: 104.53 CNY, which is 27.6% lower than the API billing reference of 144.31 CNY.

All tests below use an output length of 1K. In the tables, values to the left of the arrow represent vendor baselines, while values to the right represent results optimized by FlagOS.

Table 3 | Long-Context Inference Performance Comparison on Ascend 910B (Output Length 1K)

These results provide enterprises with an evaluable path: first extend the service capability of existing hardware through software optimization, then determine whether hardware upgrades are necessary.

Case Study: Isoftstone Reduces p50 Inter-Token Latency for GLM-5.2 by 14.53%

Isoftstone’s Tianyuan Intelligent Computing Hybrid Cloud Platform integrated FlagOS capabilities—including operators, compilation, communication, and training/inference frameworks—to connect upper-layer model services with underlying heterogeneous devices. Both parties jointly optimized GLM-5.2 on four Ascend 910C servers, allocating two servers each for Prefill and Decode phases. With all five optimizations enabled, both server-side and client-side p50 Inter-Token Latency (ITL) decreased by 14.53%. For p99 ITL, which reflects long-tail latency, server-side latency dropped from 83.82 ms to 66.35 ms (a 20.84% reduction), and client-side latency dropped from 138.56 ms to 118.73 ms (a 14.31% reduction). These metrics provide platform operators with data to assess service capabilities under actual business loads.

The above practices demonstrate that cross-chip reuse optimization is evolving from simply "running models" to "efficiently producing tokens." It supports maintaining inference speeds for new models across various chip types and provides a basis for legacy devices to handle new models, helping enterprise service platforms reduce latency and calculate costs. Specific deployments still require comprehensive judgment based on business loads, latency requirements, and total operational costs.

IV. From Token Factory to Physical AI: New Workloads for World Models and Edge Computing

AI computing demand is expanding along three paths: Data center "Token Factories" pursue continuous evolution and scale efficiency; Edge AI faces constraints in memory, power consumption, and hardware space; and Physical AI embodied world models introduce workloads involving video, spatial, temporal, action, and multi-sensor data. FlagOS has proactively extended its capabilities—such as operator libraries, compilers, and frameworks—to support Physical AI and edge computing.

Supporting BAAI’s Next-Generation Embodied World Model: Synergy Between Public Operators and Training Systems

FlagOS collaborated with BAAI’s next-generation embodied world model system, optimizing around FlagGems operators, training frameworks, and distributed strategies. The VLM, Tx2V, and Action modules achieved speedups of approximately 2.3x, 1.9x, and 2.0x respectively, with Model FLOPs Utilization (MFU) reaching about 33%, 36%, and 35%. At the framework layer, data was organized using fixed sequence lengths to pass stable tensor shapes to static compilation, while dynamic metadata expressed sample variations. This resulted in a 19.7% increase in throughput and a 29.5% reduction in VRAM usage. By caching Attention outputs (O and LSE) for reuse during backpropagation, forward replays were reduced from 288 times to zero, improving generation branch throughput by 14.8%. FlagGems provided optimized implementations for critical computations such as Attention, indexing, and reduction. The Attention operator achieved a speedup of approximately 1.40–1.43x, while indexing operators reached up to 2.97x. These optimizations remain in the public operator library for reuse by subsequent models and hardware.

LightX2V-Plugin-FL: Extending a Unified Plugin System to Video and Action Generation

LightX2V is a video generation inference framework, and LightX2V-Plugin-FL is its plugin for integrating with FlagOS. The framework organizes the model workflow, the plugin connects target devices with operator execution paths, and FlagGems handles the actual Attention computation. This allows existing operators to enter the denoising loops repeatedly executed in video generation, reducing the effort required to adapt model frameworks separately for different chips. A dual-card validation run of DreamZero-DROID successfully completed the pipeline from image input to video and action output. This demonstrates that public operators can enter video generation frameworks via a unified plugin and achieve cross-chip execution and complete output on domestic hardware. Further speed improvements still require baseline testing under identical conditions.

ARM CPU: MiniCPM5-2B Achieves ~92 tokens/s Generation on M5 Pro

FlagOS 2.2 independently released FlagTree-CPU, advancing unified compilation, low-bit operators, and runtime optimizations for ARM CPUs. The toolchain features differentiated designs for paths such as I8MM, SDOT, and streaming memory access, reducing redundant overhead through operator fusion and execution scheduling. On CPUs supporting these instructions, SME2 and NEON I8MM collaborate to process GEMM operations within the same layer, increasing full-model Prefill throughput by 21% to 25%. In CPU-only tests on a Mac M5 Pro, MiniCPM5-2B achieved a Prefill speed exceeding 1000 tokens/s and a Decode speed of 91.66 tokens/s. For Qwen3.8-27B, adopting DFlash2 increased Decode speed from 12.73 to 23.40 tokens/s (approximately 1.84x the original), providing a basis for edge applications to balance model capabilities, device memory, and latency when selecting configurations.

From training embodied world models and video-action generation plugins to on-device inference on ARM CPUs, FlagOS is extending common operators, unified plugins, and compilation optimizations from Token Factory to Physical AI and edge computing. Metrics such as speedup and Model FLOPs Utilization (MFU) in world model training, cross-chip execution validation in video generation, and generation speeds on edge CPUs collectively indicate that collaborative optimization is beginning to cover new workloads. However, some performance improvements still require further validation against baseline comparisons under identical conditions and actual business workloads.

V. From Technical Co-Construction to a Global Ecosystem: Code Contributors Up 55%, Downloads Quadrupled

International Collaboration: Expanding Global Cooperation in Open Computing

BAAI’s FlagOS is rooted in China while expanding globally. International collaboration includes technical exchanges such as the China-Africa Entrepreneurship & AI Partner Seminar and the Abu Dhabi Open Compute Workshop, as well as developer and educator training programs. The second phase of the Global AI Computing Educator Development Program, supported by UNDP and co-hosted with the African Union’s ASRIC, brought together nearly 50 core university faculty members from over 30 countries. The OASIS-AI 2026 @ MICRO Workshop further focused on open architectures and software infrastructure for heterogeneous AI systems. The United Nations’ “Open Compute Working Group” was approved for establishment on September 28, with its inaugural meeting scheduled during the APEC summit in November.

Developer Community: Quarterly Downloads Quadrupled, Code Contributors Steadily Increasing

Maintaining an ever-evolving model ecosystem across diverse hardware requires long-term commitment from all parties. Each FlagOS release relies on cooperation among multiple open-source communities, continuously collaborating with upstream projects such as PyTorch, Triton, vLLM, SGLang, Megatron, TransformerEngine, and VeRL to maintain interfaces and implementations. By the end of Q3, the FlagOS ecosystem comprised 107 member organizations. Quarterly community downloads reached 464,336, which is 4.76 times the volume in Q2. Between versions 2.1 and 2.2, merged PRs increased from 1,890 to 4,890, code commits rose from 5,100 to 9,730, and new Issues grew from 420 to 1,423. The number of code contributors reached 264, representing a 55% increase.

Current AI models may evolve every three months, yet compute infrastructure must deliver value over many years. FlagOS 2.2 demonstrates that a technically viable path worth sustained investment is becoming increasingly clear: consolidating operators, compilation, quantization, and system optimizations tailored for new models into reusable public capabilities, delivering them to different chips via unified interfaces, and continuously validating benefits within real-world models and service workloads. From Token Factory to edge AI and world models, the value of open computing lies not just in increasing hardware choices, but in ensuring that every unit of invested compute power continues to deliver value as the software ecosystem evolves.