ALL INTELLIGENCE
智谱 AI 研究T1·Models

Zhipu & Huawei Open-Source GLM-Image: Multimodal SOTA Trained on Domestic Chips

By 智谱 AIOriginal title: 智谱联合华为开源国产芯片训练的多模态SOTA模型
78AI Score

Zhipu AI and Huawei have open-sourced GLM-Image, a new multimodal model fully trained on Ascend Atlas 800T A2 chips using the MindSpore framework, validating domestic hardware for frontier AI. The model employs a hybrid 'autoregressive + diffusion decoder' architecture, achieving state-of-the-art performance in complex text rendering (especially Chinese characters) with ultra-low inference costs.

Full Report

Contents4 sections

Today, Zhipu AI and Huawei jointly open-sourced GLM-Image, a next-generation image generation model. The entire training pipeline—from data processing to large-scale training—was completed on Ascend Atlas 800T A2 hardware using the MindSpore AI framework, making it the first SOTA multimodal model fully trained on domestic chips. GLM-Image employs an innovative hybrid architecture combining autoregressive modeling with diffusion decoding, integrating image generation with language models. This represents a significant step toward exploring the new paradigm of "cognitive generation" exemplified by models like Nano Banana Pro. Key highlights include:

  • Architectural Innovation for Cognitive Generation: By adopting a novel "autoregressive + diffusion decoder" hybrid architecture, GLM-Image balances global instruction understanding with local detail rendering. This approach overcomes challenges in generating knowledge-intensive visuals such as posters, PPT slides, and scientific illustrations, marking a crucial advancement toward "knowledge + reasoning" cognitive generation models like Nano Banana Pro.
  • SOTA Model Trained Entirely on Domestic Chips: The autoregressive backbone was built from scratch on Ascend Atlas 800T A2 devices using the MindSpore framework, completing the full workflow from data preprocessing to large-scale training. This validates the feasibility of training frontier models on a fully domestic computing stack.
  • Open-Source SOTA in Text Rendering: GLM-Image ranks first among open-source models on both CVTG-2K (Complex Visual Text Generation) and LongText-Bench (Long Text Rendering), demonstrating exceptional proficiency in Chinese character generation.
  • High Cost-Efficiency and Speed Optimization: In API mode, generating a single image costs just ¥0.1. A speed-optimized version is coming soon.

Architectural Innovation: Understanding Instructions, Rendering Text Accurately

Recently, closed-source image generation models like Nano Banana Pro have driven deeper integration between image generation and large language models. The technical paradigm is evolving from simple image synthesis to "cognitive generation," which combines world knowledge with reasoning capabilities. These models excel in knowledge-intensive scenarios such as posters, PPTs, and scientific diagrams, delivering stunning high-fidelity details that showcase the advantages of this new paradigm. GLM-Image is our key exploration into this "cognitive generation" direction. As an open-source, industrial-grade discrete autoregressive image generation model, we aim to share our technical path and practical insights with the open-source community.

  • Innovative Architecture Enables Precise Comprehension and Rendering: Traditional models often struggle to balance complex instruction understanding with accurate text rendering. GLM-Image addresses this by introducing a hybrid "autoregressive + diffusion decoder" architecture, uniquely combining a 9B-parameter autoregressive model with a 7B-parameter DiT diffusion decoder. The former leverages its language model foundation to enhance semantic understanding and global composition, while the latter, paired with a Glyph Encoder, focuses on reconstructing high-frequency details and stroke precision, effectively mitigating the issue of "forgetting how to write characters."
  • Multi-Resolution Adaptability: Through improved tokenizer strategies, GLM-Image natively supports arbitrary aspect ratios across resolutions ranging from 1024x1024 to 2048×2048 without requiring retraining.

General Pipeline

Decoder Architecture Diagram

Open-Source SOTA: Superior Performance in Text-Dense Generation Tasks

Thanks to these architectural innovations, GLM-Image achieves open-source SOTA performance in authoritative text rendering benchmarks.

  • CVTG-2K (Complex Visual Text Generation): This benchmark evaluates a model's accuracy in simultaneously generating multiple text regions within an image. GLM-Image leads all open-source models with a Word Accuracy score of 0.9116. It also tops the NED (Normalized Edit Distance) metric with a score of 0.9557, indicating that generated text closely matches target text with fewer typos or omissions.
  • LongText-Bench (Long Text Rendering): This benchmark assesses accuracy in rendering long, multi-line text across eight dense-text scenarios including signs, posters, PPTs, and speech bubbles, with separate tests for English and Chinese. GLM-Image ranks first among open-source models, achieving scores of 0.952 for English and 0.979 for Chinese.

SOTA Model Trained on Domestic Chips

GLM-Image represents a deep exploration and validation of the domestic computing ecosystem. Its autoregressive backbone underwent the entire training process—from early-stage data preprocessing to final large-scale pretraining—on Ascend Atlas 800T A2 devices. Leveraging Ascend NPUs and the MindSpore framework, we developed a custom training suite utilizing features like dynamic graph multi-level pipelining, high-performance fused operators, and multi-stream parallelism to optimize the end-to-end workflow for data preprocessing, pretraining, SFT, and RL.

Specific optimizations included:

  • Dynamic Graph Multi-Level Pipelining: Streamlining and overlapping critical host-side operator dispatch stages to eliminate bottlenecks.
  • Multi-Stream Parallelism: Overlapping communication and computation to break down communication walls caused by gradient synchronization and feature broadcasting, maximizing performance.
  • Ascend-Friendly Fused Operators: Using operators like AdamW EMA, COC, and RMS Norm to simultaneously improve training stability and efficiency.

Scenario 3: Social Media Image & Text Covers GLM-Image is ideal for creating complex layouts such as social media covers and content graphics, giving you greater freedom and richness in your creative work.

Scenario 4: Commercial Posters GLM-Image can generate holiday posters and commercial promotional images with design-forward compositions and accurate text embedding.

Scenario 5: Photorealistic Imagery Beyond text rendering, GLM-Image excels at generating photorealistic portraits, pets, landscapes, and still life across various shot types and dimensions.

Open Source & Online Experience

Online Experience

API Integration

Open Source Deployment

Finally, we use an image generated by GLM-Image to summarize the model's core highlights.

We hope GLM-Image becomes a powerful assistant for your creativity. We also welcome valuable feedback from the community of users and developers to jointly advance the development of open-source image generation models.