Baidu ERNIE Unveils RSI System ERNIE Evolve Powering 5.5 Flash Development

On November 9, Baidu ERNIE disclosed the latest progress on RSI for the first time

On November 9, Baidu ERNIE disclosed the latest progress on RSI for the first time, unveiling ERNIE Evolve, an automated research system involved in the development of ERNIE 5.5 Flash. It can autonomously propose research directions, conduct experiments, verify results, and accumulate experience around goals and constraints. It can both continuously optimize existing paths and explore new directions based on actual results. On three evaluation metrics across two tasks, NanoChat AutoResearch and NanoGPT Speedrun, it achieved the current publicly verified SOTA performance. Currently, ERNIE Evolve has been applied to the inference optimization of the soon-to-be-released ERNIE 5.5 Flash, including performance optimization for Kunlun Chip P800 low-latency scenarios, deterministic inference throughput optimization, and extreme decoding speed optimization.

ERNIE Evolve benchmark performance

Recursive Self-Improvement (RSI) is one of the important exploration directions in the global AI field today. Its core idea is to let AI directly participate in its own R&D process, continuously optimize its own capabilities, and form a self-improvement loop. OpenAI, Anthropic, Tencent, Zhipu, and many other vendors are exploring the introduction of RSI into model R&D.

ERNIE Evolve organizes research with a two-level loop. The outer loop combines historical experimental results and existing knowledge to propose new research directions and arrange multiple research lines for parallel exploration; the inner loop continuously performs code modifications, effect evaluations, and experience accumulation for specific directions. Internally, the system consists of multiple agents working in division of labor. The Director is responsible for selecting research directions, the Selector is responsible for determining the goal and code starting point of each round of experiments, and the Executor is responsible for modifying code and generating new versions. After each round of experiments, an independent evaluation program, the Evaluator, is responsible for evaluating code performance and experimental results, while the Reviewer further checks whether the research has deviated from the goal and whether there are evaluation loopholes. Only improvements that pass evaluation and review can be retained. The code changes, results, and experience of each round of experiments are recorded for reference in subsequent research.

ERNIE Evolve architecture

After integrating ERNIE Evolve into the real inference optimization of ERNIE 5.5 Flash, the system continued to achieve significant gains in the production environment.

• In Kunlun Chip P800 low-latency scenario optimization, ERNIE Evolve spent about 168 hours and went through 4 rounds of direction exploration, successively integrating native fused operators and enabling speculative decoding, then optimizing the host scheduling and synchronization mechanisms and communication paths. The generation speed of the best version increased to 623.6% of the baseline.
• In deterministic inference throughput optimization, ERNIE Evolve spent 3 days exploring 3 directions around deterministic sampling, deterministic switches, and operator parallel strategies. Under the premise of strictly maintaining determinism, it ultimately increased throughput to 133.7% of the baseline, shortened the average per-token generation time by 26%, and reduced first-token latency by 12%.
• In extreme decoding speed optimization, ERNIE Evolve explored a total of 7 research directions, among which 3 directions brought effective improvements through optimization of MoE and mHC operators, matrix multiplication and scheduling strategies, and the critical path of the draft model, ultimately achieving a cumulative inference performance improvement to 114.9% of the baseline, with accuracy consistent with the baseline.

All the above changes have passed the stress testing and correctness checks of the ERNIE team and completed production-level verification.

The performance optimization process for Kunlun Chip P800 low-latency scenarios

The optimization process for deterministic inference throughput

The optimization process for extreme decoding speed

The ERNIE technical blog pointed out that inference infrastructure optimization is the first landing point of RSI in real R&D. In the future, the same research loop will gradually extend to training system optimization, model capability research, and evaluation system iteration, promoting AI to autonomously discover problems, propose hypotheses, design experiments, and verify results. The expansion process will be carried out within clear goals, permissions, and resource boundaries, and will be subject to continuous safety evaluation and human supervision.

On November 5, Baidu officially open-sourced the latest base model ERNIE 5.5 Flash Base, which achieved SOTA performance among models of the same size on multiple benchmarks including knowledge, reasoning, mathematics, code, multilingual, and long context. People close to Baidu revealed that the new ERNIE model may be released soon.

• Technical blog: https://ernie.baidu.com/zh/research/posts/ernie-evolve/

This article is provided by Baidu ERNIE. QbitAI is authorized to repost it, and the views belong to the original author.