Today for AI
HOT RADAR
llmHEAT 9.6°

AI CURATED HOT STORY · 10/25/2026

DeepMind Uncovers Hidden Bugs in Softmax and Cross-Entropy Implementations

1 reports archived1 independent sourcesupdated 10/25/2026, 13:28:00
Synthesis & Latest Updates
1 Sources Cross-Validated

Google DeepMind researchers identified numerical precision bugs in standard Softmax and Cross-Entropy implementations across major frameworks like PyTorch and JAX, which can cause training instability. They proposed a fix involving computational adjustments without increasing floating-point precision, with Gemini playing a key role in discovering the issue and validating experiments.

LATEST/Google DeepMind researchers identified numerical precision bugs in standard Softmax and Cross-Entropy implementations across major frameworks like PyTorch and JAX, which can cause training instability. They proposed a fix involving computational adjustments without increasing floating-point precision, with Gemini playing a key role in discovering the issue and validating experiments.

Hourly heat curve

2 fully observed hours
Fewer than 3 fully observed hours; trend not plotted yet.

Coverage timeline

Total 1 reports · Latest first
  1. 机器之心 (微信公众号)T2·85 pts

    DeepMind Uncovers Hidden Bugs in Softmax and Cross-Entropy Implementations

    • Standard Softmax and Cross-Entropy implementations in PyTorch, JAX, and TensorFlow contain hidden numerical precision bugs
    • These bugs can alter gradients during specific training phases, causing loss spikes or capability degradation
    • The issue is fixable by adjusting computation methods without increasing floating-point precision or redesigning loss functions
    • Gemini played a crucial role in discovering the bug and running extensive validation experiments
DeepMind Uncovers Hidden Bugs in Softmax and Cross-Entropy Implementations | Today for AI