Today for AI
HOT RADAR
llmHEAT 8.3°

AI CURATED HOT STORY · 10/14/2026

vLLM Achieves Bitwise-Consistent RL Training: Eliminating Numerical Drift to Zero KL Divergence on DeepSeek-V4-Flash at 91% Throughput

1 reports archived1 independent sourcesupdated 10/14/2026, 08:00:00
Synthesis & Latest Updates
1 Sources Cross-Validated

To address policy mismatch in RL caused by inference-side optimizations, vLLM engineers modified inference kernels to ensure bitwise-identical logprobs between training and rollout (Diff=0). This approach reduced train-inference KL divergence to zero on DeepSeek-V4-Flash while maintaining 91.3% of baseline throughput, and prevented significant score drops on Nemotron's hybrid architecture. The workflow is now released as an open-source agent skill.

LATEST/To address policy mismatch in RL caused by inference-side optimizations, vLLM engineers modified inference kernels to ensure bitwise-identical logprobs between training and rollout (Diff=0). This approach reduced train-inference KL divergence to zero on DeepSeek-V4-Flash while maintaining 91.3% of baseline throughput, and prevented significant score drops on Nemotron's hybrid architecture. The workflow is now released as an open-source agent skill.

HEAT TRENDHourly heat curve

3 fully observed hours
Now
8.3
Peak
8.720:00
24h change
–

The line compares only sources observed throughout; gap hours are interpolated to keep the trend continuous.

TIMELINECoverage timeline

Total 1 reports · Latest first
  1. vLLM 官方博客T1·78 pts
    • High-performance inference optimizations often cause policy mismatch, leading to spiked KL divergence and degraded rewards during RL training.
    • By reusing modified vLLM inference kernels for training forward passes, the team achieved bitwise-identical logprobs (Diff=0) for all sampled tokens.
    • The method maintained 91.3% throughput on DeepSeek-V4-Flash while ensuring numerical consistency, effectively protecting downstream performance on Nemotron.
vLLM Achieves Bitwise-Consistent RL Training: Eliminating Numerical Drift to Zero KL Divergence on DeepSeek-V4-Flash at 91% Throughput | Today for AI