Termsslug: policy-ratio

Policy Ratio

策略比率

Ratio of probabilities under new and old policies for the same action, central to PPO updates

Curated Intelligence

1 items · Page 1/1
Hugging Face BlogT1·Tools
68 pts
vLLM V1 Migration: Fix RL Correctness Before Corrections

The ServiceNow AI team discovered that logprob discrepancies between vLLM V0 and V1 disrupted training dynamics in their PipelineRL framework. By fixing four underlying issues including rollout logprob processing and weight update paths, they achieved alignment with the V0 reference. The key takeaway is to prioritize backend correctness before implementing complex mismatch correction mechanisms.