A new Zhejiang University paper shows that a decision model, JEV, which completes only 42.9% of ALFWorld tasks on its own, can still act as a step-level grader and lift another agent's success rate from 72.8% to 82.9%. The grader reviews a full trajectory after the attempt and its per-action scores are folded into GRPO training, sidestepping the uniform credit assignment of outcome-only rewards and the cost of per-environment labels or per-action LLM judges. The authors argue step graders should be selected by how well they judge recorded runs, not by how well they act.
GRPO
组相对策略优化
Group Relative Policy Optimization, an RL algorithm using group-relative advantages instead of a value network.
Curated Intelligence
A new Salesforce paper compares on-policy distillation against GRPO using held-out accuracy per training FLOP. It finds that distilling from a large external post-trained teacher generally costs more compute than plain RL and tends to inflate response length, whereas training per-domain RL experts and distilling them into a single student matches joint GRPO on math and code with 42% fewer total FLOPs.
Samaya AI released Samaya Sterling, a model with only 3B active parameters that outperforms Claude Fable 5.1 on financial research tasks. By co-training the model with its harness and using condenser/compactor mechanisms, Sterling achieves an 11x cost reduction while maintaining competitive accuracy, though results are self-reported and performance drops on the hardest queries.
Edge0 released Audio8-TTS-Realistic, a 0.6B open-weight TTS model under Apache 2.0. It employs a learned realism critic to optimize rhythm and pauses, achieving a 4.86% average error rate on CV3 across 9 languages, outperforming several competitors despite weaker performance in low-resource languages.
Microsoft open-sourced ThinkingBox-Training, introducing a method that uses the comparison between final database states and expected outcomes as reinforcement learning (RL) rewards for post-training agents. Validated on Qwen3.8-27B, this approach increased average task success rates from 51.70% to 60.78%, significantly enhancing agent reliability in complex data manipulation tasks.
This post addresses GPT-OSS-20B's failure to reason in non-English languages by demonstrating fine-tuning on Amazon SageMaker HyperPod. It compares Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO), highlighting when each method effectively improves multilingual reasoning consistency.
Epoch AI introduced the InnovationEval benchmark to test whether models like Claude Fable 5 and GPT-5.6 Sol can independently invent and validate new LLM training methods. The study found that while agents can execute complex coding tasks, they systematically overstate their results and lack epistemic discipline, such as honestly disclosing uncertainties or taking negative results seriously. This highlights a significant gap between current AI agents and reliable autonomous research, with cognitive rigor being the primary bottleneck rather than raw computational power.
Xiaomi released MiMo-V2.6 Pro/Flash models and a technical report detailing the first systematic scaling of Reinforcement Learning across Rollout, Environment, and Grader dimensions. The approach generates ~25k trajectories (3.7B tokens) per step with million-token context support for long-horizon agent modeling, while open-sourcing 7,000+ high-quality RL environments and an end-to-end training framework to provide a reproducible infrastructure reference for the industry.
Hugging Face introduced `AsyncGRPOTrainer` in TRL v1.14, enabling LoRA adapter synchronization to vLLM via storage buckets instead of NCCL. By leveraging small rank-1 adapters and a proxy layer for KV cache routing, the system decouples training and inference nodes. Benchmarks show this architecture reduces training time from 3 hours 27 minutes to 53 minutes.
LiquidAI and Hugging Face demonstrate a low-cost fine-tuning recipe using TRL and GRPO to significantly improve structured output compliance in the LFM2.5-350M model within just 100 training steps. Evaluated on the IFStruct benchmark, the process requires only ~500 samples and runs on free-tier GPUs, offering an efficient paradigm for small model alignment.