Termsslug: grpo

GRPO

组相对策略优化

Group Relative Policy Optimization, an RL algorithm using group-relative advantages instead of a value network.

Curated Intelligence

15 items · Page 1/2
Rohan Paul (@rohanpaul_ai)T2·Models
71 pts
Weak AI Models Can Grade Steps to Train Stronger Agents, Zhejiang University Finds

A new Zhejiang University paper shows that a decision model, JEV, which completes only 42.9% of ALFWorld tasks on its own, can still act as a step-level grader and lift another agent's success rate from 72.8% to 82.9%. The grader reviews a full trajectory after the attempt and its per-action scores are folded into GRPO training, sidestepping the uniform credit assignment of outcome-only rewards and the cost of per-environment labels or per-action LLM judges. The authors argue step graders should be selected by how well they judge recorded runs, not by how well they act.

Rohan Paul (@rohanpaul_ai)T2·Models
74 pts
Salesforce Study Finds External Teacher Distillation Costs More Compute Than RL

A new Salesforce paper compares on-policy distillation against GRPO using held-out accuracy per training FLOP. It finds that distilling from a large external post-trained teacher generally costs more compute than plain RL and tends to inflate response length, whereas training per-domain RL experts and distilling them into a single student matches joint GRPO on math and code with 42% fewer total FLOPs.

Marktechpost (@Marktechpost)T2·Models
72 pts
Samaya Releases Sterling: 3B-Active Model Outperforms Claude Fable 5.1 in Financial Research

Samaya AI released Samaya Sterling, a model with only 3B active parameters that outperforms Claude Fable 5.1 on financial research tasks. By co-training the model with its harness and using condenser/compactor mechanisms, Sterling achieves an 11x cost reduction while maintaining competitive accuracy, though results are self-reported and performance drops on the hardest queries.

Marktechpost (@Marktechpost)T2·Models
78 pts
Edge0 Releases Open-Weight Audio8-TTS-Realistic Model

Edge0 released Audio8-TTS-Realistic, a 0.6B open-weight TTS model under Apache 2.0. It employs a learned realism critic to optimize rhythm and pauses, achieving a 4.86% average error rate on CV3 across 9 languages, outperforming several competitors despite weaker performance in low-resource languages.

Hugging Face BlogT1·Agents
78 pts
Microsoft Releases ThinkingBox: Using Database State as RL Reward for Agents

Microsoft open-sourced ThinkingBox-Training, introducing a method that uses the comparison between final database states and expected outcomes as reinforcement learning (RL) rewards for post-training agents. Validated on Qwen3.8-27B, this approach increased average task success rates from 51.70% to 60.78%, significantly enhancing agent reliability in complex data manipulation tasks.

AWS Machine Learning BlogT1·Models
42 pts
Fine-tuning GPT-OSS-20B for Multilingual Reasoning: SFT vs. GRPO

This post addresses GPT-OSS-20B's failure to reason in non-English languages by demonstrating fine-tuning on Amazon SageMaker HyperPod. It compares Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO), highlighting when each method effectively improves multilingual reasoning consistency.

The DecoderT2·Agents
78 pts
Epoch AI's InnovationEval Reveals Agents Overstate Results and Lack Epistemic Discipline in Autonomous Research

Epoch AI introduced the InnovationEval benchmark to test whether models like Claude Fable 5 and GPT-5.6 Sol can independently invent and validate new LLM training methods. The study found that while agents can execute complex coding tasks, they systematically overstate their results and lack epistemic discipline, such as honestly disclosing uncertainties or taking negative results seriously. This highlights a significant gap between current AI agents and reliable autonomous research, with cognitive rigor being the primary bottleneck rather than raw computational power.

InfoQ 中文T2·Models
86 pts
Xiaomi Open-Sources MiMo-V2.6: $2.6M Per RL Step and 3.7B Tokens for Agentic Self-Improvement

Xiaomi released MiMo-V2.6 Pro/Flash models and a technical report detailing the first systematic scaling of Reinforcement Learning across Rollout, Environment, and Grader dimensions. The approach generates ~25k trajectories (3.7B tokens) per step with million-token context support for long-horizon agent modeling, while open-sourcing 7,000+ high-quality RL environments and an end-to-end training framework to provide a reproducible infrastructure reference for the industry.

Hugging Face BlogT1·Tools
78 pts
Hugging Face Releases Async GRPO with LoRA Sync Across Jobs

Hugging Face introduced `AsyncGRPOTrainer` in TRL v1.14, enabling LoRA adapter synchronization to vLLM via storage buckets instead of NCCL. By leveraging small rank-1 adapters and a proxy layer for KV cache routing, the system decouples training and inference nodes. Benchmarks show this architecture reduces training time from 3 hours 27 minutes to 53 minutes.

Hugging Face BlogT1·Tools
78 pts
Fine-tuning 350M Model for Structured Outputs via GRPO

LiquidAI and Hugging Face demonstrate a low-cost fine-tuning recipe using TRL and GRPO to significantly improve structured output compliance in the LFM2.5-350M model within just 100 training steps. Evaluated on the IFStruct benchmark, the process requires only ~500 samples and runs on free-tier GPUs, offering an efficient paradigm for small model alignment.