Termsslug: pass-k

Pass@k

Metric measuring the probability that at least one of k generated samples is correct.

Curated Intelligence

4 items · Page 1/1
Artificial Analysis (@ArtificialAnlys)T1·Models
72 pts
Artificial Analysis Launches VIALS-AA Visual Reasoning Leaderboard, Gemini 4 Leads

Artificial Analysis launched VIALS-AA, a leaderboard built on Handshake AI's VIALS benchmark, evaluating frontier models on 161 life-science visual reasoning tasks. Gemini 4 Carbon leads narrowly at 64% pass^3, followed by GPT-6 Astra at 63% and Claude Opus 5.5 at 61%, a 3-point spread. Refusals on several tasks show scientific artifact interpretation remains a clear weakness for frontier models.

量子位 (微信公众号)T2·Models
78 pts
PSN-RLVR Enhances LLM Reasoning Diversity via Parameter Perturbation

Researchers from Shanghai Institute of AI and others proposed PSN-RLVR, which applies consistent small random perturbations to model parameters throughout the reasoning trajectory. This method addresses the issue where RL-trained models become accurate but lose diversity in solution paths, significantly improving Pass@K success rates and out-of-distribution generalization. The paper has been accepted by NeurIPS 2026.

Apple Machine Learning ResearchT1·Models
78 pts
RLTL;DR: Breaking Zero-Success Barriers via Self-Generated Feedback

To address the failure of standard RLVR in tasks with zero success rates, this paper introduces RLTL;DR, a framework where agents generate concise 'TL;DR' insights after failures to guide subsequent rollouts. By backpropagating on these in-context insights, the model internalizes a direct task-to-insight mapping. Experiments show this method boosts Qwen 3.5 9B's Pass@1 from near 0% to 12-13% on extremely difficult datasets, achieving significant performance recovery with minimal training data.

Kimi 博客 (月之暗面)T1·Models
85 pts
Kimi K2 Launches: Outperforms Top Models on Benchmarks

Moonshot AI released Kimi K2-Instruct, demonstrating superior performance in coding, agentic tool use, and complex reasoning benchmarks. Data indicates it matches or outperforms leading open-source and proprietary models like DeepSeek-V3, Claude Sonnet 4, and GPT-4.1, with notable breakthroughs on SWE-bench Verified and LiveCodeBench.