Termsslug: batch-size

Batch Size

批次大小

The number of samples used in one iteration of parameter update.

Curated Intelligence

3 items · Page 1/1
Marktechpost (@Marktechpost)T2·Infra
74 pts
Together AI Launches Together Compiler for Agent-Driven Hot-Reloaded Inference

Together AI released Together Compiler, a system in which AI agents continuously re-tune the inference forward pass as production traffic shifts. Its core is a persistent serving engine that hot-reloads agent-written forward passes without restarts, backed by a 12-layer fast screening tier and a 60-layer slow tier that confirms gains on real traffic. Together claims up to 2.6x TPS over the best open configurations on SemiAnalysis InferenceX, though some gains are marginal and all figures are self-reported; the system is closed source and already live on its serverless endpoints.

腾讯混元 (Tencent Hunyuan)T1·Models
68 pts
Study on LLM RL Batch Scaling: Mechanisms for Accelerated Convergence and Failure Boundaries at High Throughput

A new study analyzes the non-linear impact of batch size on training efficiency in Large Language Model Reinforcement Learning (RL). Experiments show that after learning rate retuning, moderate batches (e.g., B, 2B, 4B) significantly reduce time-to-target by improving throughput. However, scaling up to 16B causes additional sample costs to outweigh throughput gains, resulting in slower convergence. This finding provides quantitative guidance for optimizing compute allocation in RLHF or RLAIF pipelines.

OpenAI 官方博客T1·Models
68 pts
OpenAI Proposes Gradient Noise Scale to Predict Training Parallelism

OpenAI discovered that the 'gradient noise scale,' a simple statistical metric, effectively predicts the parallelizability of neural network training. The findings suggest that complex tasks with noisier gradients can benefit from larger batch sizes, removing potential limits on scaling AI systems. This work rigorizes model training, moving it away from being a mysterious art toward a systematic science.