Termsslug: throughput

Throughput

吞吐量

The amount of data or requests processed by a system per unit time

Curated Intelligence

8 items · Page 1/1
量子位官网T2·Models
78 pts
MiniMax Launches M3.1 Flash, Tops Domestic Agent Benchmarks

MiniMax officially released M3.1 Flash, the revealed identity of the anonymous 'Space Bunny' model. It scored 29.6% on Agents’ Last Exam (ALE), ranking first among domestic models, and entered the global top tier in coding, security, and Computer Use tasks. With output speeds exceeding 200 TPS and API pricing at just 30% of its predecessor, it offers significant cost-performance advantages.

InfoQ 中文T2·Agents
68 pts
DHH Claims AI Agents Boosted Campfire Performance 150x with Rust Rewrite, Faces Benchmark Criticism

Rails creator DHH announced his team has ceased manual coding, using AI agents to rewrite the Campfire app from Ruby into Rust, Go, and Elixir. His benchmarks claim the Rust version achieves 150x higher throughput than Rails in chat room loading, advocating a paradigm where humans no longer read code. However, the results faced significant community backlash due to lack of implementation details and suspected measurement flaws.

Hacker News AIT2·Infra
48 pts
GPU Calculator for Million to Trillion Tokens

This tool estimates GPU requirements for processing millions to trillions of tokens, citing ~367 H100s for 1 trillion monthly tokens on Llama 3.3 70B. It derives results from throughput, utilization, and I/O ratios, highlighting sensitivity to key assumptions.

腾讯混元 (Tencent Hunyuan)T1·Models
68 pts
Study on LLM RL Batch Scaling: Mechanisms for Accelerated Convergence and Failure Boundaries at High Throughput

A new study analyzes the non-linear impact of batch size on training efficiency in Large Language Model Reinforcement Learning (RL). Experiments show that after learning rate retuning, moderate batches (e.g., B, 2B, 4B) significantly reduce time-to-target by improving throughput. However, scaling up to 16B causes additional sample costs to outweigh throughput gains, resulting in slower convergence. This finding provides quantitative guidance for optimizing compute allocation in RLHF or RLAIF pipelines.

OpenAI 官方博客T1·Infra
35 pts
OpenAI's Custom Inference Chip Jalapeño Debuts

OpenAI revealed initial results for its custom AI inference chip, Jalapeño, claiming industry-leading speed and efficiency. The announcement highlights qualitative improvements in throughput and latency but lacks specific benchmark data or architectural details.

Hugging Face BlogT1·Tools
68 pts
Hugging Face Explains Continuous Batching Principles

Hugging Face published a technical blog deriving continuous batching from first principles, starting with attention mechanisms and KV caching to optimize for throughput. The article helps developers understand the underlying logic of dynamic request scheduling and GPU utilization in LLM inference serving.

Hugging Face BlogT1·Infra
48 pts
GCP Xeon C4 Benchmark: 24x Embedding Throughput Gain

Hugging Face and Intel benchmarked language model inference on Google Cloud's 5th Gen Xeon (C4) versus previous-gen (N2). Results show C4 delivers 10-24x higher throughput for text embeddings and 2.3-3.6x for generation; factoring in price, C4 offers a 7-19x TCO advantage for embeddings. This supports the feasibility of deploying lightweight agentic AI workloads entirely on CPUs.