Termsslug: time-to-first-token-ttft

Time to First Token (TTFT)

首字延迟

Latency from request initiation to first token generation, measuring responsiveness

Curated Intelligence

8 items · Page 1/1
The DecoderT2·Tools
78 pts
Anthropic Launches Message Threads Cutting TTFT by 35% for Long Chats

Anthropic has released Message Threads in public beta on the Claude platform, reducing time to first token (TTFT) by approximately 35% for long-context requests by transmitting only incremental changes instead of full conversation history. The feature benefits coding agents and document workflows across AWS, Azure, and Google Cloud, though total token usage remains unchanged.

IT之家 · 智能时代T2·Industry
72 pts
CAICT Report: China's Daily Token Consumption Hits 500 Trillion, Throughput Doubles but Latency Rises

The CAICT released its 2026 MaaS Performance Monitoring Report, revealing that daily token consumption in China reached 500 trillion, nearly quadrupling since late 2025. While average tokens per second (TPS) doubled, time to first token (TTFT) increased due to larger model scales, with significant performance disparities observed between third-party and official platforms.

量子位官网T2·Infra
48 pts
Shi Technology Launches TokenFleet Scheduling Framework, Cutting TTFT by 32.4%

Shi Technology unveiled TokenFleet, a distributed scheduling framework at the CCF HPC conference, designed to boost effective output of domestic compute clusters via optimized request routing, cross-datacenter capacity coordination, and mixed workload governance. Featuring three core technologies—TokenPilot, FleetMind, and SLOMax—the framework claims to reduce Time-to-First-Token (TTFT) latency by 32.4% while improving cost-efficiency per unit of valid output.

IT之家 · 智能时代T2·Industry
78 pts
Infinigence AI Raises Record $310M Series C for AI Infrastructure: Heterogeneous Compute Scheduling and PD Separation Cut Costs

AI infrastructure firm Infinigence AI has completed a 2.3 billion RMB Series C round, bringing its total funding to 4.3 billion RMB and setting a new record for China's AI-native infrastructure sector. Specializing in the 'middle layer' via an M×N paradigm, the company enables heterogeneous chip mixed training and cross-domain scheduling; its proprietary PDD architecture reduces Time-To-First-Token latency by 51.5% and per-token costs by 37.5%. The company has also confidentially filed for an HKEX IPO, targeting a listing in the first half of 2027.

IT之家 · 智能时代T2·Tools
38 pts
Alibaba Qoder Launches Fast Mode on Desktop: Cuts First Response to 3s and Reduces Credits by 27%

Alibaba's Qoder introduced a Fast mode for its desktop client, reducing first-response latency to 3 seconds while cutting credit consumption to 0.8x. The update optimizes inference efficiency, with benchmarks showing it completes common dev tasks roughly three times faster than domestic rivals and four times faster than overseas competitors.

vLLM 官方博客T1·Tools
68 pts
vLLM Launches vllm-metal for Concurrent Inference on Apple Silicon

The vLLM team has released vllm-metal, bringing vLLM's scheduler, paged KV cache, and OpenAI-compatible server to Apple Silicon. By leveraging MLX and Metal for execution, it addresses concurrency issues like Time to First Token (TTFT) and memory growth in local Mac inference. The current v0.29.0 version supports Homebrew installation and introduces batched multi-token prediction (MTP) along with GGUF model support.