- Message Threads reduces time-to-first-token by 35% for requests between 500k and 1M tokens via incremental transmission.
- Primary beneficiaries include coding agents, computer-use agents, and workflows processing large volumes of documents or images.
- Threads are encrypted in memory and expire after 5 minutes of inactivity; latency improves but total token usage remains constant.
AI CURATED HOT STORY · 11/3/2026
Anthropic Tests Message Threads, Cutting Latency by 35%
Anthropic has launched a public beta of "Message Threads" for the Claude API, addressing network bottlenecks in long-context interactions. By allowing servers to cache conversation history and requiring developers to send only incremental messages, the feature reduces Time To First Token (TTFT) by approximately 35%.
Technically, this optimization targets network transmission and preprocessing overheads rather than computational costs. Request payload sizes drop dramatically from a median of ~800KB to just 1.8KB, significantly enhancing performance for coding agents, computer-use agents, and large-scale document workflows. However, total token consumption remains unchanged, meaning billing standards are unaffected. Thread data is encrypted in memory and expires after five minutes of inactivity; notably, it does not support Zero Data Retention (ZDR), necessitating local record-keeping for potential reconstruction.
This move signals a shift in AI infrastructure towards optimizing end-to-end interaction experiences. For developers building complex agent systems, Message Threads offer a more efficient communication paradigm, improving real-time application fluidity and scalability without directly reducing compute costs.
Coverage timeline
Total 2 reports · Latest first- The DecoderT2·78 pts
- 宝玉 (@dotey)T2·78 pts
- Server-side context caching reduces median request size from 800KB to 1.8KB and cuts time-to-first-token by ~35%.
- Pricing model remains unchanged; costs are still based on standard token rates, optimizing network transfer rather than compute.
- Threads expire after 5 minutes and do not support Zero Data Retention (ZDR), requiring local maintenance of full records for reconstruction.