The developer tool ttok has released version 1.0, updating its default tokenizer from GPT-4 to GPT-5/GPT-6. Although OpenAI has not officially confirmed the tokenizer consistency for new models, the author relied on third-party experimental data showing identical token counts across GPT-5 and GPT-6 fixtures to make this switch, aiming to provide more accurate cost estimation support for modern LLMs.
Termsslug: token-counting
Token Counting
令牌计数
Estimating the number of tokens consumed by text input or output for cost control and context management
Curated Intelligence
Simon Willison 博客T2·Tools
38 pts
Simon Willison 博客T2·Tools
38 pts
Simon Willison released version 0.30 of the llm-anthropic plugin, adding support for Claude Sonnet 5.5. The update introduces a `refresh` command to dynamically fetch the latest model list from Anthropic's API and a `count` command to pre-calculate prompt token usage using their free counting endpoint.