Termsslug: token-counting

Token Counting

令牌计数

Estimating the number of tokens consumed by text input or output for cost control and context management

Curated Intelligence

2 items · Page 1/1
Simon Willison 博客T2·Tools
38 pts
ttok 1.0 Released: Defaults Switch to GPT-5/6 Tokenizers Based on Compatibility Experiments

The developer tool ttok has released version 1.0, updating its default tokenizer from GPT-4 to GPT-5/GPT-6. Although OpenAI has not officially confirmed the tokenizer consistency for new models, the author relied on third-party experimental data showing identical token counts across GPT-5 and GPT-6 fixtures to make this switch, aiming to provide more accurate cost estimation support for modern LLMs.

Simon Willison 博客T2·Tools
38 pts
llm-anthropic 0.30 Adds Model Refresh and Token Count Commands

Simon Willison released version 0.30 of the llm-anthropic plugin, adding support for Claude Sonnet 5.5. The update introduces a `refresh` command to dynamically fetch the latest model list from Anthropic's API and a `count` command to pre-calculate prompt token usage using their free counting endpoint.