Time Per Output Token

Time Per Output Token (TPOT) measures the average generation time per output token during the autoregressive decoding phase following the first token. Its reciprocal represents single-stream decoding throughput (tokens per second), bounded primarily by GPU memory bandwidth, model parameter scale, and batch size concurrency.