LLM Tokens per Second Calculator (Memory Bandwidth)
Enter memory bandwidth and model size to estimate generation speed.
Your numbers
Your results
LLM Tokens per Second Calculator (Memory Bandwidth) by TheCalcTools
How the LLM Tokens per Second Calculator (Memory Bandwidth) works
When one user generates text, each new token requires reading every model weight from memory once. So generation speed is roughly memory bandwidth divided by model size.
Real systems reach perhaps 60 to 80% of peak bandwidth. Prompt processing (reading your input) is much faster because it is compute-bound and parallel, and batching many users raises total throughput.
The formula
Worked example
Using the default values shown in the calculator:
| Memory bandwidth (GB/s) | 1000 |
|---|---|
| Model size in memory (GB) | 4.9 |
| Efficiency (share of peak bandwidth achieved) | 70 % |
| Answer length (words) | 300 |
- 1,000 GB/s × 70% ÷ 4.9 GB = 143 tokens per second.
- A 300-word answer takes about 2.8 seconds.
Result:
| Tokens per second (batch of 1) | 142.9 |
|---|---|
| Words per second | 107.1 |
| Time for your answer (seconds) | 2.8 |
| Compared with reading speed (4 words/s) | 26.8 |
Frequently asked questions
Why is a smaller quantised model faster?
Fewer bytes to read per token. Halving the model size roughly doubles single-user generation speed.
Sources and further reading
Related calculators
More in AI & Computing · All calculators
This calculator gives estimates for information only. Check important results independently before you rely on them. See our disclaimer.