Chinchilla Compute-Optimal Model Size Calculator
Enter a compute budget to get the compute-optimal parameter count and training tokens.
Your numbers
Your results
Chinchilla Compute-Optimal Model Size Calculator by TheCalcTools
How the Chinchilla Compute-Optimal Model Size Calculator works
The Chinchilla study (Hoffmann et al., 2022) found that for a fixed compute budget, model size and training data should grow together, with about 20 training tokens per parameter.
Combining C = 6ND with D = 20N gives N = √(C ÷ 120). Models meant to serve many users are often trained on more tokens than this to make them smaller and cheaper to run.
The formula
Worked example
Using the default values shown in the calculator:
| Budget given as | Total FLOPs |
|---|---|
| Compute budget (FLOPs, e.g. 1e23) | 1e+23 |
| GPU-hours | 100000 |
| Peak TFLOPS per GPU | 989 |
| Utilisation (MFU) | 40 % |
| Tokens per parameter | 20 |
- 10²³ FLOPs ÷ 120 → N = 28.9 billion parameters trained on 577 billion tokens.
Result:
| Compute-optimal parameters | 28.9 billion |
|---|---|
| Training tokens | 577 billion |
| Compute budget (FLOPs) | 1e+23 |
Frequently asked questions
Is Chinchilla-optimal always best?
It minimises training loss for the compute spent. If inference cost matters, a smaller model trained longer can be better overall.
Sources and further reading
- Hoffmann et al. (2022) Training Compute-Optimal Large Language Models (Chinchilla)
- Kaplan et al. (2020) Scaling Laws for Neural Language Models
Related calculators
More in AI & Computing · All calculators
This calculator gives estimates for information only. Check important results independently before you rely on them. See our disclaimer.