A/B Test Significance Calculator (Conversion Rates)
Enter visitors and conversions for version A and version B.
Your numbers
Your results
A/B Test Significance Calculator (Conversion Rates) by TheCalcTools
How the A/B Test Significance Calculator (Conversion Rates) works
The test compares the two conversion rates using their pooled rate to estimate the standard error, giving a z-score. The p-value is the chance of seeing a difference this large if the versions were really the same.
Decide the sample size before you start and do not stop the test the moment it looks significant: peeking inflates false positives.
The formula
Worked example
Using the default values shown in the calculator:
| Visitors to A (control) | 10000 |
|---|---|
| Conversions in A | 420 |
| Visitors to B (variant) | 10000 |
| Conversions in B | 495 |
| Confidence level | 95% |
- A converts 4.20% and B 4.95%, an uplift of 17.9%.
- z = 2.538, p = 0.0111: Significant at 95%: B is better.
Result:
| Conversion rate A | 4.20% |
|---|---|
| Conversion rate B | 4.95% |
| Relative uplift of B | 17.9% |
| z-score | 2.538 |
| p-value (two-sided) | 0.0111 |
| Result | Significant at 95%: B is better |
Frequently asked questions
What does 95% significance mean?
If there were truly no difference, a result this extreme would happen less than 5% of the time.
Sources and further reading
- Wikipedia: A/B testing
- Wikipedia: Two-proportion Z-test
- NIST/SEMATECH e-Handbook of Statistical Methods
Related calculators
More in Business & Marketing · All calculators
This calculator gives estimates for information only. Check important results independently before you rely on them. See our disclaimer.