How to use the A/B Test Significance Calculator
- Enter the visitors and conversions for version A, your control.
- Enter the same for version B.
- Pick a confidence level. 95% is the usual choice.
- Read the result, then check the p-value and the range of the difference.
What statistical significance means
Random variation means two identical pages rarely show identical rates. A significance test asks how likely a difference this large would be if the versions performed the same. That chance is the p-value. At 95% confidence, a result counts as significant when the p-value is below 0.05.
How it’s calculated
This tool uses a two-sided, two-proportion z-test. The formula is z = (rate B − rate A) ÷ √(p(1 − p)(1/visitors A + 1/visitors B)), where p is the combined conversion rate.
Avoid common mistakes
- Don’t stop early. Checking daily and stopping at the first significant result produces false winners.
- Plan your sample. Use the A/B Test Sample Size Calculator before you start.
- Run full weeks. Weekday and weekend visitors behave differently.
Also searched as: ab test calculator, ab test significance calculator, statistical significance calculator, split test calculator, ab test p value, conversion significance.
Tools that pair well with the A/B Test Significance Calculator
- CTR Calculator — Enter clicks and impressions to get your click-through rate, for ads, email or Search Console.
- ROAS Calculator — See what each ad dollar brings back.
- UTM Builder — Add utm_source, utm_medium, utm_campaign and friends to any link, see which GA4 channel the visit will land in, and tag a whole list of URLs at once.
- Percentage Calculator — Seven everyday percentage calculations on one page.
Frequently asked questions
Is anything I enter saved or sent?
No. The calculation runs in your browser. Your last inputs are remembered on this device only, so the page opens where you left off.
What p-value is good for an A/B test?
Below 0.05 is the usual standard, which matches 95% confidence. Use 0.01 for high-stakes changes.
Is 95% confidence the chance that B is better?
Not exactly. It means that if there were no real difference, a result this extreme would appear less than 5% of the time.
Why is my result not significant?
Usually the sample is too small for the size of the difference. Keep the test running to your planned sample size.
Can I install A/B Test Significance Calculator as a web app and use it offline?
Yes. A/B Test Significance Calculator is an installable web app (PWA) — installed from your browser, not downloaded from an app store. Click Install app at the top of this page in Chrome or Edge (Windows, macOS, Linux, ChromeOS) or on Android, use Share → Add to Home Screen on iPhone and iPad, or File → Add to Dock in Safari 17+ on a Mac. It then opens in its own window with its own icon, and because it runs entirely in your browser it keeps working offline. Firefox on desktop can’t install web apps. How installing works.