Tool World

A/B Test Sample Size Calculator

Calculate required sample size for A/B tests

The A/B Test Sample Size Calculator precisely determines the number of participants needed for each test variation and the total experiment. This calculation prevents tests from concluding prematurely with insignificant results or running excessively long, wasting resources. It ensures your experiment has adequate statistical power to detect a real difference, based on your specified inputs. Conversion optimization specialists, product managers, and digital marketers use this tool for planning A/B tests. Users provide their `Baseline Conversion Rate (%)` (current control group conversion) and `Minimum Detectable Effect (%)` (smallest meaningful percentage change). The `Confidence Level` refines the calculation, reflecting the desired certainty in test outcomes, vital for making robust data-driven decisions. The calculator outputs `Per Variation`, `Total Sample`, and `Baseline`. `Per Variation` shows the user count needed for each test group. `Total Sample` is the total user count across all variations. `Baseline` reflects the input `Baseline Conversion Rate (%)`. For valid results, each test group must reach its `Per Variation` sample size, allowing reliable conclusions aligned with the chosen `Confidence Level`.

Inputs

Provide the following 3 values to run the A/B Test Sample Size Calculator:

  • Baseline Conversion Rate (%) [number] (required)
  • Minimum Detectable Effect (%) [number] (required)
  • Confidence Level [select] (required)

What it calculates

This tool returns:

  • Per Variation — primary result
  • Total Sample
  • Baseline (%)

Worked example

For the sample inputs below:

  • Baseline Conversion Rate (%): 5
  • Minimum Detectable Effect (%): 5
  • Confidence Level: 1.645

the A/B Test Sample Size Calculator produces:

  • Per Variation: 42105
  • Total Sample: 84210
  • Baseline: 5%

How it works

The result is derived through the following steps:

  1. p1 = baselineRate / 100
  2. p2 = p1 * (1 + mde / 100)
  3. z = parseFloat(confidence)
  4. pAvg = (p1 + p2) / 2
  5. sampleSize = ceil(2 * pow(z, 2) * pAvg * (1 - pAvg) / pow(p2 - p1, 2))

Frequently Asked Questions

What is an A/B test?

An A/B test compares two versions of a webpage or app to determine which one performs better in achieving a goal.

Why do I need to calculate sample size for A/B testing?

Calculating sample size ensures that your test results are statistically valid and helps avoid misleading conclusions.

How does the A/B Test Sample Size Calculator work?

You enter your expected conversion rates, desired confidence level, and statistical power, and the calculator calculates the required sample size.

What factors influence the sample size for A/B tests?

Factors such as baseline conversion rate, minimal detectable effect size, desired confidence level, and statistical power all impact the sample size required.

Can this tool help reduce the time needed for A/B testing?

Yes, by accurately determining the required sample size, this tool helps you run tests more efficiently and effectively.