A/B Test Sample Size

Loading A/B Test Sample Size…
1

Enter your baseline conversion rate

Start with your current conversion rate, the share of visitors who already complete the goal. It anchors the whole calculation, so take it from a reliable window of recent traffic.

2

Set the effect you want to detect

Choose the minimum improvement worth acting on, plus your confidence and statistical power. Smaller effects need much larger samples, so be realistic about what actually matters to the business.

3

Read the required sample size

The tool returns how many visitors each variant needs before the result is trustworthy. Run the test until you reach it, stopping early is the most common way A/B tests produce false winners.

What Is an A/B Test Sample Size Calculator?

An A/B Test Sample Size Calculator estimates how many visitors each variation needs before an experiment begins. Planning this number in advance helps teams avoid making decisions from incomplete data or treating random fluctuations as genuine performance improvements.

The calculation is based on the current conversion rate, the minimum effect worth detecting, the desired confidence level and statistical power. The tool returns the required sample size per variant and performs the calculation directly in the browser.

How Is A/B Test Sample Size Calculated?

Begin by entering the baseline conversion rate of the control experience. This should come from a recent and representative period of reliable traffic.

Next, define the minimum detectable effect, or MDE. This represents the smallest improvement that would be meaningful enough for the business to act on. Testing for a smaller change generally requires a larger sample because subtle differences are harder to distinguish from normal variation.

You must also select the confidence level and statistical power. The significance level controls the risk of identifying a difference when no real difference exists. Statistical power represents the probability of detecting the effect when it is genuinely present. Higher confidence and power requirements generally increase the number of visitors needed.

Once these values are entered, the calculator displays the estimated visitors required for each variant.

Why Calculate Sample Size Before an A/B Test?

A predefined sample target helps determine whether the website has enough traffic to run the proposed experiment within a practical period. Dividing the total required sample by the expected daily experiment traffic can provide an approximate test duration.

Calculating the sample size in advance also reduces the temptation to stop the experiment as soon as one variation appears to be winning. Ending a fixed-horizon test before the planned sample has been reached can increase the risk of selecting a false winner.

The result should be treated as a planning estimate rather than a guarantee. Traffic allocation, tracking accuracy, audience changes, seasonality and overlapping experiments may affect the final outcome. After launch, keep the test configuration stable and evaluate the result using the statistical method selected before the experiment began.

Track the digital heartbeat with Kriko

Subscribe to receive curated insights, news, and ideas shaping the digital landscape.