How this calculator works
The null hypothesis states that both groups share a conversion probability. Pool conversions to estimate this common probability for the z-test standard error. The two-sided interval uses separate observed rates and an unpooled Wald standard error. Random assignment and independent units matter; repeated visits from the same person can violate independence. Low conversion counts can make normal inference unreliable. The confidence setting also determines alpha for the decision. Count/rate mode recovers each sample size only when a whole count is consistent with a non-zero rate.
Formula
pA = a/nA; pB = b/nB; ppool = (a + b)/(nA + nB); z = (pB - pA)/√(ppool(1 - ppool)(1/nA + 1/nB)). Uplift = (pB - pA)/pA.
Worked example
With 100 of 1000 converting in A and 120 of 1000 in B, the increase is 2 percentage points and relative uplift is 20%. The pooled z statistic is about 1.429301 and the two-sided p-value about 0.1529.
Advertisement
Frequently asked questions
Are percentage points and uplift the same?
No. Moving from 10% to 12% is 2 percentage points, or a 20% relative increase.
Can I stop when p falls below 0.05?
Repeatedly checking a fixed-sample test and stopping at significance can inflate false positives. Use a planned sample size or sequential method.
What if conversions are rare?
The count check flags a weak normal approximation. Fisher's exact test is an alternative for a small independent 2 by 2 table.