Daniils Visnakovs

Free tool · No signup

Ad A/B test: winner or chance?

Enter impressions, clicks and conversions for your variants. The calculator applies Wilson 95% confidence intervals and gives you one of three honest answers.

Evaluate test The rule behind it

Your numbers

Two ads, three numbers

From your campaign ad report: impressions, clicks and conversions for each variant over the same period. Nothing leaves your browser.

Variant A

how often the ad was served

clicks on the ad

enquiries, purchases, calls

Variant B

how often the ad was served

clicks on the ad

enquiries, purchases, calls

Add a third variant
Variant C

how often the ad was served

clicks on the ad

enquiries, purchases, calls

With three variants, a chance lead is more likely. The calculator names a winner only when the leader separates from every other variant.

No account access, no email, no upload. The calculation runs in your browser, and your numbers are neither stored nor sent anywhere.

Trend, not proof yet

There is not enough conversion data yet, so only click-through rate counts: Variant B leads by +25.0 %, but the interval still overlaps with Variant A. The lead may be chance.

What does that mean in practice?

Let the test run. Click-through rate decides only because conversion data is still too thin, and clicks are not an outcome. Wait until both variants have 15 conversions and 100 clicks, then the calculator will judge the metric that matters.

The weaker variant has 12 conversions, below 15. That is not enough for a reliable conversion rate, so click-through rate decides as a fallback and answers a different question. Once both thresholds are met, the verdict switches to conversion rate.

Conversion rate (conversions per click)

context only
Variant A 5,00 %

95% interval: 2.88 to 8.53 %

Variant B 6,00 %

95% interval: 3.83 to 9.28 %

Variant B leads by +20.0 %; the threshold for action is +20.0%.

Click-through rate (clicks per impression)

fallback metric
Variant A 2,40 %

95% interval: 2.12 to 2.72 %

Variant B 3,00 %

95% interval: 2.68 to 3.35 %

Variant B leads by +25.0 %; the threshold for action is +20.0%.

The calculator uses Wilson 95% intervals. Conversion rate counts conversions per click and decides once each variant has at least 15 conversions from 100 clicks. Below that, the verdict falls back to click-through rate, clicks per impression, which answers a different question.

Next tool Ad preview — write the challenger before you upload it Clean up first Search terms analysis — a test on polluted traffic measures the traffic, not the copy

The rule behind it

Three answers instead of one percentage

Most significance calculators always produce a result, even when the data cannot support one. This calculator has three states: clear winner, trend, and inconclusive.

  • Clear winner. The 95% intervals do not overlap and the relative lead exceeds 20%.
  • Trend. One variant leads, but the intervals overlap or the lead is too small to act on.
  • Inconclusive. The sample is too small. Under 100 impressions per variant, the rule deliberately makes no judgement.

Conversion rate, conversions per click, decides once each variant has at least 15 conversions from 100 clicks. Below that, the calculator falls back to click-through rate.

The four mistakes I see most often in other people's accounts

Decided too early
After three days one variant is ahead, so the other gets paused. At 60 impressions that is a coin that came up heads twice, not a result.
Optimised for clicks
Click-through rate is the convenient metric because it produces numbers fast. But it only tells you who makes people curious. Who brings more enquiries is in the conversion rate.
Winner uploaded again
After the verdict the winner gets added again as a fresh ad. The copy splits impressions with the original and makes the next test useless. Leave the original in place.
Uneven delivery
If one variant gets 80% of the impressions, you are not comparing copy, you are comparing ad rotation. Set rotation to even, otherwise every verdict is off.

The calculator is a statistical model running on your numbers, not a promise. It measures whether a difference is measurable, not whether it holds next month. Season, budget changes and learning phases are invisible to it; I look at those with you in the account during an intro call.

FAQ

A/B test calculator FAQ

When is an ad test meaningful?
Below 100 impressions per variant, the calculator deliberately does not make a judgement. For a conversion-rate judgement, each variant needs at least 15 conversions from 100 clicks. Otherwise it uses click-through rate as a fallback. A lead only counts once the 95% intervals no longer overlap.

What is a Wilson confidence interval?
It is a range for the true rate behind your ad. Wilson intervals remain reliable with small numbers and rates close to zero, which is where real ad tests usually begin.

Why can the result be inconclusive?
Because that is often the honest answer. A premature winner costs budget and can discard the better ad. Waiting only costs patience.

Which metric decides?
Conversion rate per click decides whenever there are enough conversions. Click-through rate is only the fallback while conversion data is still too thin.

Conversion rate per click or per impression?
Per click. That is the number you also see in the account, and it answers the question that matters: how many clicks turn into an enquiry? A conversion rate per impression mixes two things with different causes, how often somebody clicks and how convincing it is after that. So click-through rate stays measured against impressions and conversion rate against clicks. Two metrics, two denominators, and that is exactly why click-through rate is not an equal substitute here, only the fallback.

What does the 20% threshold mean?
Separated intervals tell you a difference exists, not that it is big enough to act on. A statistically clean 3% lead is not worth the effort, and rebuilding a running ad group costs you a learning phase. So I only call a winner from a 20% relative lead upwards. Below that the calculator shows a tendency and advises letting it run.

Can I compare three ads at once?
Yes, via \u201cAdd a third variant\u201d. With every extra variant the chance rises that one of them leads purely by accident. The calculator is stricter because of that: it only names a winner when the leader separates from every other variant, not just from second place.

Are my numbers stored or transmitted?
No. The whole calculation runs in your browser. There is no form, no upload and no account access. Your numbers are stored nowhere and never leave the page. I use anonymous web analytics as described in the privacy notice; impressions, clicks and conversions are not part of it.

Are you testing without ever deciding?

I set up clean ad tests, assess them using this rule, and pause ads only when the numbers justify it.

Write a message Book an intro call
Free & no obligation Reply within 24h You talk to me directly