Know if your test actually has a winner.
Enter visitors and conversions for two variants to get lift, confidence, and a clear verdict. Free, no signup, runs in your browser.
Enter your test results
Visitors or impressions both work, just stay consistent.
Variant A, control
Variant B, challenger
Verdict
97.7%Variant B winsSignificant at the 95% confidence level, p = 0.023.
- Conversion rate AConversions divided by visitors for A.
- 2.31%
- Conversion rate BConversions divided by visitors for B.
- 3.12%
- Relative liftChange in conversion rate, B against A.
- +35.2%
- p-valueTwo-sided, from the pooled z-test.
- 0.023
- Chance B beats AOne-sided probability B truly converts better.
- 98.9%
The test told you which angle won. It cannot tell you what to test next.
Adlicio scrapes real comments and reviews from Reddit, YouTube, Amazon and more, then ranks the pains, objections, and hooks your buyers respond to. Your next five variants are already in their words.
How to check a test
- 01
Enter both variants
Add the visitors and conversions recorded for the control and challenger.
- 02
Check the verdict
Read the confidence, p-value and relative lift calculated from the test.
- 03
Keep or call
Keep testing when confidence is low, or use the significant winner for the next round.
How to call an A/B test
An A/B test is only as good as the discipline around it. Test one variable at a time, the hook, the thumbnail, the offer, never two at once, or the result cannot tell you what worked. Decide the confidence threshold before the test starts and let the data reach it. The most common testing mistake is calling a winner in the first two days because one variant jumped ahead, which is exactly what random noise looks like.
Read the three numbers together. Relative lift is the size of the prize, how much better B converts than A. The p-value is the reliability of that prize, the chance you would see a gap this big from noise alone. A huge lift with a weak p-value is a rumor, not a result. And the chance-B-beats-A number is the one-sided read: useful for a leaning, not a substitute for significance.
Sample size does the heavy lifting. Two conversion rates a few tenths of a point apart can take thousands of visitors per variant to separate, so budget the traffic before you launch, and treat anything under about 30 total conversions as directional. If a test will not reach a useful sample in two weeks, test a bigger swing, a different hook angle rather than a comma.
Significance tells you which angle won. It cannot tell you what to test next. That comes from your customers, and the fastest way to hear them is scraping real comments into ranked angles and objections.
A/B test significance FAQ
What does statistical significance mean in ad testing?
How does this A/B test significance calculator work?
How many conversions do I need for a reliable A/B test?
Should I use 90% or 95% confidence for ad creative tests?
Is this A/B test significance calculator free?
Other free tools
All free tools on Adlicio
Comments & giveaways
Research & insights
- Competitor Finder
- Product Finder
- Objection Finder
- Hook Finder
- Creator Finder
- Competitor Ad Finder
- YouTube Question Finder
- Customer Language Checker
- Amazon Review Analyzer
- Shopify Review Analyzer
- App Review Analyzer
- Ad Longevity Checker
- Reddit Mention Monitor
- Reddit Viewer
- Threads Viewer
- Twitter Viewer
- LinkedIn Viewer
- TikTok Transcript
- Instagram Transcript
- Review sentiment analyzer
- Customer Avatar Generator
- Subreddit Finder