Definition
In an A/B test, traffic is split randomly across two (or more) versions of a page, email or ad - variant A against variant B. One clearly defined target metric (click, signup, purchase) is measured. The version with the significantly better result wins.
Statistical significance is decisive: a 51-to-49 split across 100 visitors is noise, not a result. A valid test needs sufficient sample and runtime, otherwise fluctuations get misread as insights - the most common mistake in practice.
A/B testing is core to conversion-rate optimisation, but not an end in itself. You test what carries a hypothesis ('a shorter form raises completions'), not random button colours. Good teams test few, well-reasoned variants - not a hundred arbitrary ones.
Why it matters
A/B testing replaces opinion with evidence. Instead of arguing in a meeting over the better headline, the market decides - and every improvement applies to all future traffic.
In practice
- 01Headline A 'Book a demo' against B 'Try for free' - B lifts click rate by 30%.
- 02Two subject lines in an email send, the winner goes to the rest of the list.
- 03At 200 visitors almost nothing is significant, at 20,000 even small differences are.
