What is A/B Testing?
In an A/B test, traffic is split randomly across two (or more) versions of a page, email or ad - variant A against variant B. One clearly defined target metric (click, signup, purchase) is measured. The version with the significantly better result wins.
Statistical significance is decisive: a 51-to-49 split across 100 visitors is noise, not a result. A valid test needs sufficient sample and runtime, otherwise fluctuations get misread as insights - the most common mistake in practice.
A/B testing is core to conversion-rate optimisation, but not an end in itself. You test what carries a hypothesis ('a shorter form raises completions'), not random button colours. Good teams test few, well-reasoned variants - not a hundred arbitrary ones.
Why does A/B Testing matter?
A/B testing replaces opinion with evidence. Instead of arguing in a meeting over the better headline, the market decides - and every improvement applies to all future traffic.
A/B Testing in practice
- 01Headline A 'Book a demo' against B 'Try for free' - B lifts click rate by 30%.
- 02Two subject lines in an email send, the winner goes to the rest of the list.
- 03At 200 visitors almost nothing is significant, at 20,000 even small differences are.
From the journal
- Measure before the relaunch: those four numbers have an expiry dateAfter the rebuild you want to know whether it is better. The numbers that would answer that are still there today, but they expire: Google Analytics deletes event data after 2 or 14 months depending on the setting, the field data history reaches back 40 weeks, Search Console 16 months. Proving a 30 per cent improvement on 20 enquiries a month takes about 23 months of measurement.
- Landing page or homepage: where the ad should leadIn small accounts the ad almost always points at the homepage, because the homepage is already there. Yet 57 per cent of viewing time falls on the first screenful, and Google explicitly judges whether the destination meets the expectation your ad copy created. When a dedicated page earns its keep, when the service page you already have is enough, and from what volume you can test at all.
- Pricing on your website? The case for and against, from a conversion perspectiveOne camp fears that public pricing gives away negotiating room. The other quietly loses enquiries to competitors whose price is one Google search away. This piece sorts out what the 2026 data says: how many buyers walk away when prices are missing, who actually publishes them - and when keeping them private remains legitimate.
