[VIDEO] 95% Test Confidence and How to Learn from Tests that Don’t Reach It

Blog/A/B Testing

28th Mar 2019 –
Originally published March 28, 2019 – Updated December 07, 2020
In this video, Justin Christianson, the author of the CRO best-seller Conversion Fanatic walks you through:
While opinions abound and the debate is still raging, we are not concerned with busting the myth (or not) of 95% statistical significance.
Each test is an investment of resources. Make the most of the numbers it pulls in.
We all want to hit the mythical 95% statistical significance number.
  • Concrete steps to take and tweaks to make if your tests are simply refusing to reach the 95% statistical confidence mark.
  • A couple of things you can look at to derive key insights from tests that haven’t hit the desired confidence level.

It’s a guideline that has morphed into a hard, non-negotiable requirement for most testers.

[embedded content]


[embedded content]