13th Mar 2019 – 
The same applies if you choose to add a delay before your pop-up:
If you’re still doubting these stats, here’s a real-life illustration at the website level.
The second row aggregates data from campaigns that did not include an offer. Their only pitch was a request to join an email list, subscribe to the website’s newsletter or receive exclusive offers.
All in all, timing is a double-edged sword. If you want to focus on the number of emails you can collect, then it’s best to display your pop-up sooner. If you want to preserve the UX, it’s better to wait. Personally, I lean towards the second option.
Offers do work. But what kind of offers? This A/B test gives us an additional hint.
Offers Matter (a Lot)
The results of this test show that:
How Much Do Offers Matter?
When setting up overlays on a website, one of the most difficult questions to answer right off the bat is: When should I display the pop-up?
That’s why I felt lucky to have access to the pop-up stats of 50 top e-commerce retailers. This gave me the chance to review real-life data and learn the secrets of effective pop-ups.
In this article, I’ll share our findings. Here are the results of an AB test run by a high-end audio system retailer (I took this one as an example, but we’ve reproduced these results more than once):
Do Offers Always Matter?
Does this mean that the best email pop-ups are displayed as soon as a user opens a web page?
But again, it turns out the copy is worth testing. In this section, I’m going to share an A/B test that changed my mind about the importance of wording for pop-ups.
Until now, we’ve reviewed pop-ups’ fundamental elements: the offer, the targeting, and the adaptation to mobile devices.
First, experiments can have a huge impact. Most of the tests mentioned had a dramatic impact on conversions and UX.
What Kind of Offers Work?
While the second focuses on the number of watches you could win.
When the sweepstakes are up, their pop-up can convert up to 23% of their visitors into email subscribers (!!). When the offer is removed, their conversion rate goes down to less than 2%.
In August 2016, Google declared war on “instrusive interstitials.”
Timing Is a Double-Edged Sword
This skateboard shoe retailer alternates between a pop-up inviting her visitors to join its email list and a pop-up offering a chance to win a free pair of shoes.
Why Displaying a Pop-Up on Landing Can Be a Good Idea
Last year, we reviewed 300 ecommerce websites. 76% of the stores using pop-ups were displaying them as soon as the visitors landed on the website.
The first version insists on the total amount.
Webmasters had no choice but to adapt their campaigns to the new policy. It meant either creating a mobile campaign that displays well on a desktop or separating campaigns altogether.
Sometimes, it’s worth sacrificing conversions if it means that your users have a better experience on your website.
This client tested two versions of his campaign. The first version mentions a coupon. It had a conversion rate of 7.6%.
We thought this change would benefit all sides: from visitors who would be happier about their navigation experience to marketers who would collect more leads with improved forms.
- The first version where the pop-up is displayed right away has a lower conversion rate.
- But, it ended up collecting 173% more emails than the version displayed after 2 pages.
These pop-up stats taught us a few things.
As you can see:
In other words, I usually don’t focus too much on the copy of my pop-up campaigns.
Should All Pop-Ups Be Displayed Right Away?
We also assume that pop-ups displayed on landing work better if the user already knows the brand or has visited the website before. Since she already knows what she can find on the website, she’s more eager to share her email right away if it can get her a good deal than if she didn’t know what she would find on the store. Unfortunately, we’ve never had a chance to test this hypothesis so far.
The first takeaway of our study is that a significant proportion of retailers haven’t adapted their strategy to these new guidelines. 17 out of the 50 retailers that we reviewed are still displaying the exact same non-mobile-friendly pop-ups on mobiles and desktops.
2) In general, all the details that make a pop-up matter are worth testing: from the targeting, to the offer, to the design and the wording that you use.
(Although to be honest, I’m still not convinced that the color of a CTA alone can have a huge impact).
Creating Mobile-Friendly Pop-Ups Won’t Help
Google Guidelines for Mobile Pop-Ups
It turns out, things didn’t really go that way.
The upper row aggregates the data of pop-ups that included an offer. By offer, we mean everything that might convince the visitor to subscribe (from a coupon, to a chance to win something, to an ebook, etc.).
This watch store tested two headlines. As you can see, the two mention exactly the same prize: 20 watches, worth 00 in total.
The Impact of the Guidelines on Conversion Rate
Having a higher conversion rate means that you’re annoying a smaller percentage of your users. In the first AB test we mentioned, the difference between version 1 and version 2 is that only 92% of the visitors who saw the second version chose to close it, while more than 96% of the users who saw the second version chose to dismiss it. Put differently, the second version annoyed two times fewer users.
E-mail pop-ups are one such case in point where it’s difficult to find guidance. The available data about their performance is scarce. Case studies are often promotional and look biased. And to my knowledge, it’s difficult to find the results of A/B tests involving overlays and modals online.
What Lesson Can We Draw from These Stats?
Again, the conversion of the version that’s displayed later is slightly better. But it ends up collecting -9% fewer emails.
We were quite shocked by this statistic.
Don’t Forget to Optimize Your Copy
And it looks like it doesn’t impact their pop-ups’ conversion rate. These retailers have an average conversion rate of 4.41%, 27% higher than the stats of their counterparts who chose to comply.
Get ready for a whole lot of stats and A/B tests’ results!
All things being equal, the first headline (13.40% conversion rate) achieved 2x the subscription rate of the second variant (6.70%).
We mentioned stats. So let’s start with metrics about the impact of offers on the performance of opt-in pop-ups.
Hell, no!
Why? Because the longer you wait, the fewer people will see your campaign. And even if the subscription rate is higher, it doesn’t compensate the smaller size of your sample.
Let’s review one last example.
Well, stats reveal that this strategy is not as dumb as it looks.
When looking at these results, it’s no wonder why a majority of the retailers in our sample (42 out of 50 retailers) included an offer in their pop-up. The retailers that include an offer have a conversion rate that’s almost 2 times higher than those who don’t.
Before You Invest in Pop-Ups
Indeed, as a visitor or a CRO specialist, it could sound weird. Why bombard your visitors with a modal right away? It doesn’t give them a chance to learn about your brand, services, etc.
1) The words you choose to describe your offer or content do matter. And they can have a huge impact.
