“We implemented it, and two or three days later we started seeing all of the Phrasee subject lines going up, and the human subject lines starting to go down, down, down,” he says. “It was really interesting to see in real-time, seeing our human control, and seeing all the alternatives that Phrasee was getting – and arriving at more or less our winner one, which was actually the one that was the most personalised.
“I always like to start with a business problem first, and ask if personalisation is a good solution. I think that’s really important, because the word ‘personalisation’ is bandied around everywhere, and as Parry said, it’s not the answer for everything.”
At its heart, the methodology used by Phrasee X is a multi-variant test, but instead of the test being carried out at a single moment in time, it’s carried out continuously, in what Malm refers to as “a very statistically efficient manner”.
Real-time optimisation of email subject lines sounds fantastically sophisticated, but how does the technology work in practice?
“Especially in marketing, personalisation is sometimes used as a destination – ‘We want to be personalised,’” adds Lopes. “It’s like a train station, where you arrive – ‘Oh hey, look at us, we’re a personalised company!’ You sometimes see it as a company objective: personalisation.
The importance of language
“They’re the highest-revenue, because they’re so timely, and they hit the customer right at the best point,” says Lopes. “So, we decided, ‘Let’s start here first, and then we can think about the other channels where we can migrate this type of technology and this type of thinking.’
“I don’t think personalisation as a pursuit is important,” says Malm. “Ultimately, results are the most important thing. Now, often personalisation for a specific purpose is the most effective way to optimise that end goal – but not always.”
“We were doing really good personalisation on a channel-by-channel level,” explains Saul Lopes, Head of CRM at Currys plc. “[But] we found the opportunity to actually take a step back and look at end-to-end personalisation moments – how could we give customers a complete personalised experience, end-to-end?”
At present, Phrasee’s product suite is focused on enterprise-rate B2C customers like Currys, but otherwise Lopes and Malm believe that there isn’t any reason why more brands couldn’t make use of this technology. “I sometimes think, ‘What are people waiting for?’” says Lopes. “It’s such an easy part of tech – they’re taking the complexity away.
Which is not to say, of course, that marketers should focus exclusively on quick wins or that it’s never worth carrying out more complex marketing activations – but it’s important not to overlook the simple, fundamental things.
The importance of personalisation is such a talked-about subject within marketing, with practitioners constantly discussing how to “do personalisation”, how to carry it out more effectively, how to personalise at scale, what the future of personalisation will be, and much more besides. However, both Malm and Lopes emphasise that personalisation shouldn’t be an end in and of itself.
“It’s not creating the touchpoint, or even the main content message – it’s not deciding that. It’s just deciding, ‘Okay, this communication has this objective; the current human language is this. How can we better optimise this language so that it communicates better to customers, and drives better ROI and conversion?’”
“But when we started chatting with Phrasee, it kind of opened my eyes to – why not start with language first? How can we personalise our language so that we can better communicate with customers and better engage?”
“The way that the performance optimisation system works is that it learns from past behaviour to project onto the future, while bringing in things that it’s not sure about,” says Malm. “By bringing in all this uncertainty in conjunction with past actions, you can actually create a very strong, highly accurate model, which can take on ambiguity that happens in marketplaces and react very, very quickly. That’s what makes it inherently powerful.
The future of AI in marketing
How real-time subject line optimisation works
“But marketers should not aspire to personalise everything, because that’s not how marketing works. They should aspire to identify the specific use cases where personalisation can be most effective, double down on those, and then focus on how to optimise their top-level messaging for the mass market.”
A couple of months on from implementation, the team was seeing an overall increase in open rates of 57%, with double the amount of traffic coming to the Currys website, and a 9% improvement on conversion when compared with the human subject line. All of this led to an additional £2.5 million in yearly incremental online revenue for Currys.
Harnessing the Power of Personalisation Best Practice Guide
“We personalised on the brand the customer was researching on; we referred to the price; for credit customers, we’d refer to the monthly price; and Phrasee learned all of this by itself, and was able to improve from our human subject line, which was, ‘Oops! Did you leave something in your basket?’
They then began to drill down deeper into the “I’m hunting for something” use case and consider what it meant for things like CRM communications; for a customer who went into the My Account section of the website; for a customer who went into a store.
“Another area that we’re also looking at is end-of-product life cycle; so, if a washing machine is seven years old, if a TV is seven years old, that’s a long time ago that you engaged with us – so, how can we use Phrasee, and how can we use Phrasee X, to get that saliency and get the message out there of reactivating with Currys after a long purchase cycle?
Announced in May of last year, the merging of the four brands was complete by October. In its wake, the newly-unified Currys decided to take another look at the way it approaches personalisation across the business, and improve its approach to personalised communications.
“We’re always working with Saul, and Saul’s team, to find new applications and new use cases,” adds Malm. “The great thing about Phrasee’s customers is that we’re always pushing them, and they’re always pushing us. That’s why the technology – in terms of capabilities – has advanced so quickly, and why Phrasee as a business has grown so much in the last few years.”
The Phrasee tool also learns based on the language that a brand is already using in their communications, in order to closely replicate the brand’s voice. In other words, it’s following a path laid down by human copywriters, and then optimising the language of each subject line based on performance, and using this to learn about what works.
AI copywriting: An overview
Midway through 2021, electricals company Dixons Carphone announced that it would be merging its four technology and retail businesses – Currys PC World, Dixons, Carphone Warehouse, and Team Knowhow – under the Currys brand name.
“If you think about it, we’re all consumers here,” adds Malm. “Though we may not realise it, we’re all impacted on a daily basis by things as simple as a subject line in an inbox, or a strapline on an ad somewhere – little simple things like that.
“By doing this personalisation exercise, what I felt is that we sometimes forget the importance of language within these use cases,” says Lopes. “Generally, when we talk about personalisation, what’s the very first thing that comes to everyone’s minds? It’s, ‘Okay, how are we going to personalise the product on the webpage? How are we going to create some piece of content? How am I going to create a new video?’
“It’s all a balancing act,” says Lopes. “My boss says this many times – in retail especially, you have to have “the big mo”, which is “momentum”. And I think quick wins and improvements like this give you that momentum that you need in order to invest on the bigger projects, on the higher-cost projects. By having all of these really good stories to start off with, you gain that momentum internally to be able to invest on things that are a little bit more complex and harder to do.”
“We had a really great opportunity to trial … language that is completely personalised and customised to each one of our customers,” Lopes goes on. “The language that we’re using right now is completely unique on a customer-by-customer level, because we added a lot of data variables within that language; but not only that, it’s also optimised by Phrasee in real-time. And in the future, we can leverage that language and leverage those learnings to the other channels.
“That was very interesting to see in real-time, and see week-by-week how certain ones were going higher, certain ones were going lower, then we’d inject a few more, then we’d inject a few more… And it’s quite incredible that we got an improvement on something that has a very high bar. The engagement on these type of communications – we’re talking about open rates of 50, 60% already. And even on that, we were able to increase the engagement.”
He uses the example of a 30-second Superbowl ad, widely considered to be one of the most effective forms of advertising in the world – and which is fundamentally unpersonalised. “There are points in time where personalisation should absolutely not be used, and marketers cannot – and should not – aspire to personalise everything.
“We purpose-built this AI to make the choices that are statistically most likely to maximise ROI,” he says. “It’s very important that we don’t apply pre-conceived notions about how an experiment could be run. The methodology is not fixed – depending on the circumstances of the given experiment, Phrasee’s AI might augment existing language that is working well, or it might interject new, never-seen-before language into the mix.”
The results
This thought process gave rise to a new collaboration with AI-powered copywriting platform Phrasee, with whom Currys had already been partnering for several years, and led to Currys becoming the first brand to experiment with Phrasee’s new real-time language optimisation technology, Phrasee X.
Malm references a multi-variant optimisation methodology called the “multi-armed bandit” methodology – a more complex version of A/B testing in which marketers have different proportions of different variants operating alongside each other. “Much like a horse race – if you have ten horses running down the track at Ascot, then you invest more in the horse at the front, and you invest less in the horse at the back. And over time, we can add in more horses to the race, so we’re constantly doing the champion versus challenger thing,” he explains. So, for email subject line optimisation, substitute ‘horses’ with ‘email subject line variants’.
The team highlighted moments such as, “When we do fail, how do we better say I’m sorry? How do we better communicate that message and recover that customer?” and, “If a customer is hunting for a product, how can we better assist that sale?” They looked at ways to treat VIP customers differently, and how to reconnect customers who had gone “completely cold”.
Prior to this, Currys’ collaboration with Phrasee had mostly been focused on its broadcast messages and re-engaging its mass communications. However, with Phrasee’s realtime technology, Phrasee X, Currys identified an opportunity to focus on communications that were “targeted and timely”, and personalise them on a one-to-one level. The idea came about at one of Currys’ regular catch-ups with Phrasee, and the two teams alighted on abandoned basket emails as a “quick win” for trialling Phrasee X in Currys’ marketing communications.
I spoke to Lopes, together with Phrasee CEO Parry Malm, about how Currys came to implement real-time optimisation of its abandoned basket email subject lines, how the technology works and what results it gained, and why marketers shouldn’t view personalisation as an end in and of itself – but why it can be a powerful solution to certain business problems.
Lopes reflects on how such a simple change – optimising an email subject line (albeit in a technologically sophisticated way) – was able to bring about such impactful results. “We just personalised one sentence,” he says. “I constantly go back to it, and I think to myself, ‘Here am I, spending months and months of my life trying to do so much more complex and hard stuff … and here by just changing a sentence, I was able to increase conversion by 9%. So that was a big, ‘Wow’.
“That’s why the abandoned basket example is such a nice example. What a layperson would do is they’d say, “Let’s just run an experiment, find the best one, and stick to that for the rest of time.” But time stops for no-one. We cannot rest on our laurels and assume that what worked yesterday will work tomorrow.”
“There are very specific and very powerful use cases for personalisation – for example, personalising abandoned basket messaging makes perfect sense. People are deep in the buying cycle; they need the extra prompts to go through to actual conversion. Hyper-personalising that makes sense.
Upon implementing Phrasee X across Currys’ cart abandonment emails, the team began to see “immediate results in terms of our open rates going up,” in Lopes’ words.
“It’s something that we’re constantly changing and reviewing. There’s always that element of human review of this language.”
Speaking of destinations, where do Phrasee and Currys plan to go next with their partnership? One area that Currys is currently looking at is its programme for helping customers to make better use of their newly-purchased tech. “We do have a really well-structured welcome programme for our customers where we help them better use the tech that they just bought; so it’d be really interesting to use Phrasee, and Phrasee X technology, to improve that customer journey that’s completely experience-led and service-led,” Lopes says.
“We’re really working on an end of product lifecycle programme where we can reactivate customers and upgrade their tech to greener and more environmentally friendly products.”
“The problem with that is that when your goal is to optimise the overall ROI on a campaign-to-campaign basis, there are some inherent inefficiencies in your methodology – and we’ve recognised this for some time.”
“The best way to think about it is like this: back in the day, before Phrasee X existed, we had a lot of customers – including the good folk at Currys – who would use Phrasee in the broadcast sense; a standard test and deploy style of campaign,” says Phrasee’s Parry Malm.
When Currys initiated its end-to-end personalisation project, the team set out to isolate ten moments that had the greatest customer value.
“We sometimes think about personalisation as something a lot bigger and grander – it just wasn’t something I was [initially] considering. But we stumbled on it, and just found such a great opportunity.”
“People can spend weeks and months trying to create this huge, intellectually pure, academically sound system that has all these bells and whistles – but if you’re not memorable, if you’re not noticeable, if you don’t get that subject line right – then it’s a big waste of time.”
On top of this basic methodology is the real-time language optimisation that makes Phrasee X unique. “We’ve got all of these different variants running simultaneously, being optimised in the statistically most efficient way possible; but we can then in real-time amend, augment and inject a new language into that competition,” Malm explains. “So, you’re constantly learning and constantly improving as time goes on.
Personalisation is not a destination
“One more thing to bear in mind is that the more uses Phrasee has and the more language it tests out on a customer-by-customer basis, the more data points there are from which it can draw conclusions to project forward. So, there’s a certain network effect that stands out from heavy use.”
“So luckily, our three years’ worth of data and learnings really helped us to start high when we introduced it!” adds Lopes.
“I know it can sometimes sound a little bit scary – ‘Oh, AI, what does that mean? Do I have a robot writing my copy?’ I think there’s still those type of ideas. But what it is, is optimising an existing touchpoint, an existing communication, an existing CRM strategy, and optimising that language in order to say it better.
I ask whether any brand could benefit from implementing technology like Phrasee’s, and if they can, what might be holding them back from making use of it.
However, assessing performance in the context of email marketing presents some unprecedented challenges. “The delivery of response metrics – opens, clicks, conversions – are not uniform,” says Malm. “You can’t just roll out a multi-armed bandit and hope for the best. We had to do a huge amount of research and engineering work to make it feasible.”
“And it isn’t Phrasee introducing brand-new language without any human oversight or control – I know that that can sound a little bit scary for many. It’s Phrasee coming to us and saying, “We’ve looked at your language, and we’ve looked at your algorithms; we think we should put these new ten subject lines within the pot and test”. Of course, we have to go through business standards, we have to see if everything is above-board; then we input them into the pot, and we optimise. And then you’ll see, three or four weeks later, three or four have left the pot and are no longer important, and we start seeing the winning subject line on the top.
He also emphasises that the AI doesn’t make choices about language without any human input or oversight. “Every week or every two weeks, Phrasee will come back to us and say, “Hey, why don’t we try these ones out?”
“I think when we started off the tests, we went through 40 or 50 different variants, until we actually landed on the top three,” recalls Lopes. “And it’s still optimising. We’re never stopping.”
“But [Currys] kind of flipped that around – we focused first on, ‘What are the business problems we’re trying to solve? And is personalisation a good solution to solve that business problem?’ For example, a big business problem might be, ‘How do we retain our most loyal customers?’ Maybe if we do a few more touchpoints, if we personalise a little bit better, if we give them visibility across our business and give them special treatment – then, we’re able to save them.
A graph showing how quickly open rates began to climb once the Currys team implemented real-time subject line optimisation on their abandoned basket emails. (Image: Currys & Phrasee)
“We sometimes get so hyped up in the data that we have, the data that we don’t have, on complex use cases and complex activations, that we forget the low-hanging fruit that’s right in front of our faces.”
“We’re really, really focused on: what are the ten moments that have the most customer value? And how we can improve the way we personalise not just on email, but on web, call centres, in a retail shop, et cetera, et cetera – throughout all of our channels,” explains Lopes. “That was a really good exercise that we did across the business; and out of it came these opportunities.”
