How to Buy GDPR Compliant A/B Testing Software

01st Feb 2018 –
Example answer
No
Ah one last thing… US vs EU based servers
Analytics in GDPR is possible as long as any personal data collected is only processed by the first party. ePrivacy is specific regarding cookies, where GDPR is not. Read ePrivacy Regulation draft 15333, article 21;
Example answer
Find all about out company GDPR compliance program, here.
If you’re sending information on what variation and experiences the end-user participated in, we suggest consent for this.
Vendor Question?
What fields do you store by unique visitor in your database (after roadmap completion)?
No more…
Personal data, as defined by GDPR, if different than previous definitions of Personal Identifiable Information (PII). Now, IP addresses, cookies, and UserID are all personal data.

In this article…

Data Minimization

Vendor Question?
How does your tool support Do Not Track?
So when you have an A/B testing tool that stores personal data, and your tests reach significance, you need a tool that freezes those results once end-users fill out the forms you need to have on your website. If the previous paragraphs didn’t already convince you to have personal data stored in your analytics tool, this part of the GDPR law will. I’ll paste the entire article that spells it out here.
Meaning, there’s no “certificate” from an A/B testing tool guaranteeing you’ll avoid that 20 million euro lawsuit from a European privacy authority. But what is possible is that you choose a testing tool designed with privacy in mind. One that won’t get you into trouble when using its default configuration.

• Variation seen
• Goal reached: yes/no (on revenue goals, product count, and revenue)

Here I have my doubts. Will Order ID’s (order numbers or transaction ID’s) be personal data?

• Subsegment: yes/no

So what’s the minimum data needed to run experiments? I mean, do you need to collect every detail of the IP, browser, and device ID, to get results for your A/B test? No. You are collecting them because the tool can collect them. With privacy in the back of your mind, focus on the elements you need to collect and do not store the rest. What is essential to run an experiment is:

• Total end-users that saw specific variation (total and per subsegment)
• Total end-users of a variation that triggered a goal

Technologies which enable mobiles, tables or desktop devices can be connected to their current location. Other organizations may also want to use this type of information to provide location-specific content or advertising, however, location data, especially data about the precise pattern of an individual’s movements over time, can reveal very intimate details about that person’s personal life. This is personal data.

• The average products per variation and
• The average order value per variation…

So when you decide you will store personal data then you should offer a Subject Access Request concerning data access, rectification, and erasure.
Example answer
Nothing
Data Protection Commission says: “You should treat information about the location of a device which can be tracked or located electronically as “personal data”, and comply with the Data Protection Acts in relation to it, if:

Personal Data includes IP-address

Example answer
Frankfurt, Germany
Vendor Question?
What security systems you have in place on databases and system where end-user information is stored?
You are able to collect such data using custom dimensions, custom variables… and tons more.
Now you might ask…if we can’t store IP addresses, how do you run country targeting?
Vendor Question?
Can you describe the cookie content or link to a support article?
Example answer
https://convert.zendesk.com/hc/en-us/articles/204495429-Convert-Experiences-Tracking-Cookies-Structure
Spoiler alert: forget it.
A/B testing tools are free of consent requirements, as long as they are “assessing the effectiveness of a delivered information society service.” But as soon as you go into very specific targeting of end-users, or trying to identify the nature of who is using the website—then you’re moving into an area that requires consent.

Cookies & Personal Data

I don’t think so, but the separation of personal data from non-identifying information (e.g. an order number from the customer’s name and address) is just good practice.

Third countries’ level of personal data protection is assessed by the European Commission through ‘adequacy findings’, which are binding in their entirety to all Member States. Once the “adequacy” of a third country has been recognized, personal data can be transferred to this country without having to take further protective measures.
(8) ‘processor’ means a natural or legal person, public authority, agency or other body which processes personal data on behalf of the controller;
A general principle for transfers (Article 44) and transfers on the basis of an adequacy decision (Article 45) explains that the discussion often revolves mainly around transferring data to countries outside the EU (and Norway, Liechtenstein, and Iceland), where you now have only options:

When you’d like to dive deeper on the different outcomes per subsegment you need, for example:
(Click here to jump straight to the checklist)
The data relates to a living person (a “data subject”); and
It is possible to identify the person to whom it relates from the location data itself, or from the location data together with other information which you have or are likely to acquire.

User ID’s & Personal Data

Every Subject Access Request will need be processed and resolved within the timeframe of at the latest one month of receipt, as described in the text of GDPR.

With GDPR, the more you store, the more you will know—and the higher the chance segments of traffic become real end-users. Along with that, comes a higher demand that you protect the data, request the data (using Subject Access Requests), erase the data and update the data.
Just don’t store it, make sure your A/B testing tool does not store it, and you’ve given yourself one less thing to worry about.

Still Planning on Storing Personal Data?

Vendor Question?
Do you store unique user ID’s (session-ID, device-ID, User ID or email), and/or allow us to store emails or other personal data? If so where is it stored?

• Protected
• Accurate
• Updated on demand
• Erased when required

…that can be recalculated on each new order (without storing the individual orders).

GDPR & End-User Data Accuracy

Vendor Question?
How long are cookies stored?

This tells us that cookies which contain a unique identifier for the device and/or the individual associated with using the device, should be treated as personal data. This idea is also reinforced by Recital 26, which states that personal data is also defined by data that can reasonably be used, either alone or in conjunction with other data, to identify an individual.
GDPR Article 15, Right of access by the data subject

• I promise to not store personal data in my A/B testing tool
• I promise to not store personal data in my A/B testing tool
• I promise to not store personal data in my A/B testing tool

Your A/B testing tool can have the following options:
While ePrivacy Regulation (draft 15333) is not yet a law, and it’s intention it to compliment GDPR—GDPR itself gives us some guidance as to which types of cookies will, by their nature, involve processing of personal data. There is 1 recital that is key to this:
Example answer
Nothing
So… I’ll let you think about that for a minute…
…
…
Will you place a cookie (not suggested) or does the tool just not load at all (we selected that). The downside of the first (placing the cookie) is that cookies might trigger a warning to end-users, and the website owner might get more requests for data from users. The downside of not loading the script at all is that browser extension and users really don’t get tracked in any way (and so visitors tracked for a variation or experience might start differing from the analytics tool). In my opinion, the second issue makes more sense. The GDPR will make a mess of the numbers anyway, since different tools will get or not get consent. So matching them all up to ascertain one “true” result will become nearly impossible, anyway.
That’s it?
GDPR Recital 39; Art.5(1)(c)
Personal data must be adequate, relevant and limited to what is necessary in relation to the purposes for which those data are processed.

• The possibility for them to view the data you collected on them inside experiments, audiences and segments (and goals).
• The possibility to update/rectify some or all data concerning them.
• The possibility to delete their data upon their request.
• The possibility to export all the data. So if a user is requesting it, you will have to export the data linked to his IP address(es) or unique identifier. It can be easily exported as a .csv file for example.

(7) ‘controller’ means the natural or legal person, public authority, agency or other body which, alone or jointly with others, determines the purposes and means of the processing of personal data; where the purposes and means of such processing are determined by Union or Member State law, the controller or the specific criteria for its nomination may be provided for by Union or Member State law;

• the right to access
• the right to restrict processing
• the right to rectification
• the right to be forgotten
• the right to data portability

GDPR Recital 30
Natural persons may be associated with online identifiers…such as internet protocol addresses, cookie identifiers or other identifiers….This may leave traces which, in particular when combined with unique identifiers and other information received by the servers, may be used to create profiles of the natural persons and identify them.

I’m scared to ask, but what about sub-segmentations for variations?

Section 1 of the Data Protection Acts defines “personal data” as:
“data relating to a living individual who is or can be identified either from the data or from the data in conjunction with other information that is in, or is likely to come into, the possession of the data controller”
Vendor Question?
How does your tool assist us with end-user Subject Access Requests?

I mean, we all want to bring better insights and good news to clients and bosses. And it’s awesome to drill down into lost A/B tests and see if some subsegment won.

And, in an e-commerce setting:

And connecting third-party data? Like BlueKai?

Regardless of how you may feel about that definition—it’s law. On October 19, 2016, the Court of Justice of the European Union (the “CJEU”) published its judgment in Case 582/14 – Patrick Breyer v Germany. It held that IP addresses are personal data in certain circumstances. The judgment is broadly in line with the Advocate General’s Opinion in this case, from May 2016.

Do Not Track for Europe Only? Are you Sure?

Do Not Track (or just DNT) is a privacy setting end-users can set in most browsers. You should honor DNT as a signal for how you and your end-users want us to use data. In the last years many end-users signaled that they want a new way of engaging with brands. For example, in Canada—where Canadian federal legislation brings us changes in the Privacy Act. Or in the US Congress with a new stage in the Privacy Shield. Or in the European Union where member states voice the support for a more respectful interaction with end-users. You should hear them and respect them.
But really…do you want to be one of those people that hides the total visitors on your presentation screenshots and amazes people with 1460% wins on one variation on Chrome in Sweden?

1. Do Not Track (Opt out of tracking)
2. Track (Opt into tracking)
3. Null – No preference

This article helps you ask right questions about your A/B testing tool vendor—before the GDPR enforcement date (May 25th, 2018). I’ll deconstruct each element important in GDPR and the upcoming ePrivacy Regulation (draft 15333), and give you a checklist at the end.
So your A/B testing tool should remove any unique identifiers to comply with GDPR. In addition, never store the individual record connected to one cookie on the server side. That would make it fall outside the exception in ePrivacy Regulations.
Vendor Question?
How does your company assist us as controller with GDPR compliance of end-user data?
Example answer
Supporting by not having any personal data stored

• Do Not Track is important enough and is respected by the tool without the need for customers to turn anything on or off. It just works (full disclosure: at Convert, we picked this one…).
• Do Not Track can be turned on or off by users for all the platform (check it’s default for all your projects as admin).
• Do Not Track can be turned on by users of the platform per project, and region (Europe) and should be checked by admin on each project.

GDPR Recital 85; Article 5(2)
The controller is responsible for, and must be able to demonstrate, compliance with the Data Protection Principles.
Yeah I thought so. Now all together with me… say this out loud.

Accountability

So IP addresses are personal data. Period. You can choose to anonymize them unless you receive explicit consent from the visitor (which you may want to avoid). But even if you do, this will always be on the border of consent—and it’ll run the risk of re-identification.

We suggest you do not store IP’s in your A/B testing tool, to stay 100% on the safe side of this new law.

So you decided to store everything in your A/B testing tool? Well storing a session-ID, User ID, or any other identifier means you have to make sure it’s
GDPR Recital 39; Art.5(1)(d)
Personal data must be accurate and, where necessary, kept up to date. Every reasonable step must be taken to ensure that personal data that are inaccurate are either erased or rectified without delay.

Revenue Tracking? Order Numbers as Personal Data?

Rec.50; Art.5(1)(b) Personal data may only be collected for specified, explicit and legitimate purposes and must not be further processed in a manner that is incompatible with those purposes. (Further processing of personal data for archiving purposes in the public interest, or scientific and historical research purposes or statistical purposes, in accordance with Art.89(1), is permitted—see Chapter 17).
Example answer
C5, DoD SRG, FedRAMP, FIPS, ISO 9001, ISO 27001, ISO 27017, ISO 27018, PCI, DSS Level 1, SEC Rule 17-a-4(f), SOC 1, SOC 2, SOC 3
Pseudonymous identifiers (e.g. strings of numbers or letters), which is what cookies often contain, also qualify as personal data. So under the GDPR, any cookie or other identifier that is uniquely attributed to a user and therefore capable of identifying an individual, counts as processing of personal data once this identifier is stored on the server side.
Vendor Question?
Do you store visitor IP? So for your A/B testing tool this means that you could store an individual user in multiple segments and together this individual can be identified as a person. So the advice is not to store individual records ever—just groups of segments in the database. And ask for consent when you start collecting multiple subsegments that involve location data on locations more specific than country. Your A/B testing tool can give you an advice when you are collecting too many segments but, in my opinion, storing individual records is just asking for trouble.

Damn, any good news?

Since we don’t want customers to worry about any possible modification in ePrivacy Regulations, we, as company decided to not store this information in our revenue tracking section of the tool.
Example answer
No, we also don’t allow storing information (also see terms of use). Exception there is Universal ID feature when turned on by customer consent is needed
Disclaimer: Note that this article represents the views of the author solely, and are not intended to constitute legal advice.
Then what actually happens when it’s turned on is important.
Vendor Question?
What fields do you store by the unique visitor in your database (after roadmap completion)?
Article 4 defines data controllers and data processors as below:

By default, web browsers use the null value (no preference), indicating that the end-user hasn’t expressed a desire of whether they want to be tracked or not. Your A/B testing tool should have a setting to not not load ANY scripts when option one, the Do Not Track (Opt out of tracking) in the browser is set.
Jonathan Mayer and Arvind Narayanan are the two principal researchers at Stanford University who have been working on the Do Not Track technology. Their model uses information in the HTTP header to universally opt out of all online tracking.
An additional note on geolocation is that country data itself is too broad, but your subsegments of region, city or zip code will require consent from the end user. Combined with other elements, it may be subsegments that are really small (even just 1 person!). So again, you could store personal data, and this would require the consent of the user.
“As a technology model, Do Not Track is clearly superior to an opt-out mechanism,” said Mayer, referring to commitments by Google other browser to support the technology on their websites.
Example answer
Yes, by default script not loaded (see also our GDPR A/B testing app roadmap)
DNT has three possible values:

• Privacy Shield (for US) and adequate countries (The European Commission has so far recognised Andorra, Argentina, Canada (commercial organisations), Faroe Islands, Guernsey, Israel, Isle of Man, Jersey, New Zealand, Switzerland, Uruguay and the US (limited to the Privacy Shield framework) as providing adequate protection.
• Standard contractual clauses, also called model contractual clauses
• Binding Corporate Rules (covered in Article 47 of the GDPR).

GDPR is massive, and not just restricted to companies in the European Union. Looking now, at ePrivacy Regulations draft 1533 and GDPR, it’s our opinion that you can A/B test using Convert Experiences on all visitors without consent.

Conclusions

Think about it. What would someone do with just the totals of an experiment? If there is no end-user data stored on your A/B testing tool, not even pseudonymous/hashed data, there is no way that end-users can be re-identified.
Example answer
End-user information is not stored and not transferred outside the EEU, unless customer requests debugging logging (stored max. 6 days). Customer information and debugging logs could be transferred on bases of Privacy Shield to US servers.
Most likely your A/B testing vendor is a US company—and then the easiest choice seems to be the Privacy Shield and SCC, as it’s only a one-time effort. For our company this means even though all our customer data (of their website end-users) is stored in carbon neutral servers in Frankfurt–our legal base for the US company would be Privacy Shield and SCC (we just like to be extra safe and have all servers with customer datasets in Germany).
The easiest solution is to use the call from the end-user to call the tracking script and identify the country. Store the combination of variation/country in one sub-segment but not connected to a user. Just add an extra view to that subsegment, so it’s mixed with the hundreds of others in that segment and basically just +1. So again, no way to deconstruct the individual end-user from the database.

A/B Testing: GDPR Compliance Checklist

Data Minimization

Storing a User ID specific to an individual, or any other identifier that you can cross-link to individual personal data… is …you guessed it… personal data. So, these require consent and should be avoided (Recital 30 applies also here, as does Recital 50 below).
Vendor Question?
If end-user or company information is stored outside the EEU what legal base you are using to transfer the information.

Personal Data

Vendor Question?
If end-user or company information is stored outside the EEU what legal base you are using to transfer the information.

Security

Because if your tool DOES require consent—it will have a massive impact on your marketing stack. The big fear is that with a consent-based A/B testing tool, you will lose more than 80% of your traffic to work with.
This may include the storing of cookies for the duration of a single established session on a website to keep track of the end-user’s input when filling in online forms over several pages, authentication session cookies used to verify the identity of end-users engaged in online transactions or cookies used to remember items selected by the end-user and placed in shopping basket. Cookies can also be a legitimate and useful tool, for example, in assessing the effectiveness of a delivered information society service, for example by helping to measuring web traffic to the numbers of end-users visiting a website, certain pages of a website or the number of end-users of an application.
Servers locations are mainly about data transfers – which alone would require consent if involving personal data. We have to look at Chapter 5 of the GDPR on Transfers of personal data to third countries or international organizations. This will give us insight on what to do if the data is stored outside of the European Union.
The GDPR states that cookies improve a user’s internet experience—for example, remembering one’s shopping cart history. Or when completing a form over several pages.
By collecting personal data, you will have to respect the rights of EU citizens and people in Europe at the time of their visit to your website. This includes:
The principle of data minimization is essentially the idea that, subject to limited exceptions, an organization should only process the (personal) data it actually needs to achieve its processing purposes.

GDPR compliance

You are the controller in the relationship between your website visitor (end-user). Your A/B testing tool is the processor.
So double check your A/B testing provider and make sure that there is no way that they story individual records of unique visitors.
Does it matter where you host your website? Or where the A/B testing tracker script is loaded from? Or where you actual dataset is located? Yes, it would not be in this article otherwise 🙂
You should be double-checking that the cookies used by your A/B testing tool don’t have a unique identifier. The unique identifier can make a cookie qualify as personal data.
You need end-user consent. Or, BlueKai and you can be joint-controllers; meaning, on the moment of end-user data collection BlueKai, has asked consent in your name. That being unlikely, at the moment of the writing of this piece, you need consent to enrich you audiences in A/B testing tools with a form of consent on the data. Do ask your data provider, for example, BlueKai.