A CTO’s Perspective – Control Groups

July 5, 2016 | By

White Paper Guiding the Customer Journey

As BlueConic’s CTO, every now and again I like to take an opportunity to talk about the product in a bit more depth than Cory, Todd, or some of our other writers. We’ve been up to a lot recently so you’ll be hearing from me more going forward. So without further ado!

In May, we released a whole new feature to BlueConic called ‘Control Groups.’ As our customers took advantage of it almost immediately, we thought it would be a nice idea to share our thinking behind this feature and give some more background on what it actually is, so that you get a better understanding of why we added it and what you can do with it.

For years, BlueConic has delivered messages to specific segments of visitors of your website or mobile app. We call these messages ‘dialogues.’ They are delivered on top of your existing website (integration with CMS optional) and are configurable and editable right on your own site. We designed it to make it very easy for you, the marketer, to create, configure, and edit a new interaction with your customers. There are even advanced features like A/B tests or fully automated tests using our Optimizer algorithm. So, when you want to show a segment of existing customers a specific discount, or show someone who has clicked multiple times a nice lightbox with some offer, you would use a dialogue.

When we saw the adoption of the dialogue feature rise quickly, we added click measurements shortly thereafter so you could see how many people actually clicked on your created dialogues; then we added conversion measurement, to see how many people arrived at the URL you wanted them to arrive on. A click on the dialogue is stored in the visitor’s profile, so you can retarget (or re-re target or re-re-re target etc, you get the idea but use discretion :-)). The combination of a centralized profile across all of your touchpoints combined with the dialogues on your websites proved to be extremely powerful for customers (as evidenced by the hundreds of dialogues each typically has).

So, all good stuff, but the one thing that came up frequently when we discussed with our customers what they’d want on top of those dialogues is to see what has been the difference if you DIDN’T create those dialogues. Marketers spent all this time to create dialogues, and you can see conversions happening through them, but maybe a visitor on your site would have converted anyway, but without that nice dialogue you created. What then?

Thus, Control Groups were born.

Fun fact:, control groups actually originated from medical research. When testing new drugs for example, part of the group gets a placebo drug, and others get the real new drug. When the experiment is done, you can compare the result of the control group with the new group and see if there is a difference.

The beauty of websites is that there is typically so much traffic that it’s certainly not needed to allocate 50% of all your traffic to a control group; a small percentage is already enough. This is especially useful when you are certain that a specific dialogue makes a difference, but you don’t know how much of a difference.

Now, when you create a new dialogue, there is a default control group of 5% of your traffic that is shown the original content. The tailored dialogue will be shown to the other visitors. After just a few days, you can see the lift of that dialogue (lift defined as the performance of a dialogue compared to the original content). You can actually have a negative lift, and the UI will show that in red (positive ones are in green).

control-groups3

To go along with the Control Groups, we also created a new insight called ‘dialogue vs original’ that can show you (over time) the performance of your dialogue compared to the original.

So, in summary, the control groups give a whole new meaning to testing your dialogues. You can now exactly see what a difference your work makes! We hope you will see lots of green buttons, meaning lots of lifts, across all of your dialogues!

We spent quite some time on this new feature to get everything right and we love to get feedback, so please feel free to send praise, complaints, comments, additional feature requests, etc!

Share This Article!

computer icon

See what BlueConic can do for you

Request Demo