You may have your own working set of definitions for “deterministic matching” and “probabilistic matching,” but is everyone in your organization on the same page about how customer identities should be resolved?
Since BlueConic offers both deterministic and probabilistic matching types, we’ve constructed this guide to help you understand what these terms really mean and how they fit in your first-party data strategy, so you can make the best decision for your teams.
If you’d like a primer first, we’ve written at length about some of the core concepts underlying this topic, such as customer identity, a single customer view, and, of course, the importance of first-party data.
Here we’ll discuss why you need matching in the first place, the basic definitions of deterministic and probabilistic matching and the advantages and disadvantages of both, and how they play out in the real world.
Identity Resolution Recap
To delight your customers, you have to know your customers. To know your customers, you need to create a complete picture of them as they move across devices, channels, systems, and platforms. This process is known as “identity resolution” because we are, quite literally, solving for identity.
It’s one of the reasons native identity resolution capabilities are a must-have for customer data platform (CDP) vendors, and why you should understand how each vendor approaches it.
Here are some examples of what identity resolution looks like in the wild:
Retail and Consumer Goods: Your customer, Cindy, has one record in your CRM with an email address you use to send offers via marketing campaigns. If you have the ability to collect first-party cookies when she arrives on your website, you can recognize that she’s a returning customer. But as far as your tech stack is concerned, these are two different people named Cindy and you don’t have complete information about either. If you wanted to truly delight Cindy, you would recognize that the two are one and you would display that same discount code you sent to her email when she arrives on-site to make a purchase. Cindy is happy and you’ve improved efficiency. That is CDP 101.
Media and Publishing: Rory visits your online publication via a Facebook ad on his mobile phone. He browses during his free trial period, then, two weeks later, creates an account. Without the ability to link his anonymous profile to his known profile once he gives his email address, you have no way of knowing what interests Rory, what he’s read over the last two weeks, or even if this was his first time to the site. If you can connect the dots, suddenly, you have a better picture of who Rory is and you have a better chance of recognizing him across channels and devices in the future. Now you can serve Rory personalized content recommendations based on his browsing history. Rory will become a more engaged customer and you have a better handle on your marketing dollars.
To Merge or Not to Merge
How do CDPs like BlueConic actually know who someone is and if two (or more) profiles represent the same person?
The answer is through deterministic and probabilistic matching.
Deterministic matching is the process of identifying and merging two distinct records of the same customer where an exact match is found on a unique identifier, like customer ID, Facebook ID, or email address.
These identifiers often come from a user that has authenticated (i.e. filled out a form or logged in) or from a system that generates a unique ID. The two Cindy records for the CPG/retail customer, for instance, were living in disparate systems, but now you’ve de-siloed these systems to create a single, unified customer record for that shopper.
The key here is that you are looking for an exact match on the first-party data you have the most confidence in - and the ability to use any combination of identifiers to match on.
But what about scenarios where part of your database is full of records with no exact matches? Enter probabilistic matching.
If you were to browse through your customer databases, you probably have thousands, hundreds of thousands, or even millions of incomplete or inaccurate customer profiles.
For instance, many retailers using BlueConic have physical stores, where customer information collected by sales reps is often misspelled. Not to mention most online channels are visited without an identifier, which can result in duplicate records for the same customer.
This is where probabilistic matching, sometimes called “fuzzy” matching, comes into play.
Probabilistic matching uses algorithms to score and weight the variables and inconsistencies present in these profiles, to essentially answer, “What is the probability that these records are the same person?”
A probabilistic model can determine if/when profiles should be merged or not merged, depending on whether they reached a certain threshold.
The key here is to decide what threshold you are comfortable with and matches will be made based on those rules.
For instance, you may decide that “Cindy Johnson” is likely the same person as “Cindi Johnson” given enough common attributes, but for a “Cindi Jonsen” you might wait until more information becomes available. All of these decisions should be informed by your specific business use cases, as you’ll see below.
Better Together
Regardless of where you stand on which method should be used, there is one incontrovertible truth: deterministic matching is more accurate than probabilistic, by definition.
But don’t make the mistake of thinking the conversation ends there. Because the answer to which is actually “better” is really a false dichotomy. Everything depends on your goals.
The characteristics of each match type tell us a lot about their utility in the real world.
For instance, we know that deterministic matching is more precise but occurs less frequently. We have a lower chance of someone identifying themselves via login or filling out a form. It’s far more common for anonymous users to interact with your site, then disappear without a trace, especially with third-party cookie deprecation.
There are levers you can pull to encourage users to identify themselves, such as access to more content or a discount code. That said, unknown customers will always outstrip known customers. We also know that errors happen all the time in collecting information about customers.
The dynamic thus becomes precision vs. scalability. What is more important to you for that job in particular?
The Golden Record Approach
The ultimate goal of any CDP should be to liberate your data so that you can unify it into a single customer view.
Using deterministic matching to merge profiles of customers from disparate systems and platforms not only makes it accessible but also gives you confidence in the quality of the data and mitigates your customer data risk.
Because you are using exact matches, you can be sure that the two records represent the same person. All BlueConic customers use this approach to build their single customer view.
This allows for increased agility and flexibility, improved analytics and insights, smarter customer engagement, all of which lead to measurable growth outcomes.
The House Cleaning Approach
BlueConic does a ton of work behind the scenes to clean and normalize the data as it is ingested from other systems. CINDY can easily become Cindy through this type of process, but our earlier example of Cindy vs. Cindi would not be corrected, despite being the same person.
For companies where this is common, such as those with a lot of manual data entry like bricks-and-click retailers, a probabilistic matching strategy layered on top of the deterministic merge rules could recognize more profiles as being the same and help consolidate them.
What’s more, you can introduce logic that will merge the records and drop the name that the algorithm determines is inaccurate.
You may have many users flowing in and out of your channels anonymously. With a sound collaboration between probabilistic and deterministic matching rules, you could increase your likelihood of recognizing them.
The result? You’ve increased the number of customers you can recognize and cleaned up your databases.
The Campaign Approach
Suppose you’ve taken the Golden Record approach but have some campaigns where scalability is more important than specific targeting, like the launch of a winter clothing line targeting the northern United States.
Rather than modifying your existing profile database and reducing your confidence level, you can generate a segment using probabilistic matching and then activate against that segment.
This campaign is not as targeted as others, so the cost of misidentifying is much lower and the potential upside of recognizing more individuals across channels and devices is potentially huge.
In this case, it makes sense to use probabilistic matching to boost your reach.
As you can see from these examples, the choice depends on your goals and the risk of error.
At the end of the day, having a sound deterministic matching strategy is absolutely essential to building a robust profile database. Probabilistic matching can then enable you to expand your use cases when necessary.