The use of reporting and analytics among marketers has continued to rise—and that’s a good thing. Smart companies are using data to track the performance of their marketing programs and make them work harder. At the same time, the issue of data anomalies can cause uncertainty and make strategists wonder if they’re on the right track.
Sooner or later, you’ll see an unexpected and major spike—either positive or negative—in some measurement of your marketing programs. Or, you’ll make a big shift in your campaign and expect to see a change in your metrics, but none occurs. Your gut reaction to such an anomaly might be to go back to the drawing board (or jump for joy, if the dramatic swing is in your favor).
But slow down for a minute. This is where having greater insight into the sources of your data is crucial. There are two key words to remember that can help you get to the bottom of things: validation and causation. In other words, “Is the anomaly legit?” and “What led to it?”
Step 1: Validation
This is all about detective work. You have to do some homework to determine if the anomaly is valid. Many times, anomalies are based on data errors or omissions. And the more varied your data sources, the more potential spots where those errors and omissions might pop up. Some typical examples:
- Pulling incomplete data—For the specific date range, product, or geography that’s needed. For example, weekly data does not include figures for all days of the week, or data for a specific market or product is missing.
- Changes in how data fields are labeled—For instance, data normally labeled as “MN” has been labeled as “Minnesota.”
- Poor practices in use of TFNs or vanity URLs—For instance, a toll-free number listed in a direct mail piece one week is included on a DRTV spot the next week. Results from one tactic will overlap with the next, distorting the response rates for both.
Validation can take time. Especially when you’re looking at multiple data sources and some of your data providers outside your organization. To keep the validation process efficient, it’s wise to establish a standard procedure for verifying questionable data. As you develop a routine process, keep track of the most common data errors or omissions that you’ve encountered, so you can spot them faster if they show up for an encore.
You also want to keep a strong working relationship with colleagues who provide data, so they’ll be as responsive as possible when you need their support.
Step 2: Causation
When your detective work is done, and you’ve confirmed that the data anomaly is real, the next step is to trace its cause. Start simply by identifying every variable that changed in your campaign, and assess the likely extent of its impact on your results.
If your campaign has a lot of moving parts, it can be a challenge to identify every possible cause. So it’s important to get all hands on deck. Talk to the experts in different areas of your team to help you zero in on the cause. Typical causes for data anomalies include the following:
- Changes in your tactical mix, media weight, offers, or creative messaging
- Changes in competitive activity, such as significant shifts in media weight, new offers, or new creative messaging
Your first goal in finding a cause is to make an educated guess. Beyond that guess, you want to test your theory to see if you can repeat the results you saw in the first anomaly—much like a scientist will repeat an experiment to verify a hypothesis.
The bottom line.
Next time you check your campaign dashboard and see a weird result, take a deep breath before you panic—or celebrate. Those unexpected spikes and dips might not be legit. First, validate that the data behind the results is accurate and reliable. Then, trace the cause of the anomaly to its source. It’s the best way to ensure you’re making informed decisions about the direction of your marketing program—and the ones to follow down the road.