Measure Marketing Campaign Success with Control Groups
How effective are your marketing campaigns? If you know your campaigns are working, you can keep doing more of the same. Otherwise, you need to stop, rethink and try something different. Using control groups is a great way to test campaigns – but what is a control group, and how do they work?
What Is a Control Group?
In a marketing context, a control group is a subset of a selected audience segment that will be excluded from receiving a specified campaign message or promotion during a test period. A control group is what science calls the experiment placebo.
A control group should be picked at random to represent the entire segment. This enables like for like comparisons. It should also be relative to the number of people in the segment. For example, in a segment of 10,000, 5 percent or 500 people, would be sufficient.
As a general rule, smaller campaigns demand a higher percentage to develop a control group. So, if your target audiences were less than say 2,000 customers, your control group should be around 10 to 20 percent.
Another variable to consider when determining the size of your control group is expected response rate. If you are targeting dormant customers and are therefore expecting a low response, you need a larger control group to produce a statistically significant result.
Why Is a Control Group Important?
A control group allows you to measure the impact of your marketing campaign on a group of specific audiences. By comparing the responses of your control group with everybody else in the segment who was exposed to your campaign (the test group), you can determine its effectiveness.
Using a control group, it is possible to compare the measurable and meaningful KPIs that actually have an impact on your bottom line, such as profits and ROI. It also allows you to check whether your efforts are doing more harm than good – adding a third variable to traditional A/B testing.
Using a control group may seem counterintuitive. After all, by removing a percentage of your audience, you are setting yourself up for a loss in revenue. However, this short-term loss is outweighed by the opportunity it offers to make better, more confident strategic decisions.
Are Control Groups Needed for Every Marketing Campaign?
While control groups are useful for gaining marketing campaign insights, they are not necessary for every campaign.
One particular instance in which control groups are not necessary or effective is when targeting very small audiences of only a few hundred people. That is because the size of the control group in these cases would be too small to be statistically significant.
In addition, using control groups for all your marketing campaigns could potentially result in a substantial loss in revenue. It would be sufficient as long as you use control groups intermittently across campaigns to ensure that your insights are still valid.
How Does a Control Group Work?
Implementing a control group in your marketing is a scientific but relatively simple process. Here is how it works step by step with an example:
1. Select a customer segment
Decide which group of customers you would like to target with a campaign. For example, you might decide to target 7,000 customers who haven’t purchased in three months by sending them a promotion offering a 20 percent discount.
2. Send campaign to your segment
Send your campaign to everyone in your chosen segment or test group (6,650 people), excluding the people in your control group (350 people). By excluding your control group, these customers can continue to act as they usually would, unaffected by the promotion.
3. Evaluate the results
Once your test period is up, compare the results of everyone in your test group to the results of those in your control group.
If 10 percent of your control group made a purchase but 25 percent of those in your test group who received the campaign message with the discount made a purchase, it would be a good indicator that your campaign worked. However, if your control group and your test group respond in a similar way, it is likely that your campaign was ineffective.
In each experiment, it is important to consider whether your results are statistically significant or not, meaning if the differences in your results are marginal. For example, if 25 percent of customers in your test group made a purchase, but 23 percent in your control group made a purchase, and then you should run another test to compare results.
4. Use findings to adjust your marketing strategy
Over time, you will start to see patterns in terms of what campaigns or offers drive the most revenue or deliver the most ROI – and which segments are most responsive. Using these insights, you can then tweak your future marketing strategy accordingly.
Using control groups is a great way to measure the effectiveness of your marketing campaigns and make better strategic decisions. When marketing control groups are combined with customer analytics and machine learning platforms, they can be even more effective.
* Have a question? Or do you want to learn more about how to leverage machine learning for effective segmentation? Get in touch with our team today for an exclusive consultation.
WE ARE HERE TO HELP
YOU MIGHT ALSO LIKE
Much continues to be made of artificial intelligence’s (AI) capabilities, and it has proven its value in business in areas such as cost-cutting and efficiency. Singapore is undoubtedly a hub for AI in Asia Pacific (APAC), particularly in Southeast Asia (SEA), and in 2019 we will continue to see steady progression towards AI adoption in Singapore. According to a survey conducted by Appier and Forrester in September 2018, 50 percent of businesses in telecom, insurance, finance, IT and retail in Singapore have adopted AI solutions somewhere in the organization. Thirty-one percent of those who have not yet adopted AI plan to do so by mid-2019. Businesses in Singapore are interested in applying AI to improve existing solutions and processes, rather than using AI to drive innovation (which is the case in places such as Taiwan, South Korea and Indonesia). Business leaders continue to understand that AI is a critical part of transforming and staying competitive, though many will face challenges as they find that AI adoption and implementation is harder than they have been led to believe. Critical research into business processes and problems- and the right AI solutions for each one- will be required for success with AI. The
If your marketing tries to appeal to everyone, it is doomed to fail. Instead, those in the know use market segmentation to divide their audience into specific groups for precise targeting. What Is Market Segmentation? Market segmentation is a way of separating your customers into different groups. Once you have done so, you can tailor your marketing campaigns in order to appeal to these different target segments. Customer segmentation can be based on various factors: Behavior (known as behavioral segmentation) Demographic information (age, income bracket, etc.) Psychographic information (social class, lifestyle, personality, etc.) Geographic (location) By breaking its customers down into various groups based on the above factors, a company can look at each audience segment individually, and identify their needs, wants and desires. Marketers can then use this information to design marketing campaigns and create ads that appeal directly to these audiences. Examples of Market Segmentation Lots of different people use a gym – despite the stereotypes, there is no one archetypal gym user. A gym will attract fitness fanatics, but even these will be split into those time-poor young professionals who favor HIIT (high intensity workouts) to get results quicker, treadmill addicts, weightlifters whose only goal is building
Artificial intelligence (AI) is heralding a revolution in how we interact with technology. Its capabilities have changed how we work, travel, play and live. But this is just the beginning. The next step is explainable AI (XAI), a form of AI whose actions are more easily understood by humans. So how does it work? Why do we need it? How will it forever change the way industries – especially in marketing – function? The Mystery of the Black Box: The Problem With Current AI No one would deny that artificial intelligence produces amazing results. Computers that can not only process vast amounts of data in seconds, but also learn, decide and act on their own have turned many industries on their heads – according to PricewaterhouseCoopers, the market worth of AI is around US$15 trillion. However, in its current form, AI does have one major weakness: explanation. Namely, it can’t explain its decisions and actions to humans. This is sometimes referred to as the “black box” in machine learning – for example, the calculations and decisions are carried out behind the scenes with no rationale given as to why the AI arrived at that decision. Why is this a problem?