I have a pretty green thumb, but I wasn’t born with it. It took lots of trial, error, and experimentation to find out which plants grow best in what kind of soil, how much light they need, and when things come into season. I had to test watering techniques, plant food, sun exposure levels, and placement in my flat—and I had to do it in an orderly, organised fashion.
The Importance of Experimenting
This might sound like a complicated way to grow a few plants, but to be successful and fill my house with healthy greenery, experimentation was the best solution.
This same is true in the experience business. As the famous John Wanamaker saying goes, “Half the money I spend on advertising is wasted; the trouble is I don’t know which half.” Marketers struggle to understand which of their advertising expenditures are driving business objectives, and which are missing the mark. Media mix decisions are often made based on correlated metrics, which are problematic because they don’t always indicate causality. Someone who clicked on your ad might have bought your product after arriving at your website, but what if they were already planning to buy?
That’s where a carefully designed experiment can come in handy. The purpose of an experiment is to determine a cause-and-effect relationship. In the world of marketing, experiments help advertisers isolate the metrics that matter, and determine what strategies are causing consumer to take incremental actions. Isolating these cause-and-effect relationships allows marketers to understand how advertising strategies are affecting their bottom line, positively or negatively. Experiments help marketers understand which half of their advertising spend is working and which is going to waste.
How do you create those experiments? How do you make sure you’re making decisions based on causality, instead of randomly correlated data points and key performance indicators? Let’s take a look.
Designing a Successful Marketing Experiment
There are three steps to executing a successful experiment that provides actionable information you can use to improve your campaigns.
- First, you’ll need to create two identical groups. One will be your control group, and the other will be your exposed group. In the exposed group you’ll introduce an independent variable, like advertising, to see what effect it has.
- Second, you’ll want to minimize contamination of the experiment. If there are any external factors that could influence the outcome of your experiment, try to control them, or, at the very least, keep them consistent for each group.
- Finally, you’ll need to evaluate your results. What were the statistically significant differences between your control group and the group exposed to a new variable? You want to be sure the change you see is due to the variable and not something that would have occurred naturally.
To better demonstrate a good experiment, let’s go back to my plant obsession. Say I wanted to determine the impact that adding plant food would have on growth. Here’s how I’d design my experiment:
- I’d purchase two identical plants (same age, size, soil type, ) from the same store. I would label one pot “control” and one pot “exposed.”
- For the two months, the control plant would only get water, while I’d give the exposed plant both water and plant food. (The plant food is the independent variable in this experiment.)
- While going about my experiment, I’d work to minimise contamination from outside influencers. I’d keep the plants in the same climate, expose them to same amount of light, and place them on the same window sill in my home. If one plant is in the bathroom and one is on the terrace, the experiment is flawed.
- After the time was up, I’d look at the difference in results between each plant. Did the exposed plant grow significantly more than the control one, or was it smaller? Was it greener? Or even, did it die? Keep in mind that the difference should be significant enough to indicate causality. If the exposed plant is only slightly taller than the control, it could easily be a natural occurrence. If, however, it’s two or three times the size, I could safely assume the plant food was the reason.
I could repeat the same experiment for other variables, such as sun exposure, water levels, or even the brand of plant food. After enough experimenting, I’d be able to zero in on a sure-fire method for growing strong, healthy plants.
Your Ticket to Higher ROI
These ideas are easily translated into the marketing world—and should be. And while this might seem basic, it’s important because experiments can help you avoid costly mistakes, point to better ways of doing things, and point the way toward innovative thinking.
But many marketers find experimentation difficult. With efforts spread across silos and platforms, it may be hard to discern where experiments should be used, let alone how to design them. That’s where omnichannel software, which helps measure metrics from all platforms and channels, as well as measurement consultants can lead the way.
A well-designed experiment shows you the cause and effect of your efforts. It sheds light on which metrics you should hone in on, monitor, and work to optimize, and can isolate the strategies leading customers to take incremental actions that deliver real return on investment. With these results, you can tweak your strategy to maximize those returns, making the absolute best use of your ad spend.