A Worked Example of Testing Whether a Message Actually Lands
Testing whether a new value proposition resonates is different from a standard ad creative test, because the thing being tested isn't a headline or an image, it's the core claim about why a customer should care. Here's a worked example of running this kind of test and reading the results without fooling yourself about what a short-term lift actually proves.
The shape of this example is common enough that it's worth walking through in detail: a company confident its new angle is stronger, an early metric that seems to confirm it, and a longer trace through the funnel that tells a more complicated story.
The setup: two messages, one product
Say a company currently leads with a message about saving time, and wants to test whether a message about reducing risk resonates more with its audience. Both messages run as the primary headline across the same landing page and ad set, split evenly, with everything else held constant.
What the first-week numbers showed, and why that wasn't the whole answer
The risk-focused message pulled a higher click-through rate in the first week. On its own, that result would suggest switching the primary message entirely. But click-through rate measures whether a message gets attention, not whether it attracts the right kind of prospect, and a message that gets more clicks isn't automatically the one that produces better customers.
A team stopping the analysis here would have a clean, confident-sounding result to report, and it would also be the wrong conclusion, which is exactly what makes this stage of a resonance test the most dangerous one to stop at.
What the attribution data showed further downstream
Tracing both groups through to demo requests and, eventually, closed deals told a different story: the time-saving message's clicks converted to demo requests at a lower rate but those demo requests closed at a meaningfully higher rate than the risk-focused message's leads did. The risk-focused message was attracting more curious clicks, but a larger share of them weren't a strong fit once a real sales conversation happened. None of this was visible from the first-week engagement numbers alone, which is exactly why the trace mattered.
What this changes about how to read the test
- A message that wins on click-through rate isn't automatically the better message for the business, since it may just be attracting more curious rather than more qualified attention
- Follow any messaging test through to a downstream business outcome, not just the immediate engagement metric, before declaring a winner
- If you can't wait for a full sales cycle to complete, use an intermediate signal like demo-to-opportunity conversion rate as a closer proxy for eventual close rate than click-through rate alone
- Be willing to keep a message that performs worse on a surface metric if it's demonstrably attracting a better downstream customer
What the company actually did with this result
Rather than fully switching to the risk-focused message, the company kept the time-saving message as primary but tested working a risk-reduction point into the supporting copy further down the page, aiming to capture some of the attention-grabbing benefit of the risk message without losing the quality of prospect the time-saving message was attracting up top. That combined version is what eventually replaced the original.
The general lesson for any resonance test
A message resonance test needs to be read on the metric that actually matters to the business, which is rarely the first metric that becomes available. Click-through rate and engagement metrics are useful early signals worth watching, but treating them as the final answer, rather than as one early data point in a longer trace through to revenue, is the single most common way these tests mislead a team into the wrong decision.
Build the habit of asking, for any messaging test result, what it would look like if you traced it one step further downstream than you currently plan to. That single question catches most of the false-early-winner problem before it leads to an actual decision.
For example, suppose a team is deciding whether to switch its homepage headline after one week of data. A useful decision rule is to write down, before the test starts, which downstream metric will decide the winner and how long the team will wait for it. If the early metric and the downstream metric disagree, the downstream metric wins, and the early metric becomes a question to investigate rather than a result to announce. This also guards against choosing the metric after seeing the data, which lets a team justify whichever message it already preferred. Put the rule in the test brief where everyone who reads the results will see it.
What Good Looks Like
The standard is reading a message resonance test through to a downstream business outcome, not settling for whichever version wins on the first available engagement metric.
Building The Capability (5-Stage Skill Ladder)
How to Get Started
Frequently Asked Questions
How long should a message resonance test run before we trust the results?
Long enough to trace results through to whatever downstream outcome actually matters, ideally at least through to demo requests or opportunities, not just the initial click-through metric. This often means running longer than a typical creative test.
Is a higher click-through rate ever a reliable signal on its own?
It can be a useful early indicator worth watching, but it shouldn't be the final word on a messaging test by itself, since it measures attention rather than fit. Follow it through to a business outcome before declaring a winner.
What if we don't have enough volume to trace a message test all the way to closed deals?
Use the closest available proxy, like demo-to-opportunity conversion rate, that reflects genuine fit rather than raw attention. It's an imperfect substitute, but it's still more predictive of eventual close rate than click-through rate alone.
About the numbers
This guide doesn't quote a sourced benchmark. Figures in it are estimates or general guidance, so check them against your own numbers.
Related Guides
A Creative Testing Checklist for Teams That Test Too Slowly
A checklist for running faster, more reliable ad creative tests, with the specific mistakes that either slow testing down or make results untrustworthy.
First-Touch, Last-Touch, or Multi-Touch: Picking the Right Model
A comparison of first-touch, last-touch, and multi-touch attribution models, with the tradeoffs that actually matter for a small marketing team.
Is Your Conversion Rate Actually Bad, or Just Different?
How to use published conversion rate benchmarks without drawing the wrong conclusion from a number that was never measuring your situation.
Content Syndication Without Cannibalizing Your Own Rankings
How to republish your best content on other sites for reach without letting the syndicated copy outrank or duplicate your own original pages.
What It Actually Takes to Rank for High-Intent Searches
Answers to the questions marketing leads actually ask about ranking for high-intent, buying-stage search terms instead of chasing broad traffic keywords.
Run a Marketing Attribution Health Check in an Afternoon
A practical checklist for auditing whether your multi-touch attribution and marketing mix data can actually be trusted before you act on it.