How to Build a Credible Incentive Case Study: Measure the Program, Not Just the Winners
Start With What the Case Study Really Shows
Do Not Compare Winners With Average Performers and Call It Uplift
Separate Associated Revenue From Incremental Revenue
Use an Appropriate Comparison
Measure Profit, Not Just Sales
Treat Satisfaction as One Measure, Not the Final Result
Do Not Overlook Operational Value
Match the Headline to the Evidence
Explain What the Study Cannot Prove
Use Case Studies to Advance the Field
Click here to subscribe to the ESM weekly e-newsletter.
Case studies do not have to meet the standards of peer-reviewed academic research. They should, however, clearly distinguish between program facts, participant opinions, business associations, and results that can reasonably be attributed to the program. The following anonymized example has been recreated—with identifying details, figures, and program elements modified—to help incentive, recognition, loyalty, and rewards companies understand what to do and what to avoid when documenting client success.
In this week’s ESM, the article “Research Shows What, or Does It? How B2B Marketers Can Separate Credible Findings From Promotional Claims,” explains how artificial intelligence now makes it easy for clients, competitors, journalists, and prospects to examine whether a case study’s methodology and conclusions support its headline. The goal should not be to avoid scrutiny, but to produce case studies that benefit from it.
Start With What the Case Study Really Shows
Consider this case study of an automotive-aftermarket company that operates an incentive program for independent resellers. The program recognizes top-performing reseller organizations with communications, awards, and a group travel experience.A vendor-produced case study reports that approximately 80 qualifying reseller companies generated more than $90 million in annual revenue. It also says the top qualifiers produced six to eight times more revenue than the average reseller and that participants expressed high satisfaction with the experience.
A client executive praises the vendor for handling hotel negotiations, participant communications, travel arrangements, on-site logistics, and other administrative responsibilities. These facts can support several useful conclusions. The program serves commercially important reseller relationships. The qualifying accounts represent a substantial amount of revenue. Participants appear to value the experience. The client appreciates the vendor’s ability to manage a complicated program and reduce internal administrative burdens. Each of those findings can make a worthwhile case study.
The problem arises only when the case study goes further and characterizes the revenue produced by the winners as evidence that the incentive program multiplied sales, created loyalty, or generated a positive financial return.
Do Not Compare Winners With Average Performers and Call It Uplift
The most important lesson is to avoid using the success of award winners as proof of the program’s incremental impact. In this example, the resellers qualify for the travel experience partly because of their revenue performance. Comparing those winners with the average reseller will therefore almost always produce an impressive difference. The analysis is essentially reporting that resellers selected for superior performance outperform average resellers.
That may be true, but it does not demonstrate what caused their superior performance. Top reseller organizations might operate in larger markets, have more experienced salespeople, represent bigger accounts, enjoy longer-standing customer relationships, or have access to more favorable territories. Their performance could also reflect pricing, product availability, competitive conditions, acquisitions, promotions, or changes in demand for automotive parts and services. The incentive may have contributed to their results. The comparison simply does not isolate its contribution.
A difference between winners and average performers should therefore be described as a difference—not as “uplift.” Uplift implies an increase attributable to the intervention. Establishing uplift requires a baseline, an appropriate comparison, and a credible method for estimating what would have happened without the program.
Separate Associated Revenue From Incremental Revenue
Large revenue numbers naturally attract attention. They can also create confusion. If qualifying resellers generated more than $90 million, that figure demonstrates their importance to the automotive-aftermarket company. It does not necessarily represent revenue created by the incentive program. A better case study would answer several additional questions.
- How much revenue did those resellers generate before the qualification period?
- How much did their sales grow while comparable nonqualifying resellers remained flat or declined?
- Did the winners increase their purchases beyond their historical growth rates?
- Did they buy more profitable products, increase their share of wallet, or continue their higher performance after the qualification period ended?
Without this context, total revenue should be described as revenue associated with the qualifying accounts—not incremental revenue produced by the incentive. The distinction matters because management ultimately needs to understand what changed because of the investment.
Use an Appropriate Comparison
A credible case study does not necessarily require a randomized control group. It does need a comparison that helps readers assess whether the program contributed to the outcome. The automotive-aftermarket company might compare participating resellers with nonparticipants of similar size, tenure, territory potential, and previous sales performance. It could compare results before, during, and after the qualification period. It could also examine reseller organizations immediately above and below the qualification threshold.
None of these approaches is perfect. Each would, however, provide more useful information than comparing the very best performers with the entire reseller population. Where possible, the analysis also should account for factors unrelated to the program, including pricing changes, new product introductions, regional economic conditions, supply constraints, acquisitions, and unusually large customer orders. The purpose is not to create an academic experiment. It is to make a reasonable attempt to determine whether the program influenced behavior beyond what probably would have occurred anyway.
Measure Profit, Not Just Sales
Revenue alone cannot determine whether a program created financial value. A reseller may increase purchases of lower-margin products, advance orders from a future period, or concentrate purchases during qualification and reduce them afterward. A company also can generate significant incremental revenue without producing enough gross profit to cover the program’s costs.
A stronger case study would calculate the additional gross or contribution margin associated with the measured improvement. It would then subtract the relevant costs of travel, rewards, technology, communications, agency services, administration, and internal staff time. The calculation does not have to be presented as an exact return on investment if the available data do not support that precision. It can be presented as a reasonable estimate, with assumptions and limitations clearly disclosed. Transparency makes the finding more credible, not less.
Treat Satisfaction as One Measure, Not the Final Result
Participant satisfaction is a legitimate program outcome. A memorable experience can strengthen relationships, create goodwill, and reinforce the perceived value of doing business with a company. On the other hand, “high satisfaction” should be supported by more than an informal impression.
A useful case study would disclose how many people responded, what they were asked, the rating scale used, the response rate, and the actual results. Instead of simply reporting “strong satisfaction,” it might state that 86% of respondents rated the experience an eight or higher on a 10-point scale, based on responses from 71 of 95 attendees. Even then, satisfaction should not be presented as proof of loyalty or increased sales. The company would need separate measures of reseller retention, purchase frequency, referrals, share of wallet, product adoption, or future performance to support those conclusions.
Do Not Overlook Operational Value
The strongest finding in many incentive travel case studies may have little to do with sales causation. In the automotive-aftermarket example, the client executive values the vendor’s ability to manage contracts, travel, participant communications, and on-site execution. That is a real business benefit. The case study could make this evidence stronger by estimating the number of internal staff hours saved, administrative expenses avoided, participant issues resolved, contractual risks reduced, or other efficiencies achieved through outsourcing. Operational results may sound less exciting than claiming that a program multiplied revenue. They can be more credible and directly measurable.
Match the Headline to the Evidence
A good headline should attract attention without promising more than the case study can deliver. A headline such as “Automotive Reseller Incentive Multiplies Sales and Loyalty” would require evidence showing that the program caused incremental sales and improved loyalty. The available evidence does not establish either conclusion. A more accurate headline might be: Automotive-Aftermarket Incentive Program Recognizes Resellers Representing More Than $90 Million in Annual Revenue.
That headline still communicates the scale and importance of the program. It simply avoids attributing all of the participants’ performance to the incentive. The narrative can then explain what the client values, how the program operates, what participants reported, and which business outcomes remain to be measured.
Explain What the Study Cannot Prove
One of the most effective ways to build credibility is to acknowledge limitations. A case study might state that the analysis was observational, that participants were selected partly because of their previous performance, and that other market factors may have contributed to the results. It can explain that the findings establish an association but not definitive causation. This does not undermine the program. It demonstrates that the vendor understands measurement and respects the reader’s intelligence. It also provides a roadmap for improving the evaluation in future program cycles.
Use Case Studies to Advance the Field
The incentive, recognition, rewards, loyalty, and motivational events business has no shortage of positive client testimonials. What it needs is more transparent evidence connecting program design with measurable business outcomes. Every case study does not have to include a sophisticated statistical model. At minimum, it should define the program objective, explain who participated, disclose how success was measured, provide an appropriate baseline or comparison, account for program costs where possible, and use language proportional to the evidence.
A descriptive client story can still be valuable. A company should simply label it accurately. The automotive-aftermarket example would be credible as evidence that a vendor successfully manages an important reseller incentive program and that the qualifying accounts represent substantial revenue. It could also support claims about participant satisfaction and administrative value if those outcomes were documented properly.
It cannot, without additional analysis, demonstrate that the program caused the participants’ revenue, multiplied performance, or produced a positive return on investment. The purpose of this example is not to criticize the organizations involved in the original case on which the analysis was based. It is to help all suppliers, agencies, clients, and media organizations create case studies that are more useful, defensible, and persuasive.
In an era when AI can evaluate a marketing claim in seconds, intellectual honesty is no longer simply good research practice. It is a competitive advantage.
Enterprise Engagement Alliance Services
Celebrating our 17th year, the Enterprise Engagement Alliance helps organizations enhance performance through:1. Information and marketing opportunities on stakeholder management and total rewards:
- ESM Weekly on stakeholder management since 2009. Click here to subscribe; click here for media kit.
- RRN Weekly on total rewards since 1996. Click here to subscribe; click here for media kit.
- EEA YouTube channel on enterprise engagement, human capital, and total rewards since 2020
Management Academy to enhance future equity value for your organization.3. Books on implementation: Enterprise Engagement for CEOs and Enterprise Engagement: The Roadmap.
4. Advisory services and research: Strategic guidance, learning and certification on stakeholder management, measurement, metrics, and corporate sustainability reporting.
5. Permission-based targeted business development to identify and build relationships with the people most likely to buy.
Contact: Bruce Bolger at TheICEE.org; 914-591-7600, ext. 230.
