The Evil Twin of “Leading the Witness” that plagues ecommerce post-purchase surveys

Between the two variations of this methodological no-no, one is easy to spot and as benign as a typo on a ‘thank you’ note.

Here’s an example I encountered last week in a Roast I did for a brand in the non-alcoholic beverage industry. The survey was sent to customers two weeks after their initial purchase.

Can you spot what’s wrong?

The phrase “How much did you like….” assumes each respondent likes the flavor, which is not necessarily true.

I’d ask, “What do you think of the flavor?” and provide these response options.

- Love it
- It’s good
- Indifferent about the flavor
- It’s not good
- Hate it

The second variation of “leading the witness” is subtler, yet potentially more consequential, like a typo in a prescription dosage.

Here’s an example from the same Roast.

Can you spot the problem?

While this question doesn’t steer you towards a specific response, the statement "when choosing a hydration drink-mix powder" assumes that assessing the category and making purchases based on benefits is something that you do. This, like the first question, is not necessarily true.

One alternative is, “When you purchased our product, which of the following benefits mattered to you?”

This is better because it centers around a real event – the purchase.

However, the purchase would have happened about two weeks prior to customers receiving the survey, and memory isn’t that reliable.

I’d go with: “When you consume this brand, which of the following benefits and attributes are most important to you?”

This works because it asks customers about something they actually do — consume the brand’s products – and opinions they actually have – the appreciated benefits/attributes.

Plus, it should provide the insights my client needs to refine their value proposition and identify their target shopper.

If you’re trying to figure out if a specific product attribute is delivering value to a segment of shoppers, ask shoppers which attributes provide the most value – not which attributes they consider in a hypothetical purchasing situation.

If you do, you’ll be better equipped to prescribe the right dosage.