I paid 160 people to take my survey. Then I asked for my money back.

Last week I recruited 160 American adults to an online survey. I paid them $0.50 to answer three trivia questions.

• Which planet is closest to the Sun?
• Which sport uses a shuttlecock instead of a ball?
• Which author wrote the Harry Potter series?

After they answered, I gave them a choice: collect the $0.50 I agreed to pay them, or answer 12 more trivia questions for no additional pay.

Surveys are infamous for low effort and high drop-off rates. Here, 43% stuck around and answered the remaining 12 questions.

After these respondents answered the extra questions, I gave them another choice. For $0.50, they could see the questions they answered correctly and how they ranked relative to other respondents. If a respondent paid me, they'd finish the survey exactly where they started — having received $0.50 from me and sent $0.50 back, netting zero.

Zero respondents paid.

I admit my offer was partly a stunt. But I fielded this survey to make a serious point.

The fact that zero people paid is a more useful finding than if I'd simply asked whether they'd pay $0.50 to see their results, in which case I'd almost certainly have gotten a meaningful percentage saying yes.

In market research, you typically pay people to tell you what they want, then build it. But I've learned more about what people want by trying to earn their time than by paying for it.

In surveys I design for clients, I often make the most important question optional, and say so explicitly. Then I design the whole survey around one goal: getting as many people as possible to answer it. I get better data than if I'd just required it.

For example: last summer I surveyed 1,100 Texans about water. Before asking about their concerns, I showed them how much they use — visualizing the state's annual consumption in football stadiums. It felt more like a pamphlet than a survey. Nearly everyone voluntarily shared a concern at the end.

Survey design has always been owned by methodology and statistics. But when you start asking why would someone want to answer this? instead of how do I field this correctly? — you write questions that are intuitive, easy to answer, and more honest. You get better data.

Cheers,
Sam

PS: The survey is still live. Take it below and see how your score compares to the 69 respondents who answered all 15 questions.