A survey question before and after: what I changed and why
I want to walk you through edits I made to one question in a Survey Roast, with notes on what I changed and why.
But first, a story.
When I was in high school, I came home from a weekend at my friend's cabin in northern Minnesota. "You smell disgusting," my mom said. My sweatshirt reeked of campfire smoke.
She then accused me of smoking weed and using the fire to mask the smell.
That confused me for two reasons. First, we hadn’t been smoking weed. Second, I hadn't noticed the smell, even though I'd been sitting next to that fire for two days.
Most people who book a Roast worry about whether their survey questions are accurate or insightful. They want to know if they’re asking things the right way and whether they’ll get useful data.
But the thing they should be worried about is simpler: jargon. Spend enough time somewhere (business, academia) and it clings to you like campfire smoke.
The edit I’m going to show you is just me removing jargon the author didn’t notice – but an outsider would.
Here’s the original question (which I’ve altered for privacy).
Read it out loud. You’ll hear the problem immediately.
"How often do you currently consume sleep aids (pills, gummies, drops, etc.)?"
Nobody says they "consume" supplements. And the question stacks two time cues ("often" and "currently") without needing either.
Now read this.
"Do you take sleep aids?"
That’s it. That’s the fix.
This version sounds like something you'd actually say.
That’s what a good question should sound like — not neutral, not deep. Just normal.
- "Currently consume" is survey voice — nobody talks like this
- Stacking two time cues ("how often" and "currently") adds friction without adding meaning
- Response options are hard to scan — they mix vague labels with numbers, forcing the eye to jump back and forth
- "Do you take" replaces "how often do you currently consume" — plain language, one time cue
- Replaced frequency options with Yes/No for easier scanning — frequency gets its own follow-up question
I made two other changes worth mentioning.
First, I replaced the frequency scale with “Yes / No.” Deciding whether you take sleep aids and how often you take them are two separate tasks, and a good question only asks respondents to do one thing at a time. Frequency gets its own followup.
Second, I dropped the parenthetical examples — clutter, mostly, since most people already know what a sleep aid is. Plus the survey asks about specific types later.
On their own, edits to a single question won’t transform your data. But they compound. And the easier your survey is to read, the more deeply people will engage with it.
Your respondents weren't sitting at the fire. They notice the smoke.
Most survey authors never find out — they just get bad data and wonder why.
I’m not here to falsely accuse you of drug use. But for $145 I'll send you a 15-minute Loom and a Google Doc with copy-and-paste edits and specific suggestions to improve your data quality.
Check out an example here.
Cheers,
Sam