What NYC voting rights and 5 trillion gallons of water have in common
In the fall of 2024, I got a text message that changed how I ask questions.
It was a survey about a New York City ballot proposal—something about letting unaffiliated voters participate in party primaries. I clicked through, shared my opinion, and expected that to be the end of it.
But then something unexpected happened.
It showed me possible consequences of the proposal—some appealing, some uncomfortable—and then asked for my opinion again.
I don't remember if my answer changed.
But I remember this: that survey didn’t just measure my opinion. It helped me form one.
A lot of consumer research runs into the same dead end: we ask people questions about things they rarely think about—like laundry detergent or banking apps—and then act disappointed when the answers don't tell us anything. We want more considered responses, but anything beyond neutral, survey-safe language feels like leading the witness.
The ballot survey showed me an alternative.
I found myself agreeing with some consequences and hesitating at others. By the time I answered the question again, I felt ready to actually answer it.
If I had to offer one guideline, it would be this: before you measure an opinion, give people something to form one against.
You can do that without telling them what to think.
A few months after I got that text, I got a chance to apply this method to a survey I was working for a nonprofit based in the southern US.
The topic was water—how residents think about it, where they think it comes from, and what concerns them about the future. Most people think about water about as often as laundry detergent or their banking app.
Instead of jumping straight to our core question—What concerns do you have about your water?—we shared basic facts about household water use, even visualized a state’s annual consumption (5 trillion gallons) in terms of how many professional football stadiums it would fill (6,500).
We explained possible solutions and their costs.
Nothing persuasive.
Just context.
The responses were specific and thoughtful. Nearly everyone shared something insightful.
The payoff wasn't just better data. Those responses helped clarify the real communication challenge: not convincing people that water matters, but helping them understand what's actually at stake.
After the water project wrapped, I went back and looked up that ballot survey technique. Turns out it has a name—deliberative polling—and political pollsters have been using it since the 1980s to see how explaining the issues and their tradeoffs changes people's opinions.
I've been borrowing from this approach ever since.
Each time I draft a survey, I’m usually not focused on whether each question is biased or methodologically correct.
I’m listening for something simpler: would a real person know how to answer these questions?
If not, the fix is rarely a full rewrite. It’s usually a sentence or two that gives people something to react to—an explainer, a comparison, a consequence.
My Survey Roasts are designed to help you spot where your survey needs that friction—and how to add it, so you’re not stuck with 500 responses that don’t tell you anything.
Book one here.
Send me your survey draft, and for $145, I’ll record a 15-minute Loom video with clear, copy-and-paste edits and practical suggestions to improve your survey.
Cheers,
Sam