Surveys
Good for measuring how widespread something is, and poor for discovering what that something is.
Definition
A survey collects structured responses from a sample. It quantifies attitudes and self-reported behaviour, and it can only measure things you already know to ask about.
Established research. Originates in published research and has been replicated. The effect is real. Its size in your product still needs measuring.
Design rules
- One idea per question. Double-barrelled questions produce uninterpretable answers.
- Avoid leading wording and loaded adjectives.
- Offer a neutral midpoint and a 'not applicable' option where they are legitimate answers.
- Randomise option order where order effects are plausible.
- Keep it short. Completion falls sharply past a few minutes, and the dropouts are not random.
- Pilot with five people before sending. It always finds an ambiguous question.
In practice
Interview first, then survey
Mixed methodsInterviews tell you what to ask about. Surveys tell you how common it is. Running the survey first usually measures the team's assumptions rather than the users' reality.
Continue from here
Each link says what the connection is, so you can tell a principle from an alternative from a thing people mix this up with.
Principles behind this
The reasoning this solution is an application of.
- Confirmation BiasCognitive bias
- Survivorship BiasCognitive bias
Alternative approach
A different answer to the same problem, with a different cost.
- User InterviewsResearch method
Surveys scale what you already know how to ask. Interviews find out what to ask.
Short definition
The one-paragraph version, for when that is all you need.
- Quantitative ResearchTerm
- Sample SizeTerm
Related concept
Connected closely enough to change how you apply this.
- A/B TestingResearch method
- Product AnalyticsResearch method
- Research SynthesisResearch method