Survey Design Best Practices for UX Research

Write UX surveys that generate reliable, actionable data. Learn question types, ordering effects, scale design, and bias prevention grounded in product

Why Survey Design Is a Product Research Decision

A badly structured survey doesn’t just produce weak data. It produces confidently wrong data, which is worse. So before you write a single question, treat the survey itself as a product decision.

UX surveys differ from generic market-research surveys in one important respect: the goal is actionable product insight, not statistically representative population data. You are trying to understand what a defined group of users thinks, does, or needs — and then change something because of it. That framing shifts everything, from question wording to sample size.

Surveys also sit within a wider mixed-method toolkit. They earn their place when you need to quantify patterns across a large group, compare segments, or track attitudes over time. They are the wrong tool when you need to understand why something is happening, when the topic is sensitive and nuanced, or when your mental model of the user is still forming. For those situations, an interview or a diary study UX research will serve you better.

The most common mistake in UX survey design is starting with questions. Start with the research question instead. Write it out in one sentence: “We need to understand X so we can decide Y.” Every survey question that doesn’t answer that sentence is a candidate for removal. Once the intent is clear, structure follows. If you haven’t yet mapped your survey to a broader plan, our research plan template is a useful starting point.

Choosing the Right Question Types

The question type you choose determines what you can do with the data. Choose badly here and no amount of statistical sophistication will rescue the analysis later.

Closed vs. open-ended questions

Closed questions — where respondents pick from predefined options — are faster to complete, easier to analyse, and comparable across respondents. Open-ended questions generate richer, more unexpected insight but cost real analysis effort and tank completion rates when overused. A well-designed UX survey leans heavily on closed questions, with one or two open-text fields placed deliberately.

Single-select, multi-select, and ranking

Single-select (radio buttons) forces a clear choice and produces clean data. Multi-select (checkboxes) is appropriate when more than one answer can genuinely be true, but it inflates the apparent importance of every selected option, because you can’t tell what respondents considered and rejected. Ranking questions reveal relative priority — useful for feature prioritisation — but become cognitively demanding beyond five or six items.

Likert scales

Likert scales present a statement and ask respondents to indicate agreement or frequency. The choice between 5-point and 7-point scales has a practical answer: 5-point scales are sufficient for most UX work and less cognitively demanding; 7-point scales add resolution when you genuinely need to distinguish fine gradations of opinion. Label wording matters more than point count. “Strongly agree / Agree / Neither / Disagree / Strongly disagree” and “Always / Often / Sometimes / Rarely / Never” are not interchangeable — pick the anchor type that matches what you are actually measuring.

Semantic differential scales

Where Likert scales measure agreement with a statement, semantic differential scales position attitude between two opposing adjectives: simple — complex, trustworthy — untrustworthy. They suit measuring product perception and brand attributes, where a single statement would feel artificial.

Standardised measures: NPS, CSAT, SUS

Net Promoter Score, Customer Satisfaction Score, and the System Usability Scale each have established benchmarks and known methodological limitations. NPS is a single-question relationship metric, not a diagnostic tool. CSAT captures transactional satisfaction. SUS is a validated 10-item scale for perceived usability. Use them when benchmarking or tracking over time — not as substitutes for understanding.

Matrix questions

Matrix questions — several statements sharing one response scale — shorten a survey visually but raise cognitive load. Respondents are more likely to straight-line (select the same column throughout) in matrices. Break them out into individual questions when the statements are conceptually distinct, and keep matrices short when you do use them.

Open-text fields

Place open-text fields at the end of a section or survey, not mid-flow. Phrase prompts specifically: “What, if anything, would make this feature more useful?” outperforms “Any other comments?” Expect 30–50% of respondents to leave open fields blank, and build analysis time into your project plan accordingly.

Writing Unbiased, Clear Questions

Question wording is where most survey bias enters. Spotting and removing it before launch is less work than explaining contaminated data to stakeholders afterwards.

Leading questions

A leading question signals the answer the researcher expects. “How helpful did you find the new onboarding flow?” assumes it was helpful. A neutral rewrite: “How would you rate the onboarding flow?” with a balanced scale. The pattern is simple: remove evaluative language from the question stem and move it into the response options, where it belongs.

Double-barrelled questions

“How satisfied are you with the speed and reliability of the product?” is two questions. A respondent who finds the product fast but unreliable has no valid answer. Split every “and” in a question stem and check whether both halves are genuinely separate constructs. If they are, write two questions.

Assumed behaviour

“How often do you use Feature X?” assumes the respondent uses Feature X at all. Precede behavioural frequency questions with a filter: “Do you use Feature X? Yes / No / I’m not sure.” Route non-users past the follow-up.

Acquiescence bias

Respondents have a slight tendency to agree with statements regardless of content. Counter this by including both positively and negatively framed items across the survey, and by using response scales that are balanced around a genuine neutral midpoint.

Social desirability bias

For sensitive topics — spending habits, security practices, how often someone actually reads documentation — respondents tend to answer in ways they think reflect well on them. A neutral permission statement before the question helps: “People use products in many different ways. Thinking about your own experience over the past month…” This gives respondents implicit permission to answer honestly.

Vocabulary and reading level

Match your language to the participant, not to your internal vocabulary. In practice, this means reading each question aloud and asking whether a user without product knowledge would parse it immediately. If it requires re-reading, simplify. Aim for sentences under 20 words and avoid passive constructions.

We saw this clearly in research on a financial product’s onboarding experience: early survey drafts leaned on internal terminology that participants in pilot interviews didn’t recognise. Terms the product team used daily had no currency with users. Replacing them with the wording participants had used in earlier qualitative sessions measurably improved response quality and cut the share of “not sure” selections.

Ordering Effects and Survey Structure

The sequence of questions in a survey is not neutral. Respondents use earlier questions as context for interpreting later ones, and the position of an option in a list shapes how often it is chosen.

Primacy and recency effects

In a list of response options, items at the top (primacy) and bottom (recency) are selected more often than those in the middle. For questions where order shouldn’t influence the answer — feature lists, reasons for a decision — randomise the options. Most survey platforms support this natively.

Question-order bias

Specific or sensitive questions placed before general ones contaminate the general question. Asking “How satisfied are you with the checkout process?” before “How satisfied are you with the overall experience?” will inflate overall satisfaction scores. Move from broad to specific: establish general context before drilling into particular features or behaviours.

Funnel structure

Open with broad, easy questions that orient respondents to the topic. Move progressively toward specific behavioural or attitudinal items. This mirrors natural conversation and warms respondents up before asking for considered judgements.

Demographics at the end

Place demographic questions at the end of the survey, not the beginning. Opening with age, role, or income can prime respondents to answer subsequent questions through those identities rather than their own experience. The exception is when demographics function as screeners — in which case they belong before the survey proper, in the screener flow.

Progress indicators and section breaks

Visible progress indicators improve completion rates for longer surveys. Section headers that flag what’s coming — “Next: your experience with payments” — reduce abandonment by setting expectations. If your survey has genuine thematic sections, make that structure visible.

Branching and screening logic

Irrelevant questions damage completion rates and data quality in equal measure. Use skip logic to route respondents past questions that don’t apply to them. This requires mapping every path before building the survey. A diary study UX research approach, where participants respond to prompts over time, can sometimes replace heavily branched surveys when path complexity becomes unmanageable.

Survey length

Completion rates drop sharply with length. As a working rule, unsolicited surveys (cold email, panel) should stay under 10 minutes; in-product intercepts with motivated users can stretch to 15. Cut ruthlessly by asking, for each question: “What decision would change if we removed this?”

Recruiting the Right Participants

A well-designed survey administered to the wrong sample produces confident nonsense. Recruitment deserves as much attention as question design.

Sampling strategy

Convenience sampling — whoever is easiest to reach — is rarely defensible for product decisions. Define your target population first: who specifically needs to answer this survey for the findings to be actionable? Then build recruitment around that definition.

Sample size

For attitudinal surveys (satisfaction, perception, preference), a minimum of 100 completed responses allows basic segmentation. For behavioural surveys where you need to detect differences between groups, calculate the required sample based on the effect size you care about. Smaller qualitative-style surveys with 30–50 respondents can surface patterns but cannot support statistical comparisons. More detail on participant recruitment is covered in our guide to how to recruit participants for user research.

Screener questions

Screeners should identify qualifying respondents without signalling which answers qualify them. “Do you use financial planning software at least once a month?” is less leading than “Do you regularly use financial planning software to manage your budget?”, which implies that regular, purposeful use is the desired answer.

Recruitment channels

In-product intercepts reach active users but over-represent those who engage frequently. CRM outreach offers more control over who you contact but requires a clean, segmented list. Research panels give access to large numbers quickly but introduce panel-effect bias — frequent survey respondents who have learned to answer efficiently rather than honestly. For B2B products, CRM outreach to defined user tiers is usually the most reliable approach.

Incentive design

Incentives increase response rates but can attract respondents motivated primarily by the reward. Match incentive value to survey length and target audience: over-incentivising a short survey attracts low-quality responses. In B2B contexts, a donation to a relevant charity sometimes works better than cash for recruiting senior respondents who are sceptical of panel-style research.

Piloting, Analysing, and Acting on Survey Data

Launching a survey without piloting it is a gamble that rarely pays off. Build testing time into the project schedule before data collection begins.

Cognitive interviews

Ask three to five people from your target population to complete the survey while thinking aloud. You are listening for hesitations, reinterpretations, and moments where they answer a different question from the one you asked. A single round of cognitive interviews will surface more bias than a team review session.

Data quality checks

Once data starts arriving, check for straight-lining (respondents selecting the same answer for every item in a scale), speeders (completions well below the median time), and open-text gibberish. Most platforms allow you to flag and exclude these responses. Set your exclusion criteria before launch, not after.

Quantitative analysis

Start with central tendency and distribution for each closed question. Means can obscure bimodal distributions — always look at the full frequency table. Cross-tabulate by relevant segments (role, tenure, plan type) to identify where patterns differ. A finding that holds across all segments is more robust than one that emerges only in a single slice.

Combining quantitative and qualitative

Likert responses tell you the shape of opinion; open-text responses tell you the texture. Bring the two together using affinity mapping — group open-text responses by theme and cross-reference theme frequency against scale scores. The process is covered in detail in our post on affinity mapping in qualitative research.

Communicating findings

Survey data is frequently over-interpreted in stakeholder presentations. Be explicit about what the data can and cannot support: a survey of 80 users in one segment cannot describe all users. Frame findings as directional evidence that warrants further investigation or a decision, not as proof. Our guide to communicating research findings to stakeholders covers how to do this without losing impact.

Quick-Reference Checklist Before You Launch

Run through this list before your survey goes live. If any item is incomplete, address it before the first responses arrive.

Research question defined: one sentence describing what you need to know and why
Question types matched to intent: closed questions for quantifiable data, open-text for texture, scales chosen to match what is being measured
Bias patterns reviewed: leading language removed, double-barrelled questions split, assumed behaviour filtered, social desirability addressed
Order tested: broad-to-specific structure in place, demographics at the end, response options randomised where appropriate
Pilot completed: cognitive interview with 3–5 target participants, feedback incorporated
Sample and recruitment confirmed: target population defined, channel selected, screener questions tested, incentive appropriate
Analysis plan documented: how each question will be analysed, who is responsible, how findings will be presented

If you finish this checklist and the survey still feels unwieldy, ask whether a survey is the right method at all. Surveys work best for quantifying known patterns. If you are still discovering what questions to ask, a qualitative approach is more appropriate. A full overview of when to use each approach sits in our UX research methods guide.

Frequently Asked Questions

How many questions should a UX survey have?

There is no universal answer, but completion rates drop sharply beyond 10–12 questions for unsolicited surveys and 15–20 for in-product intercepts with motivated users. The practical discipline is to scope the survey to one core research question and remove anything that doesn’t directly answer it. If you find yourself keeping questions because they are “interesting”, they are candidates for a separate study.

When should I use a Likert scale vs. a semantic differential scale?

Likert scales measure agreement or frequency toward a single statement — they suit satisfaction and usability measurement. Semantic differential scales measure attitude between two bipolar adjectives, such as simple–complex or trustworthy–untrustworthy, and are better suited to perception and brand-attribute mapping. Use Likert when you want to know how strongly someone agrees with a specific claim. Use semantic differential when you want to understand where on a spectrum a product sits in the user’s mind.

How do I reduce survey bias in UX research?

Address it at the question level first: remove leading language, split double-barrelled questions, and filter assumed behaviour with screening items. Then address it at the structural level: randomise response options, place sensitive questions after neutral ones, and ensure scales are balanced. Run a cognitive interview pilot before launch — it is the single most reliable way to catch bias before it contaminates your data.

Can UX surveys replace user interviews?

No. Surveys efficiently capture what and how many; interviews capture why. The two methods are complementary. A surve

About Glasgow Research — Glasgow Research helps B2B SaaS teams turn customer and market research into product decisions. Work with us.