Skip to content
UX Atlas
Plate RES·001 · Research methodGrid H2 · Rev. 2026-08-14

Usability Testing

Watching people attempt real tasks, which is the most direct evidence available about whether a design works.

Research methodEstablished researchFoundational

Definition

Usability testing observes participants completing tasks with a product or prototype. The output is behaviour: what people did, where they hesitated, and what they failed at. Opinions gathered alongside are secondary.

Established research. Originates in published research and has been replicated. The effect is real. Its size in your product still needs measuring.

On this plate
  1. 1Definition
  2. 2Running a session
  3. 3What to record
  4. 4In practice
  5. 5Commonly confused with
  6. 6Local chart
  7. 7Sources
  8. 8Continue from here

Reach for this when

  • You have a design or prototype and want to know whether it works.
  • Deciding between two approaches.
  • Before committing engineering effort.

Common mistakes

  • Leading moderation.
  • Testing only with existing enthusiastic users.
  • Reporting opinions over behaviour.
  • Assuming five users covers every segment.

Running a session

  1. 1

    Write tasks, not instructions

    'Find a hotel in Lisbon for next weekend under 200 euros' is a task. 'Click the search button' is a script.

  2. 2

    Say what you are testing

    Tell the participant you are testing the design, not them, and that any difficulty is useful information.

  3. 3

    Stay quiet

    Silence produces data. Answering their questions destroys it. When asked, redirect: 'What would you expect to happen?'

  4. 4

    Record behaviour, not just opinion

    Task success, time, errors, hesitations and the recovery attempt when something goes wrong.

  5. 5

    Debrief afterwards

    Ask opinion questions at the end, so they do not contaminate the tasks.

Do

  • Test early, with rough prototypes, when changes are still cheap.
  • Test with people from outside the team and outside the domain.
  • Report task success rate alongside quotes.
  • Include participants who use assistive technology.

Do not

  • Ask whether participants like it.
  • Lead with questions that presuppose an answer.
  • Rescue a struggling participant before the struggle has told you something.
  • Treat satisfaction ratings as evidence of usability.

What to record

Behaviour first. Satisfaction ratings correlate weakly with task success, so they belong at the end and never on their own.

MeasureWhat it tells youHow to capture it
Task successWhether the design works at allBinary per task, with partial success defined in advance
Time on taskWhere effort concentratesOnly comparable between designs, never against a target
Errors and recoveriesWhere the model breaksCount the error, then note whether they got out of it unaided
AssistsWhether the session is still a testEvery time the moderator had to intervene
Single Ease QuestionPerceived difficulty, immediately after the taskOne seven-point rating, asked before the memory fades

In practice

  1. i.

    Unmoderated testing at scale

    Remote research

    Unmoderated platforms trade the ability to probe for larger samples and faster turnaround. They suit validating a specific known question and are poor for open exploration.

  2. ii.

    Five per segment, not five in total

    Sample size

    A product serving clinicians and administrators is two populations with different vocabulary, goals and tolerance for error. Five of each finds far more than ten drawn from whoever was easiest to recruit.

  3. iii.

    The prototype that was too polished

    Fidelity

    High-fidelity prototypes attract comments on colour and wording, and suppress the structural objections that are still cheap to act on. Test structure on something that looks unfinished, on purpose.

Commonly confused with

Close neighbours that get substituted for each other. The difference is stated first, then the condition that picks one over the other.

vs.

A/B Testing

Research method

Splitting live traffic between variants to measure which performs better.

Key difference

Usability testing tells you why something fails, with a handful of people watched closely. A/B testing tells you which version wins, with thousands of people and no explanation of the result.

Use Usability Testing when

  • You need the cause behind a number you already have.
  • The design is not built or shipped yet.
  • Traffic is too thin to measure anything reliably.

Use A/B Testing when

  • Two defensible options remain and the argument is stuck.
  • There is enough traffic to reach significance in reasonable time.
  • The outcome is a metric you already trust.

Learn about A/B Testing

vs.

Heuristic Evaluation

Research method

A structured expert review of an interface against an agreed set of principles.

Key difference

Heuristic evaluation is expert judgement against known principles, done without users. Usability testing is observation of real people. They surface different problems, and the cheap one should not be treated as a substitute.

Use Usability Testing when

  • The problem is disputed and evidence has to settle it.
  • The domain is specialised enough that expert intuition is unreliable.
  • You need to know what people actually do, not what should be hard.

Use Heuristic Evaluation when

  • There are no users available or no budget for sessions.
  • You want the obvious problems cleared before testing.
  • The output needs to be a prioritised list of known issues.

Learn about Heuristic Evaluation

vs.

User Interviews

Research method

A guided conversation about how someone works and what they are trying to achieve.

Key difference

A usability test asks people to do something and watches. An interview asks people about their work, context and goals. One finds interface problems, the other finds out what to build.

Use Usability Testing when

  • There is something to use, even a paper sketch.
  • The question is whether people can complete a task.
  • You want behaviour, not recollection.

Use User Interviews when

  • You do not yet know what to design.
  • The question is context, workaround or motivation.
  • There is no artefact to put in front of anyone.

Learn about User Interviews

Local chart

RES·001 · Region IV · Grid H2

This entry and the entries it is stated to connect with. Each line is labelled with what the connection means.

Curse of KnowledgeBIAS·008 · EXPLAINSConfirmation BiasBIAS·009 · PRINCIPLES BEHIND THISHalo EffectBIAS·013 · PRINCIPLES BEHIND THISAesthetic-Usability EffectLAW·011 · PRINCIPLES BEHIND THISPeak-End RuleLAW·007 · PRINCIPLES BEHIND THISMental ModelsMOD·001 · PRINCIPLES BEHIND THISGulfs of Execution and EvaluationPRIN·010 · PRINCIPLES BEHIND THISMere Exposure EffectPSY·009 · PRINCIPLES BEHIND THISResearch PlanTPL·001 · OFTEN USED WITHUsability Test ScriptTPL·007 · OFTEN USED WITHLookbackTOOL·022 · OFTEN USED WITHMazeTOOL·018 · OFTEN USED WITH

Sources

Where a source establishes something narrower than the popular reading of it, the note says so.

  1. [1]

    A mathematical model of the finding of usability problems

    J. Nielsen and T. K. Landauer, INTERCHI, 1993

  2. [2]

    Beyond the five-user assumption

    L. Faulkner, Behavior Research Methods, Instruments, and Computers, 2003

  3. [3]

    Rocket Surgery Made Easy

    S. Krug, New Riders, 2009

    The case for small, frequent, do-it-yourself sessions over occasional large studies.

  4. [4]

    Quantifying the User Experience

    J. Sauro and J. R. Lewis, Morgan Kaufmann, 2nd edition, 2016

    Where the task-level measures and the Single Ease Question come from, with the statistics for small samples.

Continue from here

Each link says what the connection is, so you can tell a principle from an alternative from a thing people mix this up with.

Applies when you are designing

A CheckoutA DashboardA Data TableA FormA Mobile AppA Pricing PageA Search ExperienceAn AI ProductAn Ecommerce ExperienceAn Onboarding FlowAuthentication

Explains

The mechanism underneath these entries.

  • Curse of KnowledgeBIAS·008 · Cognitive bias

    Testing exists because you cannot un-know your own interface.

Principles behind this

The reasoning this solution is an application of.

Often used with

These usually appear in the same screen or the same decision.

Alternative approach

A different answer to the same problem, with a different cost.

  • Guerrilla TestingRES·016 · Research method

    Cheaper and faster, with a sample you did not choose.

Short definition

The one-paragraph version, for when that is all you need.

Related concept

Connected closely enough to change how you apply this.