Skip to content
UX Atlas
Plate BIAS·009 · Cognitive biasGrid B1 · Rev. 2026-08-14

Confirmation Bias

People seek and weight evidence that supports what they already believe, which quietly corrupts research findings.

Cognitive biasEstablished researchIntermediate

Definition

The tendency to search for, interpret and recall information in a way that confirms existing beliefs. In design research it appears in how studies are designed, how questions are asked, and which findings get reported.

Established research. Originates in published research and has been replicated. The effect is real. Its size in your product still needs measuring.

On this plate
  1. 1Definition
  2. 2Where it enters design research
  3. 3Designing a study that can fail
  4. 4In practice
  5. 5Local chart
  6. 6Sources
  7. 7Continue from here

Reach for this when

  • Planning any research or experiment.
  • Moderating sessions.
  • Reviewing findings before they become decisions.

Common mistakes

  • Leading questions.
  • Optional stopping in experiments.
  • Selective reporting of quotes.
  • Running a second study because the first one gave the wrong answer, and calling it triangulation.

Where it enters design research

  • Leading questions: 'How much easier was the new version?' presupposes the answer.
  • Recruiting only current users, who have already selected in.
  • Stopping a study once the expected pattern appears.
  • Reporting the three supportive quotes and omitting the seven that were not.
  • Running an A/B test until it reaches significance, then stopping, which inflates false positives.
  • Treating a stakeholder's hypothesis as the research question.

Do

  • Write the research question and analysis plan before collecting data.
  • Ask about past behaviour rather than future intention.
  • Recruit people who rejected the product, not only those who adopted it.
  • State the sample size and stopping rule for experiments in advance.
  • Report disconfirming findings explicitly in the summary, not only in an appendix.

Do not

  • Ask whether participants like something.
  • Let the person with the hypothesis moderate the session alone.
  • Peek at experiment results and stop when they look good.

Designing a study that can fail

  1. 1

    Write down what would change your mind

    Before you run it. If nothing would, the study is a demonstration rather than a test.

  2. 2

    Have someone else write the tasks

    Preferably someone who does not want the design to succeed.

  3. 3

    Decide the analysis in advance

    Which measures, which comparisons, what counts as success. Deciding afterwards guarantees you find something.

  4. 4

    Record the disconfirming observations first

    They are the ones that get lost in synthesis, because nobody is looking for them.

  5. 5

    Report what did not support the conclusion

    A finding with its counterevidence stated is far harder to dismiss than one without.

In practice

  1. i.

    Leading moderation

    Usability testing

    A moderator invested in the design asks 'was that straightforward?' after a struggle. The participant, being polite, agrees. The session has produced a data point that is worse than none.

  2. ii.

    Peeking at experiments

    A/B testing

    Checking results daily and stopping at the first significant reading inflates the false positive rate substantially. Fixing the sample size in advance, or using a sequential testing method designed for it, is the correction.

  3. iii.

    The quote that carried the report

    Selective evidence

    One participant articulated exactly what the team hoped, and that quote appeared in three decks. Six others had said something more ambivalent, and none of that reached a slide.

Local chart

BIAS·009 · Region I · Grid B1

This entry and the entries it is stated to connect with. Each line is labelled with what the connection means.

AI Design CritiqueAI·009 · APPLIED THROUGHResearch Notes to InsightsAI·003 · APPLIED THROUGHA/B TestingRES·009 · APPLIED THROUGHAffinity MappingRES·020 · APPLIED THROUGHResearch SynthesisRES·014 · APPLIED THROUGHSurveysRES·004 · APPLIED THROUGHUsability TestingRES·001 · APPLIED THROUGHUser InterviewsRES·003 · APPLIED THROUGHDesign CritiqueFLOW·001 · APPLIED THROUGHDiscoveryFLOW·004 · APPLIED THROUGHMeasuring UXFLOW·011 · APPLIED THROUGHDesign CritiqueTPL·012 · OFTEN USED WITH

Sources

Where a source establishes something narrower than the popular reading of it, the note says so.

  1. [1]

    Confirmation bias: a ubiquitous phenomenon in many guises

    R. S. Nickerson, Review of General Psychology, 1998

Continue from here

Each link says what the connection is, so you can tell a principle from an alternative from a thing people mix this up with.

Applied through

Where this shows up as a concrete interface decision.

Often used with

These usually appear in the same screen or the same decision.

Short definition

The one-paragraph version, for when that is all you need.

Related concept

Connected closely enough to change how you apply this.