Skip to content
UX Atlas
Cognitive biasEstablished researchIntermediate

Confirmation Bias

People seek and weight evidence that supports what they already believe, which quietly corrupts research findings.

Definition

The tendency to search for, interpret and recall information in a way that confirms existing beliefs. In design research it appears in how studies are designed, how questions are asked, and which findings get reported.

Established research. Originates in published research and has been replicated. The effect is real. Its size in your product still needs measuring.

On this page
  1. Definition
  2. Where it enters design research
  3. In practice
  4. Sources
  5. Continue from here

Where it enters design research

  • Leading questions: 'How much easier was the new version?' presupposes the answer.
  • Recruiting only current users, who have already selected in.
  • Stopping a study once the expected pattern appears.
  • Reporting the three supportive quotes and omitting the seven that were not.
  • Running an A/B test until it reaches significance, then stopping, which inflates false positives.
  • Treating a stakeholder's hypothesis as the research question.

Do

  • Write the research question and analysis plan before collecting data.
  • Ask about past behaviour rather than future intention.
  • Recruit people who rejected the product, not only those who adopted it.
  • State the sample size and stopping rule for experiments in advance.
  • Report disconfirming findings explicitly in the summary, not only in an appendix.

Do not

  • Ask whether participants like something.
  • Let the person with the hypothesis moderate the session alone.
  • Peek at experiment results and stop when they look good.

In practice

Leading moderation

Usability testing

A moderator invested in the design asks 'was that straightforward?' after a struggle. The participant, being polite, agrees. The session has produced a data point that is worse than none.

Peeking at experiments

A/B testing

Checking results daily and stopping at the first significant reading inflates the false positive rate substantially. Fixing the sample size in advance, or using a sequential testing method designed for it, is the correction.

Sources

Where a source establishes something narrower than the popular reading of it, the note says so.

  • Confirmation bias: a ubiquitous phenomenon in many guises

    R. S. Nickerson, Review of General Psychology, 1998

Continue from here

Each link says what the connection is, so you can tell a principle from an alternative from a thing people mix this up with.

Applied through

Where this shows up as a concrete interface decision.

Often used with

These usually appear in the same screen or the same decision.

Short definition

The one-paragraph version, for when that is all you need.

Related concept

Connected closely enough to change how you apply this.