Skip to content
UX Atlas
Cognitive biasEstablished researchIntermediate

Survivorship Bias

Analysing only the people who made it through hides everything the failures would have told you.

Definition

A selection bias in which conclusions are drawn from a sample that has already passed a filter, ignoring the cases that did not. The classic illustration is Abraham Wald's wartime analysis of aircraft damage, where the returning planes showed where damage was survivable.

Established research. Originates in published research and has been replicated. The effect is real. Its size in your product still needs measuring.

On this page
  1. Definition
  2. How it shows up in product data
  3. In practice
  4. Sources
  5. Continue from here

How it shows up in product data

  • Surveying only active users, who by definition did not churn.
  • Analytics on completed funnels, which cannot see people who never entered.
  • NPS from a subscription base that has already excluded everyone who left.
  • Usability testing with recruited enthusiasts.
  • Reading five-star reviews to understand why people buy.
  • Post-launch analysis that only includes customers who upgraded.

Do

  • Interview people who churned, and people who evaluated and chose a competitor.
  • Instrument the top of the funnel, including entries that never progress.
  • Segment analysis by cohort so the survivors are visible as a subset.
  • Ask what the missing population would have said, explicitly, in every readout.

Do not

  • Generalise from your most engaged users.
  • Treat completion rates as a measure of usability without knowing who dropped out.
  • Assume an unused feature is unwanted rather than undiscovered.

In practice

Churn interviews

Subscription products

The most valuable research population is the one that left. They are harder to recruit and they answer questions your active users cannot.

Funnel analysis blind spots

Analytics

A checkout with a ninety percent completion rate looks healthy until you measure how many people reached checkout at all, and why the rest did not.

Sources

Where a source establishes something narrower than the popular reading of it, the note says so.

  • A method of estimating plane vulnerability based on damage of survivors

    A. Wald, Statistical Research Group, Columbia University, 1943

Continue from here

Each link says what the connection is, so you can tell a principle from an alternative from a thing people mix this up with.

Applied through

Where this shows up as a concrete interface decision.

Often used with

These usually appear in the same screen or the same decision.

Short definition

The one-paragraph version, for when that is all you need.

Related concept

Connected closely enough to change how you apply this.