Gulfs of Execution and Evaluation
Two gaps break interfaces: not knowing how to do the thing, and not knowing whether it worked.
Definition
From Norman's seven stages of action. The gulf of execution is the gap between a user's goal and the actions the system makes available. The gulf of evaluation is the gap between the system's state and the user's ability to interpret it.
Expert heuristic. A review criterion developed by practitioners. Useful and widely taught, but evaluated by judgement rather than measurement.
On this plate
Reach for this when
- Analysing usability test failures.
- Deciding whether a problem needs better cues or better feedback.
- Structuring a heuristic evaluation.
- Analysing session notes and sorting complaints into things to change.
Common mistakes
- Treating every failure as a labelling problem.
- Applying feedback fixes to discoverability problems.
- Using the model as vocabulary without using it as diagnosis.
A diagnostic, not a rule
The value of this model is in diagnosis. When a user fails, ask which gulf they fell into. If they did not know what to do, it is execution, and the fix is signifiers, labels, discoverability and mapping. If they did the right thing and could not tell whether it worked, it is evaluation, and the fix is feedback, status and clearer state representation.
Teams routinely apply the wrong fix. Adding a tour to an evaluation problem, or better confirmation messages to an execution problem, leaves the failure in place.
Norman's seven stages, in short
- 1
Goal
The user forms an intention about the world, not about the interface.
- 2
Plan
They decide on a sequence of actions.
- 3
Specify
They translate the plan into specific operations the system offers.
- 4
Perform
They execute the operations. Stages two to four are the gulf of execution.
- 5
Perceive
They notice the system's response.
- 6
Interpret
They work out what the response means.
- 7
Compare
They judge the result against the goal. Stages five to seven are the gulf of evaluation.
Which gulf a complaint belongs to
The diagnostic is useful because the two gulfs need opposite fixes, and the symptoms sound similar in a session.
| What the participant says | Gulf | What to change |
|---|---|---|
| I do not know how to do that here | Execution | Signifiers, labels, placement, available actions |
| I pressed it. Did it work? | Evaluation | Feedback, state, confirmation |
| I thought it would do something else | Both | The system image: naming, preview, and what the result shows |
| I cannot tell if this is the right screen | Evaluation | Orientation, titles, breadcrumbs, current state |
In practice
- i.
Silent save
EditorsUsers pressing save repeatedly is an evaluation failure. They performed the correct action and the system gave them nothing to interpret.
- ii.
Invisible bulk selection
ListsUsers deleting items one at a time when a bulk action exists is an execution failure. Better confirmation messages would not touch it.
- iii.
Two problems that sounded the same
Using it in a sessionBoth participants said the save did not work. One could not find it, and one had pressed it and seen nothing change. The same sentence, opposite fixes, and the gulf question is what separates them.
Local chart
PRIN·010 · Region I · Grid B1
This entry and the entries it is stated to connect with. Each line is labelled with what the connection means.
Sources
Where a source establishes something narrower than the popular reading of it, the note says so.
- [1]
The Design of Everyday Things
D. A. Norman, Basic Books, 1988, revised 2013
- [2]
User Centered System Design
D. A. Norman and S. W. Draper, Lawrence Erlbaum, 1986
Where the gulfs were first set out.
Continue from here
Each link says what the connection is, so you can tell a principle from an alternative from a thing people mix this up with.
Applied through
Where this shows up as a concrete interface decision.
- Heuristic EvaluationRES·002 · Research method
- Usability TestingRES·001 · Research method
Short definition
The one-paragraph version, for when that is all you need.
- Think-Aloud ProtocolTERM·027 · Term
Related concept
Connected closely enough to change how you apply this.
- Visibility of System StatusHEUR·001 · Heuristic
- Affordances and SignifiersPRIN·007 · Principle
- DiscoverabilityPRIN·005 · Principle
- FeedbackPRIN·004 · Principle