Statistical Significance
The difference is probably real, not luck.
Definition
A result unlikely to have arisen by chance under the null hypothesis, at a stated threshold.
Craft convention. True because the industry converged on it. Breaking it costs familiarity, not correctness.
On this plate
In plain language
The difference is probably real, not luck.
Example
A 0.3 percent lift can be statistically significant with enough traffic and still not be worth shipping.
Local chart
TERM·025 · Region X · Grid D6
This entry and the entries it is stated to connect with. Each line is labelled with what the connection means.
Continue from here
Each link says what the connection is, so you can tell a principle from an alternative from a thing people mix this up with.
Deeper reading
The full entry behind this definition.
- Confirmation BiasBIAS·009 · Cognitive bias
- A/B TestingRES·009 · Research method
Related concept
Connected closely enough to change how you apply this.
- A/B TestTERM·100 · Term
- Effect SizeTERM·026 · Term