Statistics and risk
Multiple comparisons
The problem that testing many outcomes or groups increases the chance of a false-positive result.
What it means
Definition
Multiple comparisons. Multiple comparisons matter when studies run many statistical tests. Without appropriate adjustment or caution, one apparently significant result may arise by chance.
What it can tell you
It can help clarify what sort of evidence or claim is being discussed.
What it cannot tell you
It cannot, by itself, prove that the headline is accurate or clinically important.
Headline trap
Why this matters in headlines
If a study tests 40 outcomes, a single positive finding may not be as persuasive as it sounds.