Statistical vs Clinical Significance

The two judgements use different inputs. Statistical significance is a function of effect size, variability and sample size, asking whether the data are compatible with no difference at all. Clinical significance asks whether the magnitude crosses a threshold worth the treatment's burden, cost and risks, benchmarked against a minimal clinically important difference. Nothing in the p-value contains that benchmark: increase the sample size enough and any non-zero difference becomes statistically significant.

Concrete thresholds make the gap visible. On an eleven-point pain rating scale, differences of about two points, or roughly a 30 percent reduction, are the values usually cited as noticeable to a patient, so a highly significant mean difference of 0.4 points is a real effect nobody would feel. In obesity trials, 5 percent body weight loss is the conventional threshold at which cardiometabolic risk factors begin to shift measurably.

The operational question is therefore not whether the confidence interval excludes zero but where it sits relative to the threshold that matters. An interval lying entirely above the minimal clinically important difference supports a meaningful benefit. One excluding zero but lying entirely below that threshold is a reliable demonstration of an unimportant effect, and calling it a significant improvement is technically true and practically misleading.

The commonest abuse in peptide marketing runs the inference the other way, taking a statistically significant change in an in-vitro measure, collagen expression in cultured fibroblasts, as though it were clinical benefit, when no threshold for importance on that outcome has ever been defined. The mirror error, dismissing a modest but real and durable effect as small, matters when the outcome is a hard one.