HARKing (Hypothesising After Results Are Known)

Norbert Kerr named the practice in a 1998 paper on hypotheses written after the results are known. The problem is arithmetic rather than moral. A p-value is the probability of data at least this extreme under the null, for a test chosen before the data were seen; if the hypothesis was picked because these data supported it, the selection step never enters the calculation and the nominal error rate stops describing anything. Outcome switching is the trial version: a secondary endpoint promoted to primary, or a disappointing pre-specified primary quietly dropped.

Prospective registration exists to make this visible. Journals following International Committee of Medical Journal Editors policy have required registration before enrolment since 2005, and registry records carry the pre-specified primary outcome with a timestamp. The COMPare project checked trials in leading general medical journals against their registered protocols and found outcomes departing from the pre-specified ones on a large scale, with unregistered outcomes added and registered ones missing.

A finding's status depends on when the hypothesis was fixed, not on how convincing it looks afterwards. A pre-specified primary endpoint result is confirmatory; anything else is hypothesis-generating and needs an independent test before it counts as established. Exploratory analysis is not the offence; presenting exploration as confirmation is.

The tells are specific. A hypothesis that fits the results unusually neatly and appears nowhere in the registration. A primary outcome in the abstract that does not match the registry. A trial that missed its endpoint but leads with a significant subgroup. In the peptide literature the pattern is small trials reporting whichever of several biomarkers happened to move, with no statement that any was chosen in advance.