The null result problem.

Roughly 60% of randomized controlled trials in public policy show no statistically significant effect. In published literature, that share drops to about 30%. The gap is not random.

Null results are systematically less likely to be written up, submitted, or accepted. Researchers have less incentive to write them up — the career reward for a null result in most academic fields is small. Journals have historically been less likely to accept them — a finding of "no effect" is harder to frame as a contribution. Program implementers often prefer that null results not be published — a program that found no effect is a program that may lose funding.

The consequence

For a field trying to accumulate knowledge about what works, this is a structural problem. The consequence is not just wasted resources — it is systematic overconfidence. Every practitioner who reads only published findings is reading a biased sample. They see the programs that worked. They don't see the equally rigorous programs that didn't.

This matters most at the moment of scaling. A city official who reads the literature on a program type sees the successful trials. The failed trials — if they exist — may never have been published. The official concludes the evidence base is strong. They allocate budget. They run the program. They may or may not measure outcomes.

What a registry corrects

The Experiment Society's registry is built around a simple structural requirement: null and negative results are required for submission, not optional. A program that ran a randomized pilot and found no effect is as valuable as one that found a large effect — sometimes more so, because it narrows the search space for future experiments.

This sounds obvious. It is not common practice.

Of the 48 experiments currently in the registry, 14 showed null or negative results. In each case, those results were documented in full — effect estimates, confidence intervals, the decision the implementing institution made afterward. Seven of those programs were discontinued. Four were redesigned. Three continued unchanged, which is itself a finding worth documenting.

What practitioners can do

The most important thing a practitioner can do with a null result is write it up anyway. Not a formal academic paper — a structured report using the registry schema is sufficient. What was the question, what was the intervention, what was the effect, what did you decide to do next?

This creates cumulative knowledge. The practitioner who runs a failed mentoring program in 2025, documents it, and submits it to the registry makes the next practitioner's decision less likely to repeat the same mistake.

The alternative — the null result goes into a filing cabinet, the program continues or quietly ends, and nobody outside the institution learns anything — is the default. The default is what we are trying to change.

Next issue: Hot spots policing and the gap between what was studied and what gets implemented.