The Oregon health lottery.

In early 2008, Oregon held a lottery.

The state wanted to expand its Medicaid program to cover more low-income uninsured adults. It had funding for roughly 10,000 new enrollees. It had a waiting list of more than 90,000 people. It couldn't expand eligibility to everyone. It didn't want to prioritize by need — any such system would be contested, slow, and politically fraught. So it did what felt fair: it drew names at random.

Winners were invited to apply for OHP Standard, Oregon's Medicaid program for low-income adults without dependents. Losers stayed uninsured.

For the people involved, the lottery was a fact of life, processed and moved past. For a group of economists at MIT, Harvard, and the National Bureau of Economic Research, it was something rare enough to have no good name in their field: a randomized controlled trial of health insurance, conducted at scale, by a government that hadn't meant to run an experiment at all.

What economists saw

The researchers who recognized what Oregon had created were Amy Finkelstein at MIT, Katherine Baicker at Harvard, and their colleagues. They contacted the Oregon Health Authority in 2008, within months of the lottery drawing, and proposed a partnership to study the results.

What they were asking for was legitimately difficult to obtain by any other means.

Health insurance coverage had never been subjected to a large-scale randomized evaluation in the United States. The reason was obvious: you couldn't randomly deny people insurance. You couldn't randomly assign coverage and a control condition in any ethically defensible way. The RAND Health Insurance Experiment of the 1970s — the previous landmark in this field — had studied different levels of cost-sharing, not the difference between having insurance and having nothing.

Oregon's lottery had done what the research community couldn't. It had randomly assigned health insurance coverage to a large population of low-income adults, with a clean control group of equally eligible people who simply hadn't won the draw.

The Oregon Health Insurance Study enrolled approximately 12,000 lottery winners and a matched group of lottery non-winners. It tracked healthcare utilization, financial outcomes, health behaviors, and clinical health measures over two years. The first major papers were published in 2011 and 2012. The landmark two-year clinical results appeared in the New England Journal of Medicine in 2013.

What the first papers showed

The 2011 results — covering the first year — were largely uncontroversial.

Medicaid coverage increased healthcare utilization substantially. Lottery winners were more likely to have a primary care physician, more likely to have had a preventive screening, more likely to have visited the emergency room, and more likely to have been hospitalized. They were also significantly less likely to have unpaid medical bills sent to collections and significantly less likely to have borrowed money or skipped other bills because of medical costs.

The financial protection finding was large and consistent. Medicaid coverage reduced the probability of catastrophic out-of-pocket medical expenditures by 80%. It reduced the probability of having any out-of-pocket medical expenses at all. For low-income adults without savings, the distinction between "sick and insured" and "sick and uninsured" was not abstract — it was the difference between a manageable event and a financial crisis.

Mental health outcomes were also clearly positive. Lottery winners reported significantly lower rates of depression and significantly higher rates of reported happiness and wellbeing. The effect size was large by the standards of mental health interventions — larger, in fact, than many treatments studied in clinical trials.

The 2013 results

The two-year clinical results were the source of the controversy.

Finkelstein, Baicker, and their colleagues measured a set of specific physical health outcomes: blood pressure, cholesterol, blood sugar, and body mass index. These were outcomes where theory predicted that access to primary care and preventive medicine would produce measurable improvement over two years.

The results on these measures were null. Medicaid coverage did not produce statistically significant improvements in blood pressure, cholesterol, or blood sugar. The effect on diagnosis rates for diabetes and hypertension was positive — lottery winners were more likely to know they had these conditions — but treatment and control of those conditions did not significantly improve.

The paper was careful about what the null finding meant. The study was designed to detect moderate effect sizes. There was a specific statistical power calculation. The confidence intervals excluded large effects but were consistent with small ones. Two years was a short window for chronic disease outcomes that develop over decades. The diabetic subgroup — where blood sugar effects would most plausibly appear — was smaller than ideal.

None of this mattered to the people who had been waiting for ammunition.

The weaponization of a null

Within days of the 2013 NEJM paper's publication, it had been cited by politicians, commentators, and advocates on both sides of the healthcare debate — and badly misrepresented by most of them.

Critics of Medicaid expansion said the study proved that Medicaid "doesn't improve health." This was a misreading so obvious it bordered on dishonest. The study found no statistically significant effect on four specific clinical measures over two years. It found clear positive effects on a dozen other outcomes. Claiming that null results on a subset of measures in a two-year window disproved the health value of insurance required ignoring most of what the study showed.

Defenders of Medicaid expansion sometimes went the other direction, dismissing the clinical null findings as irrelevant or methodologically suspect. This was also wrong. The null findings were real and pre-specified. They told us something: that two years of Medicaid coverage did not produce large improvements in blood pressure, cholesterol, or blood sugar in this population. That's a finding. It belongs in the evidence base.

The honest reading was what the authors wrote in their discussion section: "Our results within the two-year window of our study show the effects of Medicaid on a range of outcomes, not all of which are what many people hoped or expected." Insurance coverage clearly improved financial security, mental health, and access to care. It did not — in two years, in this population — produce large measurable improvements in chronic disease control.

Both things are true. They have different implications for different policy questions.

What the study was actually designed to answer

Understanding what the Oregon study showed requires understanding what it was designed to measure.

The study was powered to detect moderate effects on the primary outcomes. It was not powered to detect the long-term effects of chronic disease prevention — effects that accrue over ten or twenty years, not two. It could not tell us what would happen to a 45-year-old diabetic's cardiovascular risk if they had continuous Medicaid coverage for a decade. It wasn't designed to answer that question.

It could tell us — and did — what happens in the first two years when a low-income adult gains insurance coverage. The answer is: they use more healthcare, they have fewer financial crises, they feel better, they are less depressed, and some specific clinical markers don't move much in the short run.

If your policy question is "does Medicaid reduce the financial devastation of illness for low-income adults?" the Oregon study answered it clearly and in the affirmative. If your policy question is "does two years of Medicaid coverage cure chronic disease in adults who've been uninsured for years?" the answer is unsurprisingly no — and that's not what anyone should have expected.

The lesson for civic experimenters

The Oregon Health Insurance Study is a master class in what happens when rigorous evidence enters a politicized environment.

The study was well-designed, carefully conducted, and honestly reported. Its authors were appropriately cautious about what they could and couldn't conclude. None of that prevented it from being weaponized by people who wanted a single sentence they could quote in a debate.

This is the environment in which all policy-relevant evidence lives. It doesn't mean you shouldn't run the experiment. It means you should pre-specify your outcomes, be precise about what you're measuring and why, and write your limitations section before the critics do.

For civic experimenters working at the local level — library programs, benefits enrollment, permitting processes — the Oregon lesson is also more mundane and more useful.

Study design determines what you can conclude. A two-year window with modest sample sizes will miss effects that accrue slowly or express in rare outcomes. If you want to know whether a simplified form reduces incomplete applications next month, you can probably answer that question with 300 participants and a six-week study. If you want to know whether it changes long-term benefit receipt rates, you need a different design — more time, more people, or a different primary outcome.

Know what your study can answer before you run it. Pre-specify that question. Report the answer honestly, including the null findings. And then say, clearly, what the study can't tell you.

The Oregon researchers did all of this. The fact that people ignored it is not their failure. It is a feature of the environment. You will work in that environment too.

Amy Finkelstein and Katherine Baicker's full data and code from the Oregon Health Insurance Study are publicly available at the NBER. The study's methods have been taught in health economics and policy evaluation courses for over a decade.