Why most job training programs fail — and what the ones that work have in common.

In 1993, Mathematica Policy Research concluded the largest randomized evaluation of job training ever conducted in the United States.

The program was the Job Training Partnership Act — the federal government's primary workforce investment vehicle since 1982. Researchers randomly assigned more than 20,000 eligible applicants across 16 program sites to either receive JTPA services or to be barred from them for 18 months.

The result for adult men: no statistically significant effect on earnings at the 30-month follow-up. For out-of-school male youth: a negative short-term effect. For adult women and female youth: modest positive effects of around $1,500 per year.

This was not a methodologically weak study. It was a rigorous, pre-registered, multi-site randomized trial covering a national program. The null result for adult men was real.

The puzzle

Twenty years later, a research team from Harvard and Social Policy Research Associates published results from a lottery-based RCT of Year Up — a six-month sectoral training and internship program for young adults aged 18 to 24 from low-income backgrounds.

The effect on quarterly earnings at three years: +$1,000 per quarter. A 30% increase over the control group, sustained through at least five years of follow-up.

Both studies were rigorous. Both measured job training. They produced opposite results.

The natural conclusion — that one program is better designed than the other — is correct, but it's incomplete. The more useful question is: what is the structural difference between a training program that produces null results and one that produces 30% earnings gains?

The mechanism problem

Most job training programs are built around a supply-side theory of unemployment: people are unemployed because they lack skills, so providing skills will produce employment.

This theory is partially true. Skills matter. But it misses the other half of the labor market: the demand side. Employers hire based on their specific needs, their internal screening processes, and their cultural norms — none of which respond to generic skill certificates.

JTPA provided classroom training in occupational skills — welding, data entry, medical terminology — that certified participants had acquired knowledge. What it did not provide was a connection between that knowledge and the specific employers willing to hire someone with it.

Year Up builds its program from the demand side. Employers tell Year Up what skills they need. Year Up designs training around those skills. Year Up places participants in real internships with those employers, where participants work alongside full-time staff and demonstrate performance in an actual work environment. Employers hire from the intern pool at high rates.

The training produces the same outcome: a person with skills. The difference is whether there is a prepared buyer for those skills on the other end.

Why this pattern recurs

The JTPA null result is not an outlier. A substantial body of job training evaluations — many of them rigorous quasi-experiments — shows similarly weak effects for generic training programs.

What predicts positive results, consistently across contexts, is the combination of three elements that Year Up represents:

**Employer partnership.** The program is designed around documented employer demand, not generic skill categories. Employers are involved in curriculum design, not just as occasional guest speakers.

**Real work experience.** Participants work in actual job environments — not simulated ones — during training. This produces references, demonstrates performance under real conditions, and creates direct relationships with potential employers.

**Specific local labor markets.** The program is calibrated to the industries and hiring processes of a specific city. A healthcare training program in Pittsburgh is built around Pittsburgh health systems, not generic healthcare employers.

Generic programs fail because they optimize for the supply side of a matching problem. Effective programs treat placement as the outcome and work backward from employer demand to curriculum design.

What this means for municipalities

Many cities and counties fund job training programs — often through federal Workforce Innovation and Opportunity Act dollars — without rigorous evaluation of whether participants are actually placed in jobs or whether earnings gains persist.

The JTPA/Year Up comparison offers a diagnostic framework. For any existing or proposed training program, ask:

**Is the curriculum driven by documented employer demand?** If the answer is "we trained people in skills that seemed valuable," that is a supply-side answer. If the answer is "we have letters of intent from 40 employers who commit to interviewing our graduates," that is a demand-side answer.

**What happens at the moment of placement?** Programs that hand participants a certificate and wish them luck are ending at the wrong place. Programs that facilitate direct introductions, manage employer relationships, and track placement rates are treating placement as an output, not an afterthought.

**What is the comparison group doing?** JTPA's null for adult men was partly explained by the comparison group's behavior during the 18-month exclusion: many sought and found training elsewhere. A program that produces no differential effect may be replicating what participants would have done anyway. The relevant question is always relative to the counterfactual.

The harder lesson

The job training literature is a useful test case for a broader claim: rigorous evaluation changes the content of programs, not just their measurement.

Before the JTPA evaluation, federal workforce policy assumed that generic training was effective and that null results were an artifact of poor program administration. The evaluation showed that the null was real and that the mechanism — skills training disconnected from employer demand — was the problem. The response was the Workforce Investment Act of 1998, which placed greater emphasis on employer partnerships and performance accountability.

That response was imperfect. But it was a response to evidence, not ideology. The program changed because a rigorous experiment produced a finding that the existing theory couldn't explain away.

That is exactly what civic experimentation is supposed to produce.

Next issue: Collaborative care for mental health — how integrating treatment into primary care changed what's possible for depression outcomes.