Which outreach message increases attendance at free library programs?
Columbus Metropolitan Library · Columbus, Ohio, USA · Published November 8, 2024
Designed
August 12, 2024
Launched
September 3, 2024
Closed
October 22, 2024
Pre-registration
OSF Pre-Registration #osf.io/demo-colb-2024
License
CC-BY 4.0
Summary
We randomly assigned 1,231 Columbus Metropolitan Library card-holders to receive one of three outreach emails promoting a free fall programming series: a standard informational message (control), a social proof message citing neighborhood registrations, or a personalized message referencing prior attendance. The social proof variant increased registration by 8.7 percentage points over control (27.1% vs. 18.4%, p = 0.003). The personalized variant increased registration by 5.2 pp (23.6%, p = 0.041). Neither variant significantly affected attendance rates among registrants or opt-out rates. Two complaints were received in the social proof arm (0.49%), none in control. We recommend the social proof message for future programming outreach, with monitoring of complaint rates at larger scale.
Background and question
Columbus Metropolitan Library runs approximately 80 free community programs per quarter across its 23 branches. Attendance at these programs averaged 34% of registered capacity in 2023 — well below the 60% target the programming team had set. Staff hypothesized that the standard outreach email, which listed program logistics but provided no social context, was not sufficiently motivating registration.
The behavioral science literature offers two candidate mechanisms. Social proof messaging — informing recipients how many of their neighbors have already taken an action — has been shown to increase compliance in tax, energy, and voting contexts. Personalization — using the recipient's name and referencing prior behavior — reduces the psychological distance between message and action.
Pre-specified question: Does including social proof or personalization in a program outreach email increase registration rates among eligible card-holders, without meaningfully increasing opt-outs or complaints?
Design
Assignment method
Individual-level randomization via stratified random assignment (stratified by branch, age group, and prior program attendance)
Eligible population
Card-holders with a valid email address who had visited a branch or checked out materials in the prior 12 months and had not previously registered for this program series
Sample size
1,231 card-holders (412 control, 408 social proof, 411 personalized); pre-specified power calculation required n ≥ 340 per arm to detect a 6 pp difference at 80% power
Blinding
Participants not informed they were in an experiment. Library staff who sent emails were unaware of arm assignment at the individual level.
Message arms
Standard email: program name, date, time, location, and registration link. Current practice.
"127 families in your neighborhood have already signed up." Same logistics as control.
Addressed by first name; referenced the patron's prior program attendance. Same logistics as control.
Results
All four outcomes were pre-specified. The primary outcome was registration rate. Analysis used intention-to-treat; all randomized participants were included regardless of whether they opened the email.
| Outcome | Control (A) | Social proof (B) | Δ vs. A | p-value | Personalized (C) | Δ vs. A | p-value |
|---|---|---|---|---|---|---|---|
| Registration rate (primary)PRE-SPECIFIED | 18.4% | 27.1% | +8.7 pp | 0.003 | 23.6% | +5.2 pp | 0.041 |
| Attendance rate (among registrants)PRE-SPECIFIED | 71.2% | 68.9% | −2.3 pp | 0.44 | 74.1% | +2.9 pp | 0.31 |
| Opt-outs / unsubscribesPRE-SPECIFIED | 1.2% | 1.8% | +0.6 pp | 0.29 | 0.9% | −0.3 pp | 0.51 |
| Complaints receivedPRE-SPECIFIED | 0 | 2 | — | — | 1 | — | — |
p-values from two-sided Fisher's exact test for binary outcomes. Bonferroni correction applied across two treatment comparisons; significance threshold p < 0.025. Confidence intervals available in the replication dataset.
Finding
Social proof messaging increased registration by 8.7 percentage points (47% relative increase) with no significant effect on attendance rates among registrants and no meaningful increase in opt-outs. The effect on registration did not persist to attendance — social proof drew more people to register, but did not close the registration-to-attendance gap. This suggests the remaining attendance problem is a different barrier (friction at the event level, not motivation at the sign-up level) requiring a different intervention.
Limitations
Replication notes
The full analysis dataset (anonymized), message text for all three arms, randomization code, and analysis script are available under CC-BY 4.0 at the registry entry below. A library system wishing to replicate this experiment can use the materials as-is or adapt the social proof figure to local data.
Minimum viable replication: 300 eligible card-holders per arm, accurate local registration count for the social proof figure, and administrative registration data retrievable by arm assignment. Contact us if you want to discuss the design before launching.
Decision taken by Columbus Metropolitan Library
Columbus Metropolitan Library is adopting the social proof message as the default for programming outreach effective January 2025, replacing the current informational template. The programming team will monitor complaint rates for two quarters. A follow-on experiment to address the registration-to-attendance gap (via day-before reminder and simplified check-in) is planned for spring 2025.