In June 2021, The U.S. Social Security Administration held a full-day virtual event dedicated to discussing lessons learned from its past tests of policies and programs to improve beneficiary work outcomes and directions for future research.

At the event, disability and social policy researchers presented papers on lessons learned about which policies, programs, and other operational decisions could provide effective supports for disability beneficiaries who want to work. Following each presentation, academic and policy discussants provided additional context for the research.

Please see the event agenda and presentation summaries below to learn more.

Agenda

Time

Session

9:45 – 10:00 a.m. EDT

Virtual Event Opens

10:00 – 10:10 a.m. EDT

Opening Remarks
Jeffrey Hemmeter, U.S. Social Security Administration
Austin Nichols, Abt Associates

10:10 – 10:45 a.m. EDT

Design of Demonstrations
Panel Chair: Laura Peck, Abt Associates

Presenters: Burt Barnow, The George Washington University
David Greenberg, University of Maryland, Baltimore County

Discussants: Jesse Rothstein, University of California, Berkeley
Jack Smalligan, The Urban Institute

10:50 – 11:25 a.m. EDT

Use of Demonstrations
Panel Chair: Laura Peck, Abt Associates

Presenters: Austin Nichols, Abt Associates
Robert Weathers, U.S. Social Security Administration

Discussants: Jonah Gelbach, University of California, Berkeley
Elizabeth Curda, U.S. Government Accountability Office

11:25 – 11:45 a.m. EDT

Design & Use of Demonstrations Audience Q&A

11:45 am – 12:00 p.m. EDT

Break

12:00 – 12:30 p.m. EDT

Keynote
Kilolo Kijakazi, U.S. Social Security Administration

12:35 – 1:10 p.m. EDT

Return to Work
Panel Chair: Sarah Prenovitz, Abt Associates

Presenters: Robert Moffitt, Johns Hopkins University
Jesse Gregory, University of Wisconsin – Madison

Discussants: Hilary Hoynes, University of California, Berkeley
Kathleen Romig, Center on Budget and Policy Priorities

1:15 – 1:50 p.m. EDT

Early Intervention
Panel Chair: Sarah Prenovitz, Abt Associates

Presenter: Kevin Hollenbeck, W.E. Upjohn Institute for Employment Research

Discussants: Jeffrey Liebman, Harvard University
Jennifer Sheehy, U.S. Department of Labor

1:50 – 2:10 p.m. EDT

Return to Work & Early Intervention Audience Q&A

2:10 – 2:20 p.m. EDT

Break

2:20 – 3:05 p.m. EDT

Youth Transition
Panel Chair: Daniel Gubits, Abt Associates

Presenters: David Wittenburg, Mathematica
Gina Livermore, Mathematica

Discussants: Lucie Schmidt, Williams College
Manasi Deshpande, University of Chicago
Jennifer Sheehy, U.S. Department of Labor

3:10 – 3:45 p.m. EDT

Heterogeneity: Subgroup Findings
Panel Chair: Daniel Gubits, Abt Associates

Presenter: Till von Wachter, University of California, Los Angeles

Discussants: Howard Goldman, University of Maryland
Nick Hart, The Data Foundation

3:45 – 4:05 p.m. EDT

Youth Transition & Heterogeneity: Subgroup Findings Audience Q&A

4:05 – 4:15 p.m. EDT

Break

4:15 – 4:50 p.m. EDT

Benefits Counseling & Case Management
Panel Chair: Sarah Prenovitz, Abt Associates

Presenter: Vidya Sundar, University of New Hampshire

Discussants: John Kregel, Virginia Commonwealth University
Leslynn Angel, Michigan United Cerebral Palsy

4:55 – 5:30 p.m. EDT

Implementation
Panel Chair: Sarah Prenovitz, Abt Associates

Presenters: Michelle Wood, Abt Associates
Debra Goetz Engler, U.S. Social Security Administration

Discussants: David Stapleton, Tree House Economics
Calvin Johnson, U.S. Department of Housing and Urban Development

5:30 – 5:50 p.m. EDT

Benefits Counseling & Case Management,
Implementation Audience Q&A

5:50 – 6:00 p.m. EDT

Closing Remarks
Jeffrey Hemmeter, U.S. Social Security Administration
Austin Nichols, Abt Associates

Presentation Summaries

Design of Demonstrations 

Burt Barnow and David Greenberg reviewed the design of evaluations that are typically part of a demonstration. They reviewed the designs of past demonstrations’ evaluations and discussed the implications for what questions those designs can address. They also offered thoughts on alternative designs that the U.S. Social Security Administration (SSA) might consider in the future for greater learning. They encouraged SSA to consider multiple, varied treatment arms and factorial designs to help determine the role of the component parts of the interventions. Additionally, Barnow and Greenberg suggested additional designs, for example stepped wedge designs, that could help provide additional estimates. They also suggested going beyond the intent-to-treat (ITT) estimates that have been the default in SSA’s existing demonstrations and encouraged treatment-on-the-treated (TOT) estimates as well. Finally, they noted how important process analyses are and suggested that an increased use of fidelity measures could help SSA learn more about the specific interventions.

Use of Demonstrations

Robert Weathers and Austin Nichols discussed ways to improve the use of evaluation findings and implications for how demonstrations should be used. They focused on questions policymakers have historically wanted answered and how to better communicate findings to meet those needs. Weathers and Nichols encouraged strong theoretical models (including logic models) to underpin the demonstrations and clarify the goals of the demonstration. While acknowledging the existence of tradeoffs, they also urged SSA to consider going beyond the single intervention tests by looking at broad ranges of similar policy options. Looking inside the black box, through multiple treatment arms, factorial designs, etc., is important to answering questions related to why something worked. They also encouraged additional uses of qualitative findings and reanalyzing the data from past demonstrations to extend the analyses included in evaluation contract reports.  

Return to Work

Jesse Gregory and Robert Moffitt reviewed the lessons from SSA’s return-to-work demonstrations such as the Benefit Offset National Demonstration and similar demonstrations. They described the incentives that individuals receiving Social Security Disability Insurance (SSDI) and Supplemental Security Income (SSI) benefits face. They described the history of efforts to improve this population’s work outcomes, including how the demonstrations align with economic theory and what researchers expected to find. They also proposed several considerations for SSA’s next generation of demonstrations, including ideas related to the prior two presentations. One important lesson they noted is that most efforts to increase employment, earnings, and labor force engagement do not have large effects. As a result, it may be necessary to reconsider expectations about how many beneficiaries will go back to work. Promising areas to explore could include work incentives like the earned-income tax credit that provide additional income above and beyond the current benefit (as opposed to just not taking as much away) at first.

Early Intervention

Kevin Hollenbeck provided an overview of SSA’s efforts to explore policies related to individuals not yet receiving disability benefits. This included early interventions that might prevent some of them from needing the SSDI or SSI programs for support. Programs that effectively reduce the need for SSDI or SSI could both improve individuals’ economic well-being and reduce government expenditures on these programs, allowing for a more efficient use of program resources. Hollenbeck shared lessons from his review of both U.S. and international programs related to SSDI and SSI. He noted that Individualized Placement and Support has been tested in several settings and while it does appear to have some success in improving labor market outcomes, there is little evidence that this translates into reductions in SSI or SSDI benefits. He recommended testing interventions targeting older denied applicants rather than a large swath of potential applicants.

Youth Transition

David Wittenburg and Gina Livermore reviewed lessons from SSA’s efforts to support youth receiving benefits in transitioning to a successful and more self-sufficient adulthood. These efforts generally estimate mixed effects on benefit receipt, but are promising in improving participants’ social connections. Wittenburg and Livermore discussed new policy directions and partnerships SSA could explore. They noted that youth needs are different from those of adults and are not separable from families. As a result, services that focus on family outcomes are important. Additionally, interagency collaborations, such as those utilized in the Promoting Readiness of Minors in SSI project are important. Given the large number of services and program models for youth, building on existing programs, like Job Corps, could provide fruitful next steps.

Heterogeneity: Subgroup Findings 

Till von Wachter reviewed the importance of looking at the heterogeneous impacts of the demonstrations. He showed why subgroup impacts are important to estimate. He also provided some suggestions for groups that might be important to look at more closely. von Wachter noted that each demonstration uses different definitions for earnings outcomes, age groups, disability groups, etc., which hinders cross-demonstration comparisons. He noted that standardized outcomes and subgroup definitions would be helpful when comparing across demonstrations. He also described the state of the art on identifying subgroups with different impacts.

Benefits Counseling & Case Management

Vidya Sundar reviewed the use of benefits counseling and case management in SSA’s demonstrations. Sundar explored the challenges in measuring the effectiveness of these commonly used but often differently implemented services. She noted that we need more information on the timing and nature of benefits counseling and its interaction with other services (e.g., vocational rehabilitation). Additionally, more information is needed on models that focus on sustaining employment rather than just getting a job.

Implementation

Michelle Wood and Debra Goetz Engler drew lessons from demonstrations’ implementation reports. Wood and Engler delved into the qualitative and process analyses to glean lessons from how the beneficiaries were recruited, how the demonstrations were run, and how the interventions were delivered. They also highlighted the importance of recruitment and measurement of fidelity to a program model. On recruitment, they noted that dedicated recruitment staff often have advantages over staff responsible for both recruitment and delivery. They also noted that site selection and intervention fidelity are important for structured, specialized services. Wood and Engler pointed out tradeoffs between centralized and decentralized funding and implementation with respect to the feasibility of data systems, operational policies, monitoring and other factors. Finally, they noted that emergency and basic needs of participants may impede participation and engagement in the intervention. Their insights are helpful for understanding how SSA could implement future demonstration efforts as well as conduct general outreach.