Afterschool Programs Make a Difference: Findings From the Harvard Family Research Project

In February, the Harvard Family Research Project (HFRP) published After School Programs in the 21st Century: Their Potential and What It Takes to Achieve It (Little, Wimer, & Weiss, 2008), a brief that summarizes 10 years of research on afterschool programs and discusses implications for the future. Featured in the brief are studies that evaluate large afterschool programs with experimental or quasi-experimental designs. The authors, Priscilla M. D. Little, Christopher B. Wimer, and Heather B. Weiss, drew on those evaluations to address two primary questions: 1) Does participation in after school programs make a difference, and, if so, 2) What conditions appear to be necessary to achieve positive results? In this article, we summarize their findings and discuss the characteristics of programs leading to positive student outcomes.

Does participation in afterschool programs make a difference?

According to Little, Wimer, and Weiss,

The short answer is yes. . . .A decade of research and evaluation studies, as well as large-scale, rigorously conducted syntheses looking across many research and evaluation studies, confirms that children and youth who participate in after school programs can reap a host of positive benefits in a number of interrelated outcome areas—academic, social/emotional, prevention, and health and wellness. (2008, p. 2)

Academic Achievement

Afterschool programs can have an impact on academic achievement. Improved test scores are reported in evaluations of The After-School Corporation (TASC) programs in New York City (Reisner, White, Birmingham, & Welsh, 2001; White, Reisner, Welsh, & Russell, 2001) and in Foundations, Inc. elementary school programs (Klein & Bolus, 2002). A more recent longitudinal study showed significant gains in math test scores for elementary and middle-school students who participated in high-quality afterschool programs (Vandell, Reisner, & Pierce, 2007), and a meta-analysis of 35 studies of at-risk youth found that out-of-school time programs had a positive effect on reading and math achievement (Lauer, Akiba, Wilkerson, Apthorp, Snow, & Martin-Glenn, 2006).

The HFRP brief emphasizes that many studies “repeatedly underscore the impact of supporting a range of positive learning outcomes, including academic achievement, by affording children and youth opportunities to learn and practice new skills through hands-on, experiential learning,” (p. 3) citing evaluations of Citizen Schools (Espino, Fabiano, & Pearson, 2004; Fabiano, Pearson, & Williams, 2005) and of LA’s BEST (Huang, Coordt, La Torre, Leon, Miyoshi, & Pèrez, et al., 2007), among others. These programs not only offered academic support to improve academic performance, but also combined it with other enrichment activities to achieve positive academic outcomes. Little, Wimer, and Weiss noted,

Thus, extra time for academics by itself may be necessary but may not be sufficient to improve academic outcomes. Balancing academic support with a variety of engaging, fun, and structured extracurricular or co-curricular activities that promote youth development in a variety of real-world contexts appears to support and improve academic performance. (2008, p. 4)

Social and Emotional Development

Programs with a strong intentional focus on improving social and personal skills were found to improve students’ self-esteem and self-confidence (Durlak & Weissberg, 2007). Examples include Go Grrls, an Arizona program of structured group sessions that helps improve girls’ body image, assertiveness, self-efficacy, and self-liking (LeCroy, 2003) and mentoring programs such as Across Ages (Taylor, LoSciuto, Fox, & Hilbert, 1999), which pairs older adults with students.

Prevention of Risky Behaviors

The hours after school, between 3 p.m. and 6 p.m., offer opportunities for juvenile crime, sexual activity, and other risky behaviors such as drug and alcohol use. Research and evaluation studies have shown that participation in afterschool programs have a positive impact on juvenile crime and help reduce pregnancies, teen sex, and boys’ marijuana use (Goldschmidt, Huang, & Chinen, 2007; Philliber, Kaye, & Herrling, 2001; Philliber, Kaye, Herrling, & West, 2002).

Health and Wellness

The afterschool setting presents an opportunity to address the growing problem of obesity among children and youth. Research has shown that afterschool programs can contribute to healthy lifestyles and increased knowledge about exercise and nutrition. Girlfriends for KEEPS (Story, et al., 2003) and the Medical College of Georgia’s FitKid program (Yin, Gutin, Johnson, Hanes, Moore, Cavnar, et al., 2005) are two such programs that benefit their participants; similar results are reported in a longitudinal study of more than 650 students who participated in 25 Connecticut afterschool programs (Mahoney, Lord, & Carryl, 2005).

What conditions appear to be necessary to achieve positive results?

Little, Wimer, and Weiss wrote that while afterschool programs “have the potential to impact a range of positive learning and development outcomes,” some programs do not maximize this potential. They identified the following three factors as critical to achieving positive youth outcomes:

• Access to and sustained participation in the program
• Quality programming and staffing
• Strong partnerships among the program and other places where students are learning, such as their schools, their homes, and other community institutions

Access to and Sustained Participation

The HFRP brief discussed a number of research syntheses (American Youth Policy Forum, 2006; Redd, Cochran, Hair, & Moore, 2002; Simpkins-Chaput, Little, & Weiss, 2004) and evaluations such as those of the After School Matters program in Chicago (Goerge, Cusick, Wasserman, & Gladden, 2007), Louisiana’s 21st Century Community Learning Center (CCLC) program (Jenner & Jenner, 2004), and LA’s Best (Huang, et al., 2007) that show that students experience greater gains if they participate regularly in afterschool programs, with greater frequency (more days per week), and in a sustained manner over a number of years.

Much like gaps among students in regular day school, Little, Wimer, and Weiss noted differences among students whose families have higher incomes and more education and those students whose families are less advantaged. They wrote (p. 6) that students whose families have higher incomes and more education:

• Are more likely to participate in afterschool activities
• Do so with greater frequency during the week
• Participate in a greater number of different activities within the week or month
• Are more likely to participate in enrichment programs, whereas disadvantaged students are more likely to participate in tutoring programs

Quality Programming and Staffing

According to Little, Wimer, and Weiss, research on the quality of afterschool programs is mostly descriptive, with only “a handful of rigorously designed studies.” They have drawn from a set of studies they describe as “a small but powerful set of studies.”

Regarding program structure and supervision, Little, Wimer, and Weiss (p. 6) conclude, “Without the structure and supervision of focused and intentional programming, youth participants in after school programs, at best, can fail to achieve positive outcomes and, at worst, can begin to perform worse than their peers” (Vandell, Pierce, Brown, Lee, Bolt, & Dadisman, 2006; Pearson, Russell, & Reisner, 2007). They continue, “In fact, some research finds that when youth are concentrated together without appropriate structure and supervision, problematic behavior follows, suggesting that focused, intentional activities with appropriate structure and supervision are necessary to keep youth on an upward trajectory and out of trouble” (Jacob & Lefgren, 2003).

In a meta-analysis of the impact of 73 afterschool programs, Durlak and Weissburg (2007) found that programs missing any of the following four characteristics did not achieve positive results:

• Sequenced – Used sequenced set of activities designed to achieve skill development objectives
• Active – Used active forms of learning to help students develop skills
• Focused – Devoted program components to developing personal or social skills
• Explicit – Targeted explicit personal or social skills

Other studies (Gerstenblith, Soule, Gottfredson, Lu, Kellstrom, Womer, et al., 2005; Arbreton, Goldsmith, & Sheldon, 2005) found that programs with structured and focused, well-organized activities foster engagement and facilitate high quality learning opportunities.

According to Little, Wimer, and Weiss, the quality of a program’s staff is one of the most critical features of a high-quality afterschool program. A follow-up study to the TASC evaluation found that positive relationships were found in sites where staff modeled positive behavior, actively promoted student mastery of the skills or concepts presented in activities, listened attentively to participants, frequently provided individualized feedback and guidance during activities, and established clear expectations for mature, respectful peer interactions (Birmingham, Pechman, Russell, & Mielke, 2005). Other research found that in low-quality programs, the staff “engaged in negative and punitive interactions with youth” instead of “engaging in supportive behavior and practicing positive behavior management techniques” (Vandell, Shumow, & Posner, 2005; Gerstenblith, et al., 2005).

Strong Partnerships

Little, Wimer, and Weiss also found:

Programs are more likely to exhibit high quality when they effectively develop, utilize, and leverage partnerships with a variety of stakeholders like families, schools, and communities. However, strong partnerships are more than a component of program quality: they are becoming a nonnegotiable element of supporting learning and development across all the contexts in which children learn and develop. (p. 8)

After School in the 21st Century

Little, Wimer, and Weiss summarize:

The research and evaluation studies and syntheses highlighted in this brief demonstrate how complex a task it is to provide high quality, effective supports for youth and their families, but they also provide powerful evidence that after school programs do work when key factors are addressed—factors of access, sustained participation, program quality and strong partnerships. (p. 10)

They also conclude that the research and evaluation results from the past decade raise the following important questions about the future of afterschool programs and their role:

• Moving forward, how can the research-based practices known to be effective in after school programs be adopted more broadly within after school programs and other expanded learning models?
• How can after school programs work with schools, families, and other community and health supports to ensure a complementary array of learning and developmental supports across the day, the year, and the developmental continuum from kindergarten through high school?
• Moving forward, how can and should “success” of after school programs be measured, particularly as the field moves toward greater emphasis on shared responsibility and partnerships?
• How can choice be built into after school and extended day options to ensure that programs are responsive to the needs of working families and youth participants alike?

References

• American Youth Policy Forum. (2006). Helping youth succeed through out-of-school time programs. Washington, DC: Author.
• Arbreton, A. J. A., Goldmith, J., & Sheldon, J. (2005). Launching literacy in after-school programs: Early lessons from the CORAL initiative. Philadelphia: Public/Private Ventures.
• Birmingham, J., Pechman, E. M., Russell, C. A., & Mielke, M. (2005). Shared features of highperforming after-school programs: A follow-up to the TASC evaluation. Washington, D. C.: Policy Studies Associates.
• Durlak, R., & Weissberg, R. (2007). The impact of after-school programs that promote personal and social skills. Chicago: CASEL.
• Espino, J., Fabiano, L., & Pearson, L. M. (with Kirkwood K. P., Afolabi, K., & Pasatta, K.). (2004). Citizen Schools: Evidence from two student cohorts on the use of community resources to promote youth development. Phase II report of the Citizen Schools evaluation. Washington, DC: Policy Studies Associates.
• Fabiano, L., Pearson, L. M., & Williams, I. J. (2005). Putting students on a pathway to academic and social success: Phase III findings of the Citizen Schools evaluation. Washington, DC: Policy Studies Associates.
• Gerstenblith, S., Soule, D., Gottfredson, D., Lu, S., Kellstrom, M., Womer, S., et al. (2005). Afterschool programs, antisocial behavior, and positive youth development: An exploration of the relationship between program implementation and changes in youth behavior. In J. Mahoney, J. Eccles, & R. Larson (Eds.), Organized activities for contexts for development: Extracurricular activities, after-school, and community programs (pp. 457-477). Mahwah, NJ: Erlbaum.
• Goerge, R., Cusick, G. R., Wasserman, M., & Gladden, R. M. (2007). After-school programs and academic impact: A study of Chicago’s After School Matters. Chicago: Chapin Hall Center for Children.
• Goldschmidt, P., Huang, D., & Chinen, M. (2007). The long-term effects of after-school programming on educational adjustment and juvenile crime: A study of the LA’s BEST after-school program. Los Angeles: UCLA/CRESST. Available at http://www.lasbest.org/resourcecenter/LASBEST_DOJ_Study_Brief.pdf
• Huang, D., Coordt, A., La Torre, D., Leon, S., Miyoshi, J., Pérez, P., et al. (2007). The afterschool hours: Examining the relationship between afterschool staff-based social capital and student engagement in LA’s BEST. Los Angeles: UCLA/CRESST. Available at http://www.cse.ucla.edu/products/reports/R712.pdf
• Jacob, B. A., & Lefgren, L. (2003). Are idle hands the devil’s workshop? Incapacitation, concentration, and juvenile crime. American Economic Review, 93, 1560-1577.
• Jenner, E. J., & Jenner, L. W. (2004). Academic outcomes in Louisiana’s 21st Century Community Learning Centers. Baton Rouge, LA: Policy & Research Group.
• Klein, S. P., & Bolus, R. (2002). Improvements in math and reading scores of students who did and did not participate in the Foundations After School Enrichment Program during the 2001–2002 school year. Santa Monica, CA: Gansk & Associates.
• Lauer, P. A., Akiba, M., Wilkerson, S. B., Apthorp, H. S., Snow, D., & Martin-Glenn, M. L. (2006). Out-of-school time programs: A meta-analysis of effects for at-risk students. Review of Educational Research, 76, 275–313.
• LeCroy, C. W. (2003). Experimental evaluation of “Go Grrls.” Tucson, AZ: Author.
• Little, P. M.D., Wimer, C. , & Weiss, H. B. (2008, February). After school programs in the 21st century: Their potential and what it takes to achieve it. Issues and Opportunities in Out-of-School Time Evaluation Brief No. 10. Cambridge, MA: Harvard Family Research Project.
• Pearson, L. M., Russell, C. A., & Reisner, E. R. (2007). Evaluation of OST programs for youth: Patterns of youth retention in OST programs, 2005–06 to 2006–07. Washington, DC: Policy Studies Associates.
• Philliber, S., Kaye, J. W., & Herrling, S. (2001, May). The national evaluation of the Children’s Aid Society Carrera-Model Program to prevent teen pregnancy. Accord, NY: Phillber Research Associates. Available at http://www.childrensaidsociety.org/media/general/cas-Teen_Pregnancy_Prevention.pdf
• Philliber, S., Kaye, J. W., Herrling, S., & West, E. (2002). Preventing pregnancy and improving health care access among teenagers: An evaluation of the Children’s Aid Society—Carrera Model. Perspectives on Sexual and Reproductive Health, 34(5), 244–251. Available at http://www.agi-usa.org/pubs/journals/3424402.pdf
• Redd, Z., Cochran, S., Hair, E., & Moore, K. (2002). Academic achievement programs and youth development: A synthesis. Washington, DC: Child Trends.
• Reisner, E. R., White, R. N., Birmingham, J., & Welsh, M. (2001). Building quality and supporting expansion of After-School Projects: Evaluation results from the TASC After- School Program’s second year. Washington, DC: Policy Studies Associates.
• Simpkins-Chaput, S., Little, P. M. D., & Weiss, H. B. (2004). Understanding and measuring attendance in out-of-school time programs. Issues and Opportunities in Out-of-School Time Evaluation Brief No. 7. Cambridge, MA: Harvard Family Research Project.
• Story, M., Sherwood, N. E., Himes, J. H., Davis, M., Jacobs, Jr., D. R., Cartwright, Y., et al. (2003). An after-school obesity prevention program for African-American girls: The Minnesota GEMS Pilot Study [Supplement 1]. Ethnicity & Disease, 13(1), 54–64.
• Taylor, A., LoSciuto, L., Fox, M., & Hilbert, S. (1999). The mentoring factor: An evaluation of Across Ages. Intergenerational program research: Understanding what we have created. Binghamton, NY: Haworth.
• Vandell, D., Reisner, E., & Pierce, K. (2007). Outcomes linked to high-quality afterschool programs: Longitudinal findings from the study of promising practices. Irvine, CA: University of California and Washington, DC: Policy Studies Associates. Available at http://www.gse.uci.edu/docs/PASP%20Final%20Report.pdf
• Vandell, D., Pierce, K., Brown, B., Lee, D., Bolt, D., Dadisman, K., et al. (2006, March). Developmental outcomes associated with the afterschool contexts of low-income children and adolescents. Paper presented at the Society for Research on Adolescence Annual Meeting, San Francisco.
• White, R. N., Reisner, E. R., Welsh, M., & Russell, C. (2001). Patterns of student-level change linked to TASC participation, based on TASC projects in Year 2. Washington, DC: Policy Studies Associates.
• Yin, Z., Gutin, B., Johnson, M., Hanes, J., Jr., Moore, J. B., Cavnar, M., et al. (2005). An environmental approach to obesity prevention in children: Medical College of Georgia FitKid Project year 1 results. Obesity Research, 13, 2153–2161.

Next Article: SEDL's Afterschool Work Deepens with California Demonstration Program