Louisiana Striving Readers Evaluation Project (2009-2013)

Louisiana was one of eight states to be awarded a 4-year Striving Readers grant from the U.S. Department of Education on October 1, 2009. SEDL, in partnership with the Louisiana Department of Education, conducted a 4-year randomized controlled trial study of the Voyager Passport Reading Journeys curriculum for struggling adolescent readers.

The Striving Readers program aims to improve the literacy skills of middle school- and high school-aged students who are reading 2 or more years below grade level and to help build a strong, scientific research base around specific strategies that improve adolescent literacy skills. With a grant of more than $5 million, Louisiana implemented a middle school supplemental reading program, Passport Reading Journeys, developed by Voyager Expanded Learning, Inc. To date, 134 districts and 435 schools have adopted the Passport Reading Journeys curriculum across 35 states.

About the Program

The 4-year study was designed to assess whether the Voyager Passport Reading Journeys (PRJ), a supplemental curriculum program, affects reading outcomes for students relative to the control students enrolled in non-reading electives.

The study assessed under which conditions the program has the greatest impact. This study involved the random assignment of three cohorts of 1,400 sixth and seventh graders from 10 Title I middle schools and four districts in Louisiana over 3 years of data collection.

The project was funded through a Striving Readers grant from the U.S. Department of Education, Office of Elementary and Secondary Education, Academic Improvement and Teacher Quality Programs. Click here for more information.

Research Questions

• Research Question 1: Does student-level assignment to the Passport Reading Journeys (PRJ) supplemental literacy program produce impacts on reading achievement for 6th and 7th grade students significantly behind their peers (i.e., two years or more below grade level) at the beginning of the school year relative to student assignment to a supplemental non-reading elective services control condition?
• Research Question 2: Describe the extent to which the intervention was delivered as the curriculum developers indicated it should be implemented. Was there significant variation in implementation fidelity among the schools and classrooms in the treatment group?
• Research Question 3: Does student-level assignment to the PRJ supplemental literacy program produce impacts on students' motivation to read for 6th and 7th grade students significantly behind their peers (i.e., two years or more below grade level) at the beginning of the school year relative to student assignment to a supplemental non-reading elective services control condition?
• Research Question 4: For which subgroups and under which conditions does the program have its greatest impact?
• Research Question 5: Is there a significant relationship between implementation fidelity (specific and general) and student outcomes and does this relationship vary by classrooms, teachers, schools, and districts?

Research Design and Methods

• Purpose: The Louisiana Department of Education worked with SEDL and Voyager Expanded Learning to conduct an independent study of the Voyager PRJ program for striving readers in 10 Louisiana middle schools to determine: 1) whether the program impacts student reading and other student outcomes, and 2) how the effects of the program vary across students, classrooms/teacher, and schools. The widespread use of the this program, years of delivery methods found to be effective in reading instruction for adolescents, and promising findings in quasi-experimental studies suggested the program is well positioned for an independent efficacy trial to determine its effectiveness.
• Setting: Middle schools serving students in regions and locales across the state of Louisiana.
• Population sample: Approximately 1,400 students per year in Grade 6-7 from 10 middle schools and 4 parishes that are new subscribers to the PRJ program.
• Intervention: PRJ, widely used as a supplemental reading program for grades 6 through 9, relies on the research-based practices cited in the National Reading Panel (2000) report and emphasizes vocabulary, text comprehension, metacognitive development, fluency, decoding, and writing. The program also incorporates the fifteen key elements of effective adolescent literacy programs including direct, explicit comprehension instruction, motivation and self-directed learning, a technology component, and ongoing formative assessment (Biancarosa & Snow, 2004). To date, 134 districts and 435 schools have adopted Passport Reading Journeys curriculum across 35 states. The curriculum includes student materials, teacher manuals, diagnostic and assessment packages, and online data management and library resources. The program includes a 2-day launch training to prepare literacy intervention teachers to implement the curriculum as well as follow-up support by consultants. The instructional format consistently follows a standard protocol that provides daily, 50-minute lessons that provide explicit systematic instruction in critical reading skills and facilitates grouping and differentiation of instruction. It is formatted as a series of 15 two-week reading expeditions on engaging topics across a number of content areas.
• Control or comparison condition: Reading instruction as usual, and other activities in which they would otherwise participate, such as study hall, electives, or another activity that does not involve supplemental literacy instruction.
• Research method: The study design involved a multi-site randomized controlled trial (RTC) in which students who are two or more grade levels behind in reading in 10 middle schools were randomly assigned to one of two groups: 1) literacy intervention teachers that are trained in PRJ program and deliver the curriculum, 2) teachers that continue to deliver reading instruction as usual. The design followed teachers and students from three cohorts (grades6-7) over 3 school years (2010-2011, 2011-2012, 2012-2013) Power calculations indicate that 10 schools were needed to detect an effect size of .10 for main effect and subgroup analyses (n=1,200 students).

Key Measures of Student Reading Outcomes

• Measures of key outcomes: All students for each cohort were assessed in reading achievement in the fall (pretest) and spring (posttest) of each year using a group-administered reading test (GRADE). Fidelity of implementation was captured by classroom observations and interviews with teachers and principals in the fall and spring of each year. Additional student outcomes were captured each fall and spring using teacher interviews and surveys.
• Data analytic strategy: Data analyses were conducted using a two-level hierarchical linear model with fall pretests as covariates and spring posttests on GRADE as the dependent variables. Student-level intention-to-treat effects across all school and for selected subgroups were the primary outcomes of interest. Treatment fidelity data and other students and school characteristics were examined as potential moderators of the effects.