Evaluation Findings Show Positive Outcomes from Algebra Intervention

Caroline Parker has 13 years of experience conducting educational policy research at the federal, state, and local levels. Her research has a special focus on educational equity, including identifying strategies to remove barriers to achievement for students who are English language learners, students with disabilities, and other students from underserved populations. She has also published on large-scale assessment and youth motivation in formal and informal education. In this post, Caroline provides a summary of new findings from one of her studies. She shared these findings on September 10–11 at the U.S. Department of Education’s Math and Science Partnerships Program (MSP) Annual Conference in Washington, DC.

Over the past three years, I have led an evaluation of the Connecticut Common Core State Standards (CCSS) Algebra I Curriculum. In 2009, a team of high school and college teachers wrote this state-sponsored curriculum, and it aligns well with the Common Core State Standards for Mathematics (CCSSM).

When Connecticut received funding from the U.S. Department of Education’s Mathematics and Science Partnerships (MSP) program for a three-year pilot of the curriculum, state leaders asked EDC to serve as the evaluation partner. The state designed the pilot to lay the groundwork to roll the curriculum out statewide by testing a model to provide training and support to help teachers effectively implement the curriculum. Key phases included developing a professional development (PD) plan, revising the plan based on teacher feedback, and conducting a comparative field study. EDC’s team of evaluators—Christina Bonney, Becca Seston, and myself—was pleased to support the state in this important effort.

During the first two years of the project, we conducted a formative evaluation of the intervention. We collected data and shared it with the developers to guide revisions to the curriculum and PD. In the third and final year, we conducted a quasi-experimental study in which we compared the outcomes of students in the intervention group—students whose teachers used the CT CCSS Algebra I Curriculum and received the PD—with students from a comparison group, whose teachers did not use the curriculum.

The state recruited 16 districts from across the state that were interested in participating in the study. We matched similar districts into pairs, and randomly assigned one district in each pair to the intervention or comparison group. Each district had all Algebra I teachers and students (both middle and high school) take part in the intervention or comparison group. Our final sample of 3,228 students included 1,686 CCSS Algebra I Curriculum students taught by 55 teachers, and 1,542 comparison students taught by 45 teachers. Students in both groups completed a baseline algebra-readiness test at the beginning of the year and an Algebra I test at the end of the year. Both the readiness and Algebra I tests were developed and validated by EDC experts (Eric Karnowski and Craig Hoyle). Teachers in both groups completed pre- and post-tests focused on their instructional practices.

A brief overview of the study’s key findings follows:

  1. We found there was a statistically significant positive effect on Algebra I outcomes for students in the CCSS Algebra I Curriculum group. Controlling for the results of the baseline algebra-readiness test, students whose teachers used the CCSS Algebra I Curriculum had scores that were, on average, 1.3 points higher (out of 20) than their peers in the comparison group. This held true for both middle school and high school students.
  2. We found that instructional practices differed between teachers who used the CCSS Algebra I Curriculum and received the PD and teachers in the comparison group (as measured by a Survey of Teacher Instructional Practices). The CCSS Algebra I Curriculum teachers had statistically significantly higher scores on the scale of Pedagogical Preparedness, which included developing students’ conceptual understanding of mathematics and making connections between mathematics and other disciplines, as well as on the scale measuring their Emphasis on CCSS Mathematics Practices.

As we move forward, we will be prioritizing sharing these and related findings to advance the field’s knowledge of this promising curriculum to enhance mathematics teaching and learning statewide.