The Need for Better Data about K-12 Online Learning: Starting with How and Why

Peggy Clements, senior research scientist, has worked in the overlapping fields of developmental psychology and education research for over 20 years. Much of her research for EDC's Regional Educational Laboratory Northeast & Islands (REL Northeast & Islands) and the Regional Educational Laboratory Midwest’s (REL Midwest) Virtual Education Research Alliance focuses on advancing knowledge of the purposes, modalities, and outcomes of online learning in K–12 schools. In this post, Peggy reflects on the importance of conducting research about online learning and discusses the online course survey she and colleagues developed in collaboration with the Virtual Education Research Alliance.

Although the use of online courses has been increasing rapidly for at least the past 10 years, there is very little solid research about how best to use online learning to promote students’ achievement. Previously, as co-PI of the REL Northeast & Islands' Access to Algebra I: Online Mathematics for Grade 8 Students Study, I worked with colleagues to carry out one of the first rigorous investigations of K–12 online learning.

While the large-scale randomized controlled trial that we conducted clearly showed that using an online course was an effective way to expand rural students’ access to eighth-grade algebra, the study left me with many questions about how best to use online learning in ways that truly benefit students’ learning. Nationwide, K–12 policy-makers, leaders, and educators need to know not just whether online courses work or do not work, but under what circumstances they work, for what purposes they work, and what types of support—from online teachers, school staff, and even other students—benefit students when they’re participating in an online course. 

However, learning more about the effective uses of online learning is complicated by the lack of information available. Few states and districts collect data that can tell us about the myriad of ways in which online learning is being used in schools, why schools and students are turning to these kinds of courses, and how students are performing in these courses. Just a few of the ways in which students’ experiences in an online course can vary include the following:

  • Students take online courses to accomplish different educational objectives—ranging from advanced courses to core requirements to credit recovery courses—across the full range of academic subjects.
  • Some online courses provide an online teacher while others leave students to work through the courses on their own.
  • When there is an online teacher, some courses provide many ways for students to communicate and interact with the online teacher and each other while others may provide few or no ways to communicate.
  • Some schools appoint a staff member to supervise and support students in an online course, while other schools do not offer this support.
  • Sometimes students take an online course with other students in their school; in other courses, students work alone.

In addition, data systems have not yet caught up with what’s happening on the ground. This gap in the knowledge base severely impedes the ability of policy-makers and practitioners to make informed decisions. It also hampers researchers’ efforts to determine where there is the greatest need for research, as well as our ability to conduct studies. If we do not know what is happening, it is impossible for us to learn if it is working.

I had not realized the extent to which these kinds of data were lacking until I began working with the REL Midwest’s Virtual Education Research Alliance, along with EDC colleagues Erin Stafford, Angela Pazzaglia, and Jackie Zweig, and Pam Jacobs of American Institutes for Research. A quick word about the research alliance: A key goal for all 10 of the Institute of Education Sciences (IES)-funded Regional Educational Laboratories is to develop research alliances—partnerships between researchers and education practitioners—that develop research agendas to address the topics practitioners identify as most critical. The Virtual Education Research Alliance is one of many such alliances that are studying a wide range of key education reform issues—from early childhood education to college and career readiness to meeting the unique needs of students served in rural and urban schools to dropout prevention to educator effectiveness.

From our very first meeting, alliance members were very clear that they and their states needed data about how and why schools were using online courses for students. I had not expected that this would be where the alliance members would want to start, but it is an example of why researcher-practitioner partnerships are so important. What may be a pressing question for researchers may not be the most pressing question for practitioners. It was a productive place to begin our work together. Our preliminary discussions about developing a survey to gather the data the alliance members needed led to conversations about some of the bigger questions the alliance members have about how to best implement online learning opportunities for students. These early conversations continue to guide our work a couple of years later.

One of the alliance’s priority research topics quickly became: "What data can states and districts collect to document and track how and why schools use online courses?" Beneath this question lie a number of more nuanced and important questions including, but not limited to: “Are online courses used to provide access to advanced courses, to provide access to electives, or to provide credit recovery?” “Why are schools turning to online courses for their students?” “Do the courses use an online teacher or are they self-guided, requiring students to take initiative in engaging in and completing the courses?” “How are schools monitoring students’ performance in online courses?” “Do schools appoint a staff member to supervise and support students enrolled in online courses?” “If schools appoint a staff member to supervise and support online students, what kinds of training do they receive and what are their responsibilities?”

Our team of researchers worked collaboratively with alliance members to develop and validate a survey that addresses these questions and more. After two state departments of education in the Midwest sent the survey to their public high schools, the research team analyzed the data and wrote a report describing the findings. When members of the REL Northeast & Islands' Rural Districts Research Alliance heard about the survey, they wanted to use it to collect data about what was happening closer to home. We worked with them to adapt the survey for their purposes and to plan for their own data collection from high schools in the Capital Region of New York. A report on this study should be published early next year. 

Recognizing the need for these data across the country, I presented about the survey at the 2014 International Association for K–12 Online Learning (iNACOL) Symposium in Palm Springs on November 5th with Virtual Education Research Alliance members Dawn Nordine (Executive Director, Wisconsin Virtual School) and Marcel Kielkucki (Director, High School Completion Program, Kirkwood Community College). We talked about what motivated the survey, described key features of the procedures used to develop and administer the survey, and shared information about how the states are using the findings.

By sharing information about the survey and how other states and districts can use it, our goal was to help meet the need for data about online learning. The survey instrument we designed and the knowledge it generated are important first steps in supporting efforts to better understand how and why schools are enrolling students in online courses and advancing future efforts to investigate virtual education. Right now, you can read our report, Online Course Use in Iowa and Wisconsin Public High Schools—which includes the survey instrument—on the IES website.