Survey Tool Hones In on Competency-Based Learning

Joshua Cox

 Joshua Cox is the Alliance Researcher for the Northeast College and Career Readiness Research Alliance of the REL Northeast & Islands at EDC. His recent work for the Alliance has focused primarily on competency-based learning. Recently, the U.S. Department of Education Institute of Education Sciences published a new survey tool and guide co-authored by Joshua and his colleague Sarah Ryan that meets the growing need to systematically examine students' experiences with competency-based learning. In this post, first published by the REL Northeast & Islands, Joshua discusses the new survey tool, shares the process used to develop the tool, describes the kinds of data the tool can help schools and districts collect, and describes REL Northeast & Islands resources that can support use of the tool.

The Institute of Education Sciences (IES) has released a new survey tool and guide that can help schools and districts measure student beliefs about, exposure to, and understanding of competency-based learning.

Collaboratively developed by REL Northeast & Islands at EDC researchers and regional stakeholders, the “Guide to the Competency-based Learning Survey for Students” can be used by administrators, teachers, and other staff in high schools that have already implemented or are beginning to implement competency-based learning to gather information about how students understand and experience this reform. The survey guide also provides advice for users on how to adapt survey items to their local context.

Why Was the Survey Developed?
As part of their efforts to increase high school graduation rates and support students’ college and career readiness, many states are moving away from traditional approaches to instruction and assessment that base student advancement on credits and “seat time.” Instead, they are adopting competency-based learning approaches that provide schools with the flexibility to link advancement to a student’s mastery of content. In the Northeast & Islands Region, Maine, New Hampshire, Rhode Island, and Vermont have all developed policies requiring school districts to make high school graduation contingent on a student’s mastery of required competencies. Districts in Connecticut, Massachusetts, and New York can choose to implement one or more elements of competency-based learning but are not required to do so.

“As proficiency-based learning is student-centered, we need a tool that gets us the feedback at the student level, at the core level of what its intent is, and this survey—we do feel—will give us that feedback,” said Andre Messier, principal of Lake Region Union High School in Vermont and an advisory committee member.

Given this regional momentum toward, and national interest in, competency-based learning—also referred to as proficiency-based learning—members of the Northeast College and Career Readiness Research Alliance selected this reform as one of two key areas of focus for its five-year research agenda. Yet, through literature reviews and research scans, alliance members quickly learned that few high schools or districts were systematically collecting information on how they were implementing competency-based learning and the impact of these approaches. As a result, the alliance specifically requested support in developing a student survey that could be used to collect information that would help policymakers and practitioners improve implementation and communication with students.

How Was the Survey Developed?
A REL Northeast & Islands study team—comprised of my colleague Sarah Ryan and me—worked closely with an advisory committee to develop the survey. The committee included six alliance members: two teacher-leaders from a Connecticut school district, a researcher from a Maine university, a high school teacher (also a former researcher) from Maine, a district administrator from New Hampshire, and a high school principal from Vermont. All advisory committee members worked in or with schools that were at various stages in the implementation of competency-based learning.

Ongoing collaboration between the advisory committee and our study team was critical to ensuring the survey items would elicit useful, valid, and reliable information about student beliefs about, exposure to, and understanding of the key elements of competency-based learning. We held several meetings with the advisory committee to generate the list of constructs the survey would measure. Next, we developed an initial pool of survey items aimed at providing complete coverage of the constructs, and then we refined the item pool based on the advisory committee’s feedback. This iterative process continued until our study team and advisory committee agreed that the survey items covered the constructs completely while also constituting a survey of reasonable length, which means the survey could be administered without overburdening the survey respondents or the school staff that would administer it.

What Data Does the Survey Collect?
The survey collects information on students’ beliefs about, exposure to, and understanding of key elements of competency-based learning. These elements include instructional practices that allow students to:

  • progress through demonstration of mastery
  • receive personalized instruction and learning opportunities
  • demonstrate mastery through flexible assessment
  • develop specific skills and dispositions that may be especially critical under competency-based learning

Because of wide variability in how schools and districts implement competency-based learning, the survey includes an introductory section followed by six separate modules. This modular approach allows users to select and administer the parts of the survey that are most relevant to their district’s or school’s approach to competency-based instruction and assessment.

What Supports Are Available to Administer the Survey?
In addition to the guide that accompanies the survey, REL Northeast & Islands researchers have created a series of three complementary guides that offer step-by-step instructions on collaborative survey development, sampling respondents, survey administration, and analysis and reporting of survey data. 

Are you interested in learning more about competency-based learning? I invite you to join Jessica Brett (Facilitator of the Northeast College and Career Readiness Research Alliance) and me for a special all-day event held at EDC headquarters in Waltham, Mass. on October 6: "Perspectives on the Current Landscape of Competency-Based Learning Research." This event is free, but advance registration is required. Head to the REL Northeast & Islands at EDC website to learn more and to register

Date: 

Monday, August 29, 2016 - 12:30pm