Alignment Study

Purpose:  The West Virginia Department of Education was required to conduct an external alignment study of the West Virginia curriculum standards and assessments in three content areas – language arts, mathematics, and science for seven grade levels-- grades 3, 4, 5, 6, 7, 8, 9, 10 and 11. 

The external alignment study addressed both quantitatively and qualitatively the comprehensiveness of tests in terms of gaps and weaknesses, emphasis and depth of knowledge as well as thinking skill distribution to ensure the alignment of content and performance descriptors with the assessment.

Background:  Alignment is frequently used to describe the state of an education system. For purposes of the study, alignment is defined as the degree which standards and assessments in education systems are in agreement and serve in conjunction with one another to guide the system toward students learning what they are expected to know and to do.

The alignment process produced the following information:

  • Degree the standards and assessments are in agreement on four criteria: categorical concurrence, depth-of-knowledge consistency, range-of-knowledge correspondence, and balance of representations
  • Item coding of depth-of-knowledge and standards by individual reviewers
  • A report on the reliability among reviewers
  • Explicit statement if standards and assessments are aligned or not aligned and what action was needed to make the standards and assessments aligned

The first phase of the study was conducted in September 17-19, 2008 using forms from the WESTEST 2 Field Test.  Dr. Norman Webb of the Wisconsin Center for Education Research headed the study.   Twelve nationally recognized content specialists (four per content area) provided expertise as external reviewers.  Twelve West Virginia (four per content area) educators provided expertise, understanding, and knowledge of West Virginia’s Content Standards and Objectives and Performance Descriptors.  The internal reviewers represented retired teachers, county curriculum coordinators, and classroom teachers from all grade spans across the state.

The second phase of the study was conducted January 26-29, 2009, using internal and external reviewers for each content area.  The reviewers used the WESTEST 2 Operational Forms for the second phase of the study.  A final report will be submitted to the United States Department of Education.

Impact:  The Phase I report narrative interpreted the data and results of the study and was used to strengthen the alignment of the operational forms of the test. The report discussed the quality of the items based on any identification of source of challenge and other comments made by the reviewers. The study identified issues and recommended actions needed to reach alignment.  The Phase II study was used to verify that the standards and the assessments are in alignment and meet Title I requirements.