WESTEST 2 Item Development
The West Virginia Department of Education, Office of Assessment, Accountability and Research content assessment coordinators collaborated with the testing vendor, CTB-McGraw–Hill (CTB), to develop a blueprint for WESTEST 2. This blueprint included the list of CSOs per standard, per grade level and per content area that were able to be assessed by the statewide summative assessment, WESTEST 2. The blueprint reflects the alignment of the depth-of-knowledge (DOK) of items for the assessment.
The Office of Assessment, Accountability and Research selected two (2) educators per grade level to serve on each content area committee. All participants from the four content areas were trained as a group to develop universally designed items by the testing vendor. The large group was then divided according to content area for additional training on item development. The Office of Assessment, Accountability and Research and CTB continued the training, concentrating on the specific content area. This individual group training included the West Virginia 21st Century Content Standards and Objectives (CSOs) and the depth-of-knowledge levels as defined by Dr. Norman Webb. In each content area, the 21st Century Item Writing Checklist and sensitivity guidelines were explained and used during item development.
At the completion of each item writing workshop, item writers completed a mock peer review by checking items for correct alignment to the CSOs and DOK levels according to blueprint specifications. Items were also reviewed for 21st century context, relevance, rigor as well as the absence of biases. At the conclusion of these workshops, items were developed and ready for review by other stakeholder groups.
During the next step in the development process, a Content Review Committee comprised of West Virginia educators, CTB and WestEd representatives checked each item for CSO and DOK alignment. The items were then reviewed by a Bias Review Committee comprised of business and community leaders, West Virginia Department of Education staff, CTB and WestEd representatives. Suggestions from these committees were taken into consideration when preparing items for selection of WESTEST 2 Field Test forms.
In conjunction with the content and bias reviews, Dr. Norman Webb led an internal and external alignment study. Each item in each content area was evaluated for CSO and DOK alignment. Suggestions from his report were also used to prepare the WESTEST 2 Field Test forms.
A Pre-Alignment Study (Mathematics, Reading/Language Arts, Science)was conducted in September 2008. This study again consisted of internal and external reviewers under the direction of Dr. Norman Webb. The findings from this study were used in conjunction with field test statistics to prepare items for the Operation and Breach Forms for WESTEST 2 which is administered in May.
The West Virginia Department of Education received a report on the results of the WESTEST 2 Field Test derived from CTB's research department. These results contained the p-values, point biserial values, pardux and bias issues for each field tested item in each content area. All of this information, in conjunction with a CSO Pre-Alignment Study conducted by Dr. Norman Webb and lead by the Office of Instruction with support from the Office of Assessment, Accountability and Research and the content Develop Supervisors of CTB, was used to select items for Operational and Breach forms for WESTEST 2.