Process of Form Selection for WESTEST 2
In December of 2008, the West Virginia Department of Education's Offices of Accountability/ Assessment and Instruction met with the CTB Program Manager of WESTEST 2, CTB's research and publishing departments and CTB content specialists. The purpose of the meeting was to review preliminary test form selections for the 2009 and 2010 administration and ensure all final test forms were compliant with research specifications.
The West Virginia Department of Education's content specialists worked with CTB's research specialists and content experts over a five day span to select items according to research specifications. They followed the blueprint for points and percentages, with specific attention to creating comparable forms. The viable statistics, P-Values (percentage of students who answered item correctly) and Point Biserials (correlation of the quality of the item to the relationship between student performance and item difficulty) and the Fit (does the item fit the statistical mode) were considered in the selection of each item.
In every content and grade level, the items were processed into four stacks: common items, 2009 unique items, 2010 unique items and unused items. Each content group began reviewing the third grade. Each item in the common set was read and reviewed and flagged for possible discussion/replacement. The diversity of content for items in the same standard was considered. This process was repeated for the unique set of items for both 2009 and 2010 administration sets. All items flagged from the common and unique sets were considered next and if the replacement of one of these items was decided, the set of "unused" items was referenced. If a replacement item was inserted, the new selection was submitted to CTB Research Department for approval. The final step in this process was to have CTB Research Department check the overall viability of the proposed form selection to ensure the form met all research specifications.
The Reading Language Arts and Social Studies assessments are passage/stimuli dependent. Reading Language Arts items are based around themed passages and the Social Studies items are based around a set of documents from which items are written. Mathematics and Science have one gridded response item per form. Mathematics, Social Studies and Science have two graphic organizers per form. Reading Language Art is administered in three sessions, with the first session being the WESTEST 2 Online Writing session that is administered separately from the WESTEST 2 Reading/Language Arts assessment during the WESTEST 2 testing window. All of the other content areas are administered in two sessions during the WESTEST 2 testing window.
Form selection worksheets were used to ensure parallel forms. Item cards were used to merge item images, data, and attributes (shows standard, objective, grade level, item type, score points and key). Specification documents showing p-values, point biserials, biases and fit served as technical roadmaps for item selection. All of these elements worked simultaneously to provide a balanced WESTEST 2 Assessment.