Skip Navigation
Office of Research
Main Content

Assessment/Accountability

2012_reporst_tab Teaching and Learning School Improvement Climate and Culture Professional Development Educational Supports Assessment and Accountability Other Reports

Findings from the 2012 West Virginia Online Writing Scoring Comparability Study   download PDF LinkResearch Brief download PDF Link
Report
Nate Hixson and Vaughn Rhudy, September 2013
Student responses to the WESTEST 2 Online Writing Assessment are scored by a computer-scoring engine. The scoring method is not widely understood among educators, and there exists a misperception that it is not comparable to hand scoring. To address these issues, the West Virginia Department of Education (WVDE) conducts an annual scoring comparability study that compares scoring by trained human raters to scoring by the computer engine.

Findings from the 2011 West Virginia Online Writing Scoring Comparability Study download PDF LinkResearch Brief download PDF LinkReport
Nate Hixson and Vaughn Rhudy, October 2012
To provide an opportunity for teachers to better understand the automated scoring process used by the state of West Virginia on our annual WESTEST 2 Online Writing Assessment, the WVDE Office of Assessment and Accountability and the Office of Research conduct an annual comparability study. Each year educators from throughout West Virginia receive training from the Office of Assessment and Accountability and then hand score randomly selected student compositions. The educators’ hand scores are then compared to the operational computer (engine) scores, and the comparability of the two scoring methods is examined.

Writing Roadmap Usage and Additional Predictors of WESTEST 2 Online Writing Scores download PDF LinkResearch Brief download PDF LinkReport
Larry J. White, Nate Hixson, and Andy Whisman, February 2011
Writing Roadmap™ is an online essay scoring tool developed by CTB McGraw-Hill that provides students in Grades 3-12 with the opportunity to complete an unlimited number of practice sessions in which they write essays in response to a variety of prompts. It is used, among other purposes, to help students practice and build confidence in their writing ability prior to the completion of the online writing session of the statewide summative assessment, WESTEST 2. An earlier study found that, in all cases, students who had completed five or more Writing Roadmap exercises exhibited higher composite writing scores than those who did not complete any assignments. The purpose of this second study is to determine the amount of variance accounted for by the Writing Roadmap usage variable beyond that which is accounted for by students’ prior academic performance and several other variables.

An Introduction to the West Virginia Growth Model   download PDF LinkReport
Nate Hixson, Juan D'Brot, and Damian Betebenner, December 2010
The purpose of this document is to define a method for calculating student growth in West Virginia using an established and statistically defensible technique. Further, the paper describes the potential utility of growth data in several contexts including school improvement, determining educator effectiveness and school accountability. Finally, this document includes a preliminary time line for the completion of key milestones during the West Virginia Growth Model development and implementation process.

Intended and Unintended Uses and Consequences of West Virginias Assessment and Accountability System (Part I Report)   download PDF LinkReport
Larry White, Nate Hixson, Juan DBrot, and Jason Perdue, 2010
In the context of the statewide accountability system, construct validity refers to the extent to which the state’s accountability assessment accurately and consistently measures what it purports to measure. Consequential validity refers to the extent to which the accountability system produces intended and/or unintended outcomes or consequences. Intended as the first in a series of studies, this report focuses on two research questions: (RQ1) What are the Type I and Type II identification error rates for the 2008-2009 school year in West Virginia?; and (RQ2) What do stakeholders believe are the intended and unintended outcomes and consequences of the assessment and accountability system?

Research Brief: Impact of Writing Roadmap 2.0 on WESTEST 2 Online Writing Assessment Scores download PDF LinkResearch Brief  
Larry White, Nate Hixson, Juan DBrot, Jason Perdue, Sandra Foster, and Vaughn Rhudy, 2010
In November 2009, the West Virginia Department of Education Office of Assessment, Accountability, and Research conducted an internal research study to determine the impact of Writing Roadmap 2.0 on West Virginia students’ 2009 WESTEST 2 Online Writing assessment scores. This research study utilized extant data to address a single research question: Is there a statistically significant difference between the 2009 WESTEST 2 Online Writing Assessment scores of West Virginia students who used Writing Roadmap 2.0 during the 2008-2009 school year and those who did not?

Research Brief: WESTEST 2 Online Writing Scoring Comparability Study download PDF LinkResearch Brief  
Larry White, Nate Hixson, and Vaughn Rhudy, 2010
To provide an opportunity for teachers to better understand the automated scoring process used by the state of West Virginia on our annual WESTEST 2 Online Writing Assessment, the WVDE Office of Assessment and Accountability and the Office of Research conduct an annual comparability study. Each year educators from throughout West Virginia receive training from the Office of Assessment and Accountability and then hand score randomly selected student compositions. The educators’ hand scores are then compared to the operational computer (engine) scores, and the comparability of the two scoring methods is examined.

Exploratory Research Brief on Student Achievement and Referral Rates in Response to Intervention Demonstration Sites   download PDF LinkReport
Larry White, Nate Hixson, Lanai Jennings, and Jason Perdue, 2009
The purpose of this examination was twofold: (a) to determine the initial impact of the Response to Intervention (RTI) program on the WESTEST 2 Reading/Language Arts achievement of students enrolled in RTI demonstration schools; and (b) to examine the referral rates for RTI demonstration schools relative to K-5 comparison districts.