Office of Research
Main Content

 

RESEARCH OFFICE BANNER

What's New

Report details the development of the West Virginia School Climate Index—a measure of school engagement, safety, and environment
(April 2012)

The WV School Climate Index is a multidimensional measure developed by the WVDE Office of Research and Office of Healthy Schools as part of the Safe and Supportive Schools grant program (S3). The index was developed in alignment with a model put forth by the U.S. Department of Education, Office of Safe and Drug Free Schools (OSDFS). The OSDFS based its model on a synthesis of available research and expert and stakeholder opinion, which point to school climate consisting of three primary domains, each consisting of corresponding subdomains (in italics) as follows:

  • Engagement—the quality of relationships, including respect for diversity, among students, staff and families; the level of school participation and involvement by families, staff, and students in school activities; and efforts by schools to connect with the larger community.
  • Safety—the physical and emotional security of the school setting and school-related activities as perceived, experienced, and created by students, staff, families, and the community. The use and trade of illicit substances in the school setting and during school-related activities also is included in this domain.
  • Environment—the physical and mental health supports available that promote student wellness, the physical condition of school facilities, the academic environment, and the disciplinary tone of the school—i.e., the fairness and adequacy of disciplinary procedures.

The WV School Climate Index—derived from 20 indicators drawing from student and staff survey data and selected discipline incident data reported in the West Virginia Education Information System (WVEIS)—provides (a) an overall measure of school climate that measures all domains and subdomains in the OSDFS model; (b) a straightforward, easily understood scale that can be readily interpreted by those working to improve school climate; and (c) information about component parts of the index to help schools rapidly identify areas for programmatic interventions. The measure is a requirement of the S3 program, which awarded WVDE a grant in 2010.
To date, the WV School Climate Index has been deployed in 42 high schools in 18 counties that are part of West Virginia’s S3 grant program. This Office of Research publication fulfills West Virginia’s obligation to publicly document the composition of the index.
For more information about the WV School Climate Index, contact Dr. Andy Whisman, Coordinator, Office of Research (swhisman@access.k12.wv.us), or download the full report, The West Virginia School Climate Index: A Measure of School Engagement, Safety, and Environment, available at the following URL:http://wvde.state.wv.us/research/whats_new.html.


New index shows the impact of school climate on behavioral and academic outcomes in West Virginia schools
(January 2012)

The term school climate refers to the character and quality of school life. The validity of a new index designed to measure school climate—the WV School Climate Index—was tested in this study, and it was used to show the impact of school climate in West Virginia schools.
Validity of the index. The Index was developed in alignment with a model for school climate measurement put forth by the U.S. Department of Education, Office of Safe and Healthy Students.  The Index was tested based on the assumptions that a valid measure should (a) differentiate between favorable and unfavorable climate conditions and (b) based on other research, be correlated with and predictive of academic outcomes. Evidence of the Index’s ability to differentiate climate conditions was provided by School Climate Specialists working in intervention schools, who reported that the Index reflected conditions they had observed. Further, statistically significant differences in Index scores were found between intervention and nonintervention schools. The Index also was shown to correlate at moderate to moderately strong levels with school-level proficiency rates in four content areas and median growth percentiles for mathematics and reading/language arts (RLA)—accounting for noteworthy proportions of variation in these measures. 
Demonstrated relationship of school climate with student outcomes. Factors such as high poverty rates, large proportion of students with disabilities, larger school size, and certain grade-span configurations of schools are associated with poorer academic outcomes. Even when these conditions were present, this study showed the positive effect of school climate remained strong for four of six academic outcome measures tested. School climate was the most influential predictor in the social studies proficiency and mathematics growth percentile regression models, and was the second and third most influential predictor for RLA proficiency and growth percentile. Further, the study showed positive school climate substantially moderated the effect of poverty as well as the other factors included in the model. For social studies proficiency and mathematics growth percentile, the effects of poverty were entirely moderated by school climate. With all measures considered together, positive school climate lessened the cumulative negative impact of poverty, disability rate, school size, and grade-span configuration from 6% to 100%.
Conclusions. Schools have virtually no control of the demographic characteristics of the students and communities they serve, and decisions about school size and grade-span configuration reside at much higher political and administrative levels. The results reported in this study suggest that by addressing a factor that is within their sphere of influence—improving school climate—schools may substantially diminish the unfavorable effects of matters over which they have little control. Accordingly, schools should focus their improvement efforts on the needs of their students and staff as they relate to school climate. The WV School Climate Index can help schools identify areas of needed improvement and measure their progress.
Learn more. For more information about the WV School Climate Index, contact Dr. Andy Whisman, Coordinator, Office of Research (swhisman@access.k12.wv.us), or download the full report, The West Virginia School Climate Index: Validity and Associations With Academic Outcome Measures, available at the following URL:http://wvde.state.wv.us/research/whats_new.html.

 


Study shows the West Virginia Board of Education’s 2010-2011 professional development master plan to have been well implemented, overall
(July 2011)

By West Virginia statute, the West Virginia Board of Education is required each year to develop goals for professional development and to work with statewide professional development (PD) providers to develop, implement, and evaluate a master plan for statewide professional development. A multimethod exploratory research design was used to examine the implementation of the 2010-2011 PD Master Plan, including participant perceptions about its adherence to national standards for high quality professional development, its coverage of the goals set forth by the WV Board of Education, and its effect on participants’ knowledge, behaviors, and attitudes. All 18 PD providers included in the 2010-2011 PD Master Plan supplied the Office of Research with detailed reports of their activities and 1,186 (69%) of educators in a random sample of participants in professional development offered in conjunction with the plan responded. Results of the study showed that collectively, PD providers followed through by providing 195 of 244 (about 80%) of the professional development sessions they planned and had approved as part of the PD Master Plan. All three goals for professional development appear to have been well covered by the PD providers according to their reports (a fourth goal called for evaluation of the plan, which this study fulfilled). Attendance was high overall, with 37,107 participants attending sessions associated with the PD Master Plan. The top three providers of professional development in 2010–2011, in terms of attendance, were RESA 2 (6,164), Marshall University June Harless Center (5,119), and RESA 3 (3,472).Overall, participants indicated positive agreement that all of the attributes associated with high quality professional development were present in the sessions they attended—i.e., they reported it was (a) intensive in nature, (b) specific and content focused, (c) relevant, (d) hands-on with active learning opportunities, (e) supported by follow-up discussion or collaboration, and (f) supported with follow-up professional development. Of respondents known to have attended sessions focused on a particular goal, most had favorable views of the session’s success in addressing that goal. One component of the survey asked participants what they knew, did, or believed about the subject matter of the session they attended before they attended and compare it with what they knew, did, or believed after the session. Overall, the professional development was perceived as having the strongest impact on increasing knowledge of the content areas addressed in their sessions, followed by slightly less strong changes in behavior, and with generally moderate perceived effects on attitudes and beliefs.
From Implementation of the Master Plan for Statewide Professional Staff Development for 2010–2011: An evaluation study by Patricia Cahape Hammer and Nate Hixson, 93 pages. An executive summary and PowerPoint presentation are also available.


Participants indicate strong support for the Transition to Teaching program
(February 2011)

The Transition to Teaching program in West Virginia provides individuals who secure full-time teaching positions with benefits that include online instruction, extensive mentoring and coaching, a laptop computer, and other supports during the course of the participants’ 3-year commitment, culminating in teacher certification. A survey collected responses from 43 participants in the program, which represented a 61% response rate. These respondents overwhelmingly showed agreement or strong agreement that the TtT program had helped them transition from other careers into the teaching profession, especially in the critical areas of middle school and high school science, mathematics, and special education teaching.
From Results of the Transition to Teaching Teachers’ Capacity Survey, 2010-2011 by Juan D’Brot, Patricia Cahape Hammer, and Doug Cipoletti, 52 pages


Brief review of the literature on the use of labels in special education now available
(February 2011)

Labels seem to function in both negative and positive ways in education. Early evidence showed that knowing a child’s label—especially the labels of mentally retarded, emotional/ behavioral disability, and  learning disability—affected teacher perceptions and expectations for success. Other research showed that only certain labels (i.e., emotionally disturbed) influenced teachers’ expectations for student success, and that teachers may be more influenced by student behavior, such as a sample of student work. In the case of dyslexia, labels often produced a mitigating effect by providing an acceptable explanation for a student’s inability to read or spell, instead of others considering them lazy or intellectually disabled. Other positive effects of labeling described in the literature included helping parents by validating that there was a problem and enabling them to access services for their child; and by providing a medical explanation for a problem, which can serve as a welcome alternative to attributing undesirable behaviors to poor parenting. Expectations held by nondisabled students about their peers with disabilities were primarily influenced by the behavior of those students or by their placement in resource rooms versus special classrooms. Students with disabilities experienced both felt and enacted stigma and there were many reports of being teased, ridiculed, and bullied. Lastly, both students and researchers point out the need to address issues of school culture, especially stigma and mistreatment of students with disabilities by their nondisabled peers.
From Review of the Literature on the Use of Labels in Special Education, by Patricia Cahape Hammer, 9 pages.


Writing Roadmap research reveals positive results, but also identifies gender gap in writing achievement for male students
(January 2011)

The WVDE Office of Research recently completed a study to determine the amount of variance in students’ writing scores that is accounted for by Writing Roadmap usage. The study was specifically focused on determining the contribution of Writing Roadmap practice after controlling for students’ prior academic performance and other characteristics. The population for this research study included all students enrolled in Grades 4 through 11, who completed WESTEST 2 during the 2009–2010 academic year. A sample of approximately 5% of the population was randomly selected for statistical analysis, or 8,577 students. This study indicated a modest but statistically significant and positive relationship between completing Writing Roadmap exercises and students’ subsequent composite writing scores. This relationship was significant even after controlling for students’ prior academic achievement and gender, and held true for all grades with the exception of Grade 10 (it is unclear why the relationship was not present in Grade 10). While these findings supported the study hypothesis, the findings also showed that the most accurate predictor of students’ current performance is definitely their prior performance. Among other findings, it is notable that the study revealed a statistically significant difference between male and female students’ writing scores. In this study, male students’ composite writing scores were lower than female students’. Based on the findings, the Office of Research recommended (a) continued use of Writing Roadmap as appropriate along with additional complementary supports (the data do not indicate that simply increasing the number of Writing Roadmap assignments alone can be expected to lead to very large positive effects); (b) continued monitoring and evaluation to determine the impact of the customized version of Writing Roadmap, WV Writes; and (c) additional support for RLA-focused instructional interventions for male students to help close the achievement gap with female students.
From Writing Roadmap Usage and Additional Predictors of WESTEST 2 Online Writing Scores by Larry White, Nate Hixson, and Andy Whisman, 32 pages


Closing the Achievement Gap Professional Development Schools program shows promise, especially for African American students
(December 2010)

A study examined the impact of the CAG schoolsprogram on the academic achievement of students in 30 schools serving high percentages of African American and economically disadvantaged students in West Virginia, 2004–2009. Analysis of achievement testing showed promise for the program, especially for African American students, with the achievement gap narrowing compared with the state as a whole, in reading/language arts and mathematics at the elementary and middle school levels. The program assigned experienced educators, CAG liaisons, to the schools to provide school improvement coaching. Among other findings, the Office of Research staff reported that when comparing proficiency rates across all grades, the CAG schools had cumulative proficiency level increases that exceeded statewide increases for the African American, economically disadvantaged, and students with disabilities subgroups, and for the all students group. This finding held for both reading/language arts and mathematics. In some cases, differences in gains were notable. For example, in reading/language arts, the percentage of African American students who reached proficiency increased by 13.4% compared with the statewide increase of 8%. Students with disabilities showed even higher gains—14.6% compared with the statewide 8%. And while students statewide made strong gains in proficiency in mathematics, CAG school students made even higher gains—13.2% compared with the statewide gain of 7.5%. And here again, African American students and students with disabilities at CAG schools made greater gains than their peers, achieving gains of 17.9% and 19.6% respectively. Results of the focus groups and surveys provide information about which school improvement efforts received the most attention, and identified areas of strength and needed improvement in CAG schools at the end of the 5-year program.
From Examining the Effectiveness of Closing the Achievement Gap Professional Development Demonstration Schools by Larry White, Nate Hixson, Patricia Cahape Hammer, Diana L Smith, and Juan D‘Brot, 151 pages.


Study reveals challenges facing Title I Supplemental Education Service providers
(December 2010)

This study examined Title I Supplemental Education Service (SES) provider effectiveness by analyzing (a) achievement outcomes of students who received SES and (b) the perceptions of key stakeholders in participating school districts in West Virginia.
During the 2009-2010 school year most students served by providers did not meet or exceed the median percentage proficiency in mathematics and/or reading/LA for low-income students across four comparison groups. The results should be interpreted with caution due to the descriptive nature of the study. Although reasonably specified comparison groups were identified, with such small numbers of SES students represented across providers a fully adequate statistical comparison is not possible. Students attended SES services an average of 35.44 hours. While this number of hours is much higher than that reported by providers in the previous academic year, when spread over the course of a school year, it begs the question as to whether dramatic improvements in proficiency should be expected.
SES providers serving students in West Virginia during the 2009-2010 school year received predominately positive feedback from most respondent groups. District coordinators, principals/site coordinators, and teachers who participated in the evaluation were pleased with provider services overall. Providers, too, were primarily positive regarding their experiences with SES in West Virginia during the 2009-2010 school year.
While the findings of this study revealed overall satisfaction with SES implementation and services, respondent stakeholder groups did note areas for improvement. The primary areas for program improvement as identified by district coordinators, principals/site coordinators, and teachers alike were to increase the frequency with which providers (a) communicated with school personnel and (b) collaborated to set goals for student growth.
From Supplemental educational services in the State of West Virginia: Evaluation report for 2009–2010, by Andy Whisman, Juan D’Brot, Nate Hixson, and Larry White, 60 pages.