Whatever one’s position on their value, limitations and effects, school effectiveness measures play an important role in educational practice, systems and research. League tables, ‘expected progress’ measures, value-added models, threshold targets and OFSTED ratings are all salient and pervasive features of the current educational landscape. Taken individually or collectively, these measures, and many more besides, promise us the same tantalising benefit: an objective, fair and accurate standard by which we can compare and evaluate schools. In my research, I examine this claim and scrutinise the logic and claims of existing performance measures.
The central problem in the measurement of school effectiveness is creating estimates which are comparable across schools and ascertain how effective a school is for a given intake. Current value-added models control for intake characteristics using prior attainment data. Currently, this does not produce estimates which are either stable across several years or are independent of raw attainment scores. Therefore, value-added measures appear to largely reflect differences in intake across schools, fluctuations within intake characteristics for successive cohorts, measurement error and omitted variable bias. There is considerable scope for the production and/or refinement of measures which are fairer, broader and clearer from which more discerning data-usage can be practiced in policy and school decision-making.
My research can be thought of as having three broad components. Firstly, I gather official data from the National Pupil database and another Department for Education study known as ‘Making Good Progress’ (MGP). This is used to construct and compare several school effectiveness models. As well as seeking to verify previous research and further scrutinise existing models, this involves the estimation of a regression discontinuity model with the MGP dataset thereby providing an original research contribution using a relatively new approach to school effectiveness measurement.
Secondly, my research attempts to see whether the existing and nascent (i.e. regression discontinuity analysis) school effectiveness methodology can be extended to estimate school effects on attitudinal outcomes such as enjoyment of school and learning. To explore this, I am planning a primary data collection in secondary schools where I hope to gather attitudinal data along with pupil background data. Again, I hope to estimate a range of models to get some indication of how consistent these estimation techniques are.
Finally, the estimation techniques above and, especially in the case of the attitudinal data, are predicated on number of assumptions which I hope to explore. Even the fundamental notion of school effectiveness, on closer inspection, appears to be more problematic and sometimes contradictory than at first glance.
I think that there is an important debate to be had regarding how school performance measures and statistical techniques are used in the academic, public and political understanding of schools. My view is that educational professionals must become adept in using a broad range of meaningful, current and fair measures to foster ‘intelligent accountability’ and pursue aspects of educational improvement and research. For this to happen, an informed engagement with the problems and potential of performance measures is needed from all with a stake in education.