Confidence intervals, missing data and imputation: a salutary illustration

Stephen Gorard
School of Education, Durham University
This paper confirms that confidence intervals are not a generally useful measure or estimate of anything in practice. CIs are recursive in definition and reversed in logic, meaning that they are widely misunderstood. Perhaps most importantly, they should not be used with cases that do not form a complete and true random sample from a known population – the latter is a key premise underlying their calculation. This means that, whatever their merits, CIs should not be used in the vast majority of real-life social science analyses. The second part of the paper illustrates the dangers of ignoring this
premise, perhaps on some purported pragmatic grounds. Using 100 simulations of a sample of 100 integers from a uniform population with members in the range 0 to 9, it shows that CIs are very misleading as soon as there is deviation from randomness. For example, when 5% of the cases in each sample are deleted a reported 95% CI would be no better than a 66% CI in reality. If 10% of the lowest score cases are replaced with the achieved mean for the sample, then a reported 95% CI would be more like a 43% CI in reality. In addition, the simulation shows that the mean and standard deviation for any sample are correlated (an issue of linked scale). This illustrates that using the sample standard deviation as an estimate for the SD of the sampling distribution in order to try and assess whether the sample mean is close to the mean of the sampling distribution will simply make matters worse. The best and only available estimate of the sampling
distribution mean, in practice, is the sample mean

Click here for Complete article on CI,missing data and imputations

Gove education plan is a curate’s egg few will want to eat: 

Nothing official has been said, but there have been enoughleaks, previews and hints about Michael Gove’s vision for an overhaul of the GCSE exam system that all sides have a pretty good idea of what he has in mind and we’ve already got dissent coming in from the Labour-led Welsh Assembly government. (Please click here for the complete article)

Causation: The unithinking causalism: A claim that something acts as a cause of something else is a very strong one. This is partly because…. (Please click here for  the complete article)

Probability: Marbles! Confusion over significance testing… (Please click here for the complete article)

Running a randomised controlled trial:  Running a randomised controlled trial (RCT) for an educational innovation involves considerable planning at the outset, with care and attention throughout. (Please click here for the complete article)

Mixed Methods in Educational Research (Report from the March Seminar 2012, Norway): What is usually called ‘mixed methods’ research in education is really just research in education. It is relatively easy to conduct, with many possibilities and few real-life challenges or barriers. What this paper tries to do is convey part of why this is so… (Please click here for the complete article)

Stand Out and Be Counted: A guide to maximising your prospects. How quantitative skills have helped people in jobs and making a their successful career. Please click here to see the complete document.