The EPPSE is a very large and enormously influential study commissioned by the DfE to find out what types of preschool provision and early experiences are most effective. It followed 3000+ children from the age of 3 to 16 years and reaches some very significant conclusions which I would question.
The research team was from the Institute of Education, Birkbeck and Oxford and as the Institute of Education blog explains:
The EPPSE project… has become one of the highest impact educational research programmes in Europe… EPPSE’s findings underpin billions in Government spending on nursery expansion, including the Sure Start programme, the extension of free pre-school to all three and four-year-olds in 2010 and this year, to the poorest 40% of two-year-olds this year… EPPSE’s evidence documenting excellent pre-school education and its ongoing benefits, especially for the most deprived children, has fed heavily into England’s early childhood curriculum and informed curricula in countries as diverse as Australia, China and Brazil. Nursery World editor Liz Roberts has noted “how highly regarded the Early Years Foundation Stage is around the world”.
The EPPSE project findings are stunning:
Attending any pre-school, compared to none, predicted higher total GCSE scores, higher grades in GCSE English and maths, and the likelihood of achieving 5 or more GCSEs at grade A*-C. The more months students had spent in pre-school, the greater the impact on total GCSE scores and grades in English and maths… the equivalent of getting seven B grades at GCSE, rather than seven C grades.
The EPPSE project also found that:
There was some evidence of statistically significant continuing pre-school effects on social behavioural outcomes at age 16 but these were weaker than at younger ages. Having attended a high quality pre-school predicted better social-behavioural outcomes in the longer term, though the effects were small.
The IOE blog gushes that EPPSE:
…brought together a rare combination: research funded by Government with a genuinely open mind, carried out by excellent and dedicated academics savvy enough to work with and influence politicians of all stripes…And thanks to the detailed work that began 17 years ago at the IOE, we also know what excellent nursery provision looks like.
Hold on a moment! It seems these researchers did not have an open mind. I have previously blogged about the fact the EPPE (the acronym before the study moved onto secondary school outcomes) studied quality using a measure, ECERS R, which had a predefined scale based on prejudged measures of quality. Having read through much of the voluminous literature there is so much I could discuss about the EPPE findings but in this post I will focus on another claim in the IOE blog, that this study has rigour. I am really not sure it does.
That is a big accusation to make against such a large and influential study conducted by highly regarded academics but it seems to have a fundamental problem that strikes at the heart of the validity of its findings.
The problem of the EPPE/EPPSE control group.
In their report at the end of KS1 (when the study children were 7 years old) the researchers acknowledge that there were problems with the control group. Because in England the vast majority of children attend a preschool it was not possible to find a representative sample of those children who didn’t:
The ‘home’ control group are from significantly disadvantaged backgrounds when compared with the sample as a whole with most mothers having less than a GCSE qualification (p11) .
On p28 the report explains that:
…comparison of the ‘home’ sample (the control who did not attend pre-school) with children who attended a pre-school centre showed that both the characteristics and attainments of home children vary significantly from those who had been in pre-school. It is not possible to conclude with any certainty that the much lower attainments of the ‘home’ group are directly due to lack of pre-school experience.’
The writers go on to talk positively about how they have used ‘contextualised multilevel analysis’ to try and compensate for the unrepresentative nature of the control sample and they feel this means their results are worth considering but, for example, they admit that when making judgements about the impact of longer duration of pre-schooling on higher cognitive outcomes by comparing with the ‘home’ group:
“causal connections cannot be drawn”
The problems with the control are not mentioned in the overall findings but in 2004 they are acknowledged in the body of the report. There is an attempt in 2004 to show that variation in pre-school quality and in duration of attendance can have an impact and this is because these findings don’t have to use the problematic control.
It seems obvious that children coming from very disadvantaged homes as the ‘home’ control group largely do, may well benefit from pre-school in ways most children wouldn’t. Therefore, despite contextualised multilevel analysis to take account of all other variables such as SES there will always be problems using this control to reach firm conclusions on the impact of pre-schooling on the whole population.
Fast forward to 2014 and the final reports in the children at age 16. I have looked through all the reports. It is clear from the tables included that all the startlingly good educational findings rest on comparisons with this control group. However I can find NO MENTION AT ALL of the sorts of problems with the control the researchers were willing to acknowledge in 2004. It is as if the control group issue just wafted away. It seems that such a large and important study, ‘one of the highest impact educational research programmes in Europe’, had no need to concern itself with pesky issues like that annoyingly poor control, that the whole vast edifice that is EPPSE rests upon.
How can it be that in 2004 the problem of the unrepresentative control meant many findings on the impact of preschool were tentative but in 2014 the issue of the control has totally disappeared? It is as if it never existed. Given the startlingly strong impact ANY form of preschool is claimed to have the findings need to be robust if they are to be used to make policy.
The EPPSE claims to be ‘proper’ research, not the sort of stuff that gives education research a bad name. It is also enormously influential and directly used in government policy making. I can understand the researchers, invested as they are, claiming their conclusions have validity but where is the scrutiny? What is happening at peer review? If anything highlights the unhealthiness of the rigid orthodoxy in education departments, especially in early years research, it is this EPPSE study. Once again no one seems to question what they want to believe.
If you found this interesting you may also want to read these posts:
https://heatherfblog.wordpress.com/2015/07/09/a-truism-that-needs-questioning/
Esse Quam Videri 
Reblogged this on The Echo Chamber.
The Institute of Education is the home of Reading Recovery in England – I have a problem, therefore, with any of the research coming from the IOE! The entrenchment of RR at the IOE beggars belief and, in my view, diminishes the credibility of the IOE.
Was any of the work peer reviewed? Government reports not noted for their impartiality.
Looks like some seriously bad maths going on there. I am pretty shocked at the very poor handle on basic statistics these education researchers seem to have. You try and point it out and get sat on for ‘not being an experienced teacher’.
It’s like these people think they’re above the normal checks and controls, above statistical analyses.
Yes, I can’t get over the way problems that were considered significant in the early years of he study just disappear later. Normal checks and controls apply whatever your status and that of your study.
Just as the early childhood boosters have started to abandon the practice of using Perry as their sword and shield (I guess they realised how ridiculous they were looking, using a 50 year old study) they have replaced it with EPPSE, which not only has the benefit of being contemporary but also being nominally ‘universal’. Governments and the agencies tasked with ferreting out the bad research have missed it. How sad.