The Hydra Part 2

The Hydra Part 2

… or ‘Weikart and Scweinhart’s [Perry] High/Scope Preschool Curriculum comparison Study Through Age 23’ and Lifetime Effects: The HighScope Perry Preschool Study Through Age 40 (2005) For details of these studies see Part 1

This post begins with the story of two preschool approaches and their fates. One approach was ‘teacher led’ Direct Instruction and the other ‘child led’ High/Scope Perry preschool, already discussed at length in my previous post. As the NIFDI website explains:

There was an enormous educational experiment beginning in 1968 comparing these two approaches called Project Follow Through. It was the most extensive educational experiment ever conducted. Beginning in 1968 under the sponsorship of the American federal government, it was charged with determining the best way of teaching at-risk children from kindergarten through grade 3. Over 200,000 children in 178 communities were included in the study, and 22 different models of instruction were compared.

Evaluation of the project occurred in 1977, nine years after the project began. The results were strong and clear. Students who received Direct Instruction had significantly higher academic achievement than students in any of the other programs. They also had higher self esteem and self-confidence. No other program had results that approached the positive impact of Direct Instruction. Subsequent research found that the DI students continued to outperform their peers and were more likely to finish high school and pursue higher education. The Perry High/Scope approach is on the graph above. It is the ‘Cognitive Curriculum’ approach (second from the end). You can see it wasn’t quite as successful…

So which preschool method was the winner? The answer might seem obvious from the table above but you would be mistaken. The approach that now dominates is the High/Scope Perry preschool approach – and this is to some extent on the basis of two very small and problematic studies.

In my last blog I outlined the problems with these two small studies from 40-50 years ago by Schweinhart and Weikart that examined the impact of their Perry High/Scope preschool methods on the participants into adulthood. I began to explain the staggering influence over education policy these two studies have had. To find out the details of these two studies click back to my last blog but to give you the gist here is Kozloff’s summary some of the problems with the way these studies have been interpreted:

 What is “…just plain bizarre, is that [Schweinhart and Weikart] barely entertain the possibility that: (1) a dozen years of school experience; (2) area of residence; (3) family background; (4) the influence of gangs; and (5) differential economic opportunity, had anything to do with adolescent development and adult behavior.

In this post I will look at the impact of these studies on early years education.

First: These studies have been crucial in building a case for the importance of preschool education in the early years of childhood.

The National Audit Office commissioned a summary of the evidence on the impact of early years’ provision on young children with emphasis given to children from disadvantaged backgrounds in 2004. Such a paper offers a good review of the key research literature that has been influencing public policy.

The evidence of the biggest ever educational experiment “Project Follow Through” does not feature but both tiny Perry preschool studies feature heavily in the report. The second study of 123 subjects is one of six randomised controlled trials cited to provide evidence of the effectiveness of preschool programmes for disadvantaged children. These trials were all small scale and to an extent have contradictory findings. Some found reductions in antisocial behaviour but not academic gains and others had the opposite findings. None of the trials seemed to offer better evidence than the problematic second Perry study of 123 subjects.

The report then goes onto look at the evidence of preschool having an impact on the general population, rather than studies that only focus on those children that are highly disadvantaged. Perhaps it would be reasonable to argue that the majority of this research had positive findings, either for social development, academic development or both. However, there were still many contradictory findings and what seemed to be quite low effect sizes. Often, the preschool methods examined provide limited academic advantage but show positive social effects later in life. Having looked at the literature, I do begin to wonder if the commenter on this American website has a point:

“As the authors note, it is indeed quite a puzzle how pre-school education could possibly not show positive effects during schooling, yet have dramatically positive effects in adulthood. But if you look at the studies that find no effect during early schooling, you’ll find them very dense with objective facts such as testing results, etc. But if you look at those handful of studies purporting to show dramatic adulthood outcomes, you don’t find a lot of such data. In fact these latter studies aren’t scientific – they’re advocacy. They didn’t come up with rigorously selected criteria prior to pre-school to evaluate the outcomes, but instead retrospectively identified metrics to compare the control groups, leaving much room for post-hoc cherry-picking. The most parsimonious explanation of the paradox is that the adulthood-effects studies are flawed, and aren’t actually showing any real positive outcomes from pre-school.”

I’d need to do much more research to comment further. What is very interesting to me is that that there is no doubt the much publicised benefits of preschool education are built on shakier foundations than advocates would like policy makers to think. If you are interested in forming an opinion, this post , this and this post and this riposte are a great starting point.

There is a very concerning reliance on the Schweinhart and Weikart Perry preschool studies in the National Audit Office report.

1. The shockingly ‘dodgy’ first study (see my previous post) is relied upon to define ‘high quality’ child care.

I’ll explain further. One noticeable feature of the research on preschool effectiveness is the reliance on the idea that the reason some studies showed no effect was because the preschool programme was not ‘high quality’. On one level that is sensible as there must be huge variation in the quality of preschool provision but it is also a way of arguing that we should dismiss all studies with weak or no effects, presuming they are not ‘high quality’. I had noticed that the term ‘high quality’ is used frequently in the literature and repeatedly by policy makers and so I was interested to see how researchers had reached a decision on what constituted ‘high quality’. This is what the National Audit Office report had to say (I’ll highlight the key passage but thought I should include the full extract):

“In pre-school education (3+ years), quality is most often associated with the concept of developmentally appropriate practice. Bryant et al. (1994) report on several studies that illustrate the relationship between developmentally appropriate practice and child outcomes. The High/Scope study [Schweinhart’s and Weikart’s] shows that children who attend a developmentally appropriate, child-centred programme are better adjusted socially than similar children who attend a teacher-directed programme implementing a direct-instruction curriculum (Schweinhart, Weikart, and Larner 1986). In North Carolina, Bryant, Peisner-Feinberg, and Clifford (1993) found that children’s communication and language development were positively associated with appropriate care giving. Burts et al. (1992) and Hart & Todd (1995) show that children’s attendance in developmentally appropriate kindergartens is associated with fewer stress behaviours. Educational content is also important for this age group. Jowett & Sylva (1986) found nursery education graduates did better in primary school than playgroup graduates, suggesting the value of an educationally orientated pre-school. The research demonstrates that the following aspects of pre-school quality are most important for enhancing children’s development: Well-trained staff who are committed to their work with children, facilities that are safe and sanitary and accessible to parents, ratios and group sizes that allow staff to interact appropriately with children, supervision that maintains consistency, staff development that ensures continuity, stability and improving quality and a developmentally appropriate curriculum with educational content”

• Oh my goodness! How can the study referred to possibly support the weight being placed on it (see first half of my previous post)? The National Audit Office report writer considers it a central plank in research used to define ‘high quality’ child care when it is hopelessly flawed.

 

• Not only this, there is good research demonstrating the academic advantage of preschool methods with approaches which are contradictory approaches to the Perry preschool methods. Why does the definition of ‘high quality’ actually exclude these successful alternative methods? The Perry Preschool method endorsed ‘developmentally appropriate practice’ which means ‘child led’ experiential curriculum rather than teacher led instruction. The National Audit Office report actually mentions the success of a very teacher led approach, used widely in France:

“Studies of children in the French Ecoles Maternelle programme (Bergmann 1996) show that this programme enhances performance in the school system for children from all social classes and that the earlier the children entered the pre-school program, the better their outcome.”

The early results of project Follow Through (with 9000 children assigned to nine early childhood curricula) showed that disadvantaged children who received Direct Instruction (anathema to those advocating child led approaches) went from the 20th to about the 50th percentile on the Metropolitan Achievement Test. Children who received the Perry High/Scope curriculum did not do as well. They fell from the 20th percentile to the 11th percentile.

2. It is a concern that the second Schweinhart and Weikart study is used to demonstrate the cost effectiveness of preschool education in the National Audit Office report.

“The Perry Pre-school Project is the most cited study in this area and its benefits are well reported (e.g. Schweinhart et al. 1993). The cost-benefit analysis for this project (Barnett 1996) is worthy of some consideration as its findings are extensively used for justifying expenditure on pre-school education and care.”

This is troubling given the small size and context of the original study. It can’t possibly support these inferences.

Second: This research on impact of education in children’s early years has been used to justify our statutory Early Years Foundation Stage.

The EPPE study was an enormously significant longitudinal Study funded by the DfES from 1997 – 2004 and with findings in support of our ‘child led’ EYFS curriculum. It actually cites the first highly flawed Schweinhart and Weikart study where, among other problems, findings were the result of changes in outcomes of two or three people. This  is what it says of a study that should never have been taken seriously:

“Previous Research on the Effectiveness of Pre-School Education and Care: The vast majority of longitudinal research on early education has been carried out in the U.S.Two of the studies cited most often are the Abecedarian Project and the Perry Pre-school Programme (Ramey & Ramey, 1998; Schweinhart and Weikart, 1997). Both used randomised control trial methods to demonstrate the lasting effects of high quality early intervention. These landmark studies, begun in the 1970s, have been followed by further small scale ‘experiments’ (see the Early Headstart, Love et al., 2001) and larger cohort studies (See Brooks-Gunn, 2003; Melhuish, 2004a, for reviews). This huge body of literature points to the many positive effects of centre-based care and education.”

The EPPE refers to the first Perry study as having been ‘admired for decades for its internal validity.’ I am not sure the writer has actually looked at the study. Amusingly, the EPPE report writer seems bemused when their own findings contradict those of the Perry study:

“ It appears therefore that the beneficial impact of pre-school on cognitive attainment is more long lasting than that on social behaviour. Social/behavioural outcomes may be more influenced than cognitive outcomes by the primary school peer group. Still this finding is at odds with the Perry Pre-school study, which indicated that the social outcomes of pre-school were more salient than the cognitive ones by adolescence. Data from the EPPE continuation study, will follow children into adolescence, to shed light on this.”

The influence the Perry study has had on understanding of what might be meant by ‘high quality’ provision is quite concerning given these contradictory findings.  Our national EYFS curriculum presumes ‘high quality’ provision means:

“…all areas of learning [are] to be delivered through planned, purposeful play, with a balance of adult-led and child-initiated activities… Professionals should therefore adopt a flexible, fluid approach to teaching, based on the level of development of each child. Research also confirms that the quality of teaching is a key factor in a child’s learning in the early years. High quality teaching entails high levels of interaction, warmth, trust, creativity and sensitivity. Practitioners and children work together to clarify an idea, solve a problem, express opinions and develop narratives.”

However this is problematic:

1. We know the first flawed Schweinhart and Weikart Perry study contributed towards this idea of ‘high quality’ provision but it did not actually improve academic outcomes for its young participants. Evidence such as Follow Through (from the same era is) not even mentioned which did improve academic outcomes. The Perry programme was viewed as worthwhile because it was believed this intervention limited adult anti-social behaviour.

“The Perry program initially boosted IQs. However, this effect faded within a few years after the end of the two-year program, with no statistically significant effect remaining for males, and only a borderline significant effect remaining for females.

2. I believe good parenting makes a difference to children and so, although the studies seem unconvincing it is not outside the realms of possibility that the committed teachers involved in the second Schweinhart and Weikart study had some positive impact on their pupils. They were a highly committed team of extremely well qualified teachers. They must have involved themselves deeply in the lives of their very disadvantaged, very low IQ pupils, given they made 90 minutes visits every week to their homes as well as teaching them. Such a scheme is not really more widely replicable though.

Also, given the range of possible benefits the children experienced it is quite a leap to pin point the child-led learning as a crucial factor and suggest it provides a model of ‘high quality’ pre-schooling for all children today.

However some sort of  model is what the Perry preschool at Ypsilanti Michigan seems to be:

@websofsubstance Yes, and the evidence for the High/Scope Perry Pre-school in Ypsilanti, Michigan is even stronger…

— Dylan Wiliam (@dylanwiliam) August 12, 2014

3. The use of these studies to justify ‘developmentally appropriate’, ‘child led’ practices is concerning. If children are not highly disadvantaged or at any great risk of engaging in felonies (most children) it is hard to see how these studies can justify the use of such approaches. This is particularly given the success of other methods. [The results of project Follow Through (with 9000 children assigned to nine early childhood curricula) showed that all groups made greater academic gains than using the Perry High/Scope approaches.]

Some of the many subsequent studies on the effectiveness of preschool education record academic gains and others don’t. Some record social gains and others don’t. However, partly thanks to the Schweinhart and Weikart studies the importance of the ‘high quality’ preschool education is a mantra repeated by all politicians. The Perry approach has become a model for ‘high quality’ preschool education around the world and currently, about 30 percent of all Head Start centres in America offer a version of the Perry curriculum (ICPSR 2010). For those, like myself, concerned that ‘child-led’ approaches are not the most efficacious, the impact of these studies is an enormous cause for concern.

So why are two small, context dependent flawed studies still widely cited? Why are the results of the largest ever educational experiment from the same era ignored in early years research literature? There is only one possible explanation. The small studies said what educationalists wanted to hear, that a child led curriculum could be proved to effect life outcomes.  The enormous Project Follow Through had more uncomfortable findings – so it was ignored.

I examine the evidence base for what is deemed ‘high quality’ pre school education here: https://heatherfblog.wordpress.com/2015/07/09/a-truism-that-needs-questioning/?relatedposts_hit=1&relatedposts_origin=626&relatedposts_position=2