The Hydra

‘The Hydra’

or ‘Weikart and Scweinhart’s [Perry] High/Scope Preschool Curriculum comparison Study Through Age 23’ and Lifetime Effects: The HighScope Perry Preschool Study Through Age 40 (2005)

A few years ago I read a serious book on early reading instruction that stated, without question, that there was strong scientific evidence that direct instruction methods, used even at nursery level, lead to increased criminality among adults. It seemed implausible. How could the teaching style of your nursery school, in some cases over only one year, have such a significant impact that it could be measurable 20 years later? However, I then came across the claim in numerous respectable publications. Greg Ashman  drew my attention to another in Psychology Today recently. Initially I felt I had to accept the claims – but should just investigate properly first. I soon discovered that all these claims pretty much originate

‘… from about half a dozen articles in the series written by Schweinhart and Weikart, in which they claim to compare (and apparently believe that they demonstrate the superiority of) their High/Scope pre-school curriculum with (1) a preschool that used Direct Instruction in reading and language for about an hour a day for one year, and (2) a traditional nursery school [Kozloff DI creates felons, but literate ones. Contribution to the DI Listserve, University of Oregon, 31 December, 2011.]

In this post I will explain how I discovered that it is no exaggeration to state that the influence of this and another related study on education policy around the world has been enormous. You will find this study as a central plank of evidence in swathes of papers on a diverse range of education related issues.

What was the approach of the apparently markedly superior Perry High/Scope approach?

The curriculum was based on the principle of active participatory learning, in which children and adults are treated as equal partners in the learning process, and children engage with objects, people, events, and ideas. Abilities to plan, execute, and evaluate tasks were fostered, as were social skills, including cooperation with others and resolution of interpersonal conflicts. The Perry curriculum has been interpreted as implementing the theories of Vygotsky (1986) in teaching self-control and sociability. [Heckman et al 2013]

What was the quality of the research in this study?

I soon discovered that to say this Schweinhart and Weikart study is problematic is an almighty understatement – as outlined by Bereiter, Kozloff (thanks to Greg Ashman for this) and Engelmann.

    • In this study 18 subjects had been taught using Direct Instruction at their preschool for about an hour a day and 14 that had been taught using High/Scope child led methods and 16 subjects in a ‘traditional’ preschool. All were low IQ.
    • These were about a third of the original subjects, the ones who could be traced 20 years later.
    • Far more of the adults who had been in the Direct Instruction preschool had been reared by single, working mothers whose income was about half that of households in the High/Scope group.
    • Eight of the 18 original Direct Instruction group had only one year of preschool while all the High/Scope subjects had two years of preschool.
    • Their gender balance was greatly different, with the High/Scope group having nearly two thirds participants female. This was not accounted for in the study.
    • Attendance at the local high school was 84% for the DI group and 64% for the High/Scope- and again this was not considered.
    • Differences in between groups actually amount to differences in the activities of only one or two persons.
    • The early results of project Follow Through (with 9000 children assigned to nine early childhood curricula) showed that disadvantaged children who received Direct Instruction went from the 20th to about the 50th percentile on the Metropolitan Achievement Test. Children who received the Perry High/Scope curriculum did not do as well. They fell from the 20th percentile to the 11th.

As Kozloff explains. What is “…just plain bizarre, is that these two writers barely entertain the possibility that: (1) a dozen years of school experience; (2) area of residence; (3) family background; (4) the influence of gangs; and (5) differential economic opportunity, had anything to do with adolescent development and adult behavior.

In fact a much larger study then found no correlation between preschool methods and criminality. It seems ridiculous that the outcomes of 18 twenty three year olds, should ever have been taken very seriously… However, this and another highly problematic study by Schweinhart  and Weikhart  (outlined below) have been cited 2444 times.

It sounds mad and I can’t quite understand it myself but  you will find these studies as a central plank of evidence in swathes of papers on a diverse range of education related issues:

Other areas the Schweinhart and Weikart papers have had influence:

Character Education

I was preparing for a panel debate on character education recently and started to look into the evidence in favour of teaching character attributes as a skill. Character can mean so many things that there is actually a vast body of relevant research but I began with a very glossy publication by the OECD. On p40 I found a Schweinhart and Weikart study was being used as evidence. Apparently their Perry preschool programme:

“significantly enhanced adult outcomes including education, employment, earnings, marriage, health, and participation in healthy behaviors, and reduced participation in crime [Heckman et al 2013]. “

The OECD report claimed that the evidence of this study provides:

some of the most compelling evidence that non-cognitive skills can be boosted in ways that produce adult success… Arthur Jensen’s (1969) discussion of IQ fadeout in Head Start and other compensatory programmes promoted the widespread embrace of the notion that intervention efforts are ineffective and that intelligence is genetically determined. His uncritical reliance on intelligence test scores illustrates the fallacy of relying on mono-dimensional measurements of human skills. The Perry intervention provides an effective rebuttal to these arguments. The programme greatly improved outcomes for both participating boys and girls, resulting in a statistically significant rate of return around 7%-10% per annum for both genders (see Heckman et al., 2010a). ”

What was the approach of this  second Schweinhart and Weikart study?

This study had 123 participants, all low IQ and they were divided between a treatment group receiving the ‘Active Participatory Learning’ (as outlined above) in classes of 13 students with two highly qualifies teachers for each group. The control group did not receiving any pre-schooling. “Sessions lasted 2.5 hours and were held five days a week during the school year. Teachers in the program, all of whom had bachelor’s degrees (or higher) in education, made weekly 1.5-hour home visits to treatment group mothers with the aim of involving them in the socio-emotional development of their children. The control group had no contact with the Perry program other than through annual testing and assessment (Weikart, Bond, and McNeil 1978).”

Concern 1: The research findings regarding delinquency have been disconfirmed

Crime reduction in adult life is argued by Heckman, (one of the OECD report authors) to be one of the major benefits of the Perry programme. However, there have been other studies that suggest the Perry programme was not especially effective.  For example this from Mills, Cole, Jenkin and Dale.

“In a previous study of the differential effects of contrasting early intervention programs on later social behavior (Mills, Cole, Jenkins, & Dale, 2002), we found no differences in self-report of juvenile delinquency at age 15 for children enrolled in direct instruction and child-directed models. These results disconfirmed the conclusion of Schweinhart, Weikart, and Larner (1986b) that direct instruction was linked to higher rates of juvenile delinquency and other social differences. Our previous study was limited to self-report of juvenile delinquency, a very coarse measure of social development, in an attempt to replicate the key finding of Schweinhart et al. (1986b). In the present study, we examine additional measures of social development, which might be more sensitive to subtle program differences, including school satisfaction, loneliness, and depression. We administered a battery of social development measures to 174 children at age 15 who had been randomly assigned at preschool age to the two early childhood models. We found no differences on any social outcome for program type. Across a wide range of social behaviors at age 15, there is no evidence that type of early intervention program differentially influences subsequent adolescent social behavior.”

Concern 2: Small sample size

Heckman an author of the OECD report has become a leading player in character education research. Fascinatingly this is on the back of his examination of this study. In his paper on the work he has done with Schweinhart’s and Weikart’s he acknowledges:

“The small sample size of the Perry experiment (123 participants) has led some researchers to question the validity and relevance of its findings (e.g., Herrnstein and Murray 1994; Hanushek and Lindseth 2009). Heckman et al. (2010a) use a method of exact inference that is valid in small samples. They find that Perry treatment effects remain statistically significant even after accounting for multiple hypothesis testing and compromised randomization.

However, for the biggest effects claimed, differences between groups amount to the differences between the activities of about 20 people. I can’t fathom how this sort of data can possibly provide some of  “the most compelling evidence that non cognitive skills can be boosted in ways that produce adult success.” Heckman is a Nobel prize winning economist. It would certainly take a kind of brilliance beyond any I can imagine to be able to justify the enormous influence this study has had over education and public policy.

Concern 3: Scaling up

A critic of the continued reliance on this study by education policy makers is Russ Whitehead who argues that this research was:

“From a time when very little of today’s safety net for the poor was in place…Further, [Perry was a]small single-site program run by [its] developers.  Concluding that findings from these studies demonstrate that current and contemplated state pre-k programs will have similar effects is akin to believing that an expansion of the number of U.S. post offices today will spur economic development because there is some evidence that constructing post offices 50 years ago had that effect.”

There are many studies that examine the impact of preschool education but the likes of Heckman continue to rely upon the outcomes of a small single site ’boutique’ programme from 40-50 years ago. Heckman himself suggests that positive effects of the Perry program have become a cornerstone of the argument for preschool programs (e.g., Obama 2013). “Currently, about 30 percent of all Head Start centers nationwide [across America] offer a version of the Perry curriculum (ICPSR 2010).”

My next post looks at the second study in more detail and, among other things,  I will look at the influence of this study on public policy regarding preschool education.

n.b. This post has been updated to make it clear that there are two separate studies being referenced.

Advertisements

18 thoughts on “The Hydra

  1. Interesting and informative post, thank you. A few observations.

    Number of citations doesn’t necessarily reflect agreement. If a finding is counterintuitive or unusual it might be cited because of that alone.

    Ignoring multiple causal factors and the way they are related isn’t unusual (regrettably) in research that seeks to influence public policy.

    Beware economists bearing research findings. Some of them still think people are rational agents.

  2. You may be interested in http://are.berkeley.edu/~mlanderson/pdf/Anderson%202008a.pdf as it is relevant to your discussion of Perry Preschool and other similar intensive early childhood intervention programs. It is amazing how much impact on public policy and the scientific literature these tiny, boutique programs have had, conducted long ago in a very different era. The replication record of similar programs that had much larger sample sizes is not impressive, at least when the outcome variable is academic achievement or cognitive ability.

    That said, however, sibling control evaluations of Head Start, as discussed here http://www.nber.org/digest/aug01/w8054.html, do find some of the same positive effects on long-term behavioural outcomes such as arrest rates. I discussed these in my talk at London Conference on Intelligence https://docs.google.com/document/d/1Ss34dj8og5VOVHAVJsNhdU1QEz4l4mMCnwuZrQLSDfw/edit?usp=sharing

    Assuming that these effects are real and consistently replicable, it would be very much worth finding out the mechanism by which they operate. No obvious mechanism presents itself to my mind. Interestingly, in this country, Sure Start evaluations showed negative effects on behavioural outcomes, one theory being that this was due to the low quality of the preschool staff employed in the program!

  3. Hello Heather, thanks for writing such an informative post! I do love reading analytical and well-referenced posts such as this.

    As you may already know, I have already openly questioned the philosophy underpinning EYFS and infant curricula, particularly with regard to learning early number, and early writing skills. Also, and yes this is only anecdotal evidence, I distinctly remember having no trouble at all using calm, direct instruction to teach my children how to read, write and add up before they started infant school. It was a simple case of no-frills teaching without distraction or ‘entertainment’ that set them up for being put on the ‘top table’, and there they stayed for the rest of their primary years.

    It seems that many parents do this sort of thing at home, too. Just in case anyone queries, I can confirm that no psychological damage was done. For me, I thought that if they could learn how to sit quietly at a table, learn manners and use a knife and fork from the age of 2, then why not learn how to hold a pencil and mark the paper in a systematic way just before they started school? It just seemed a natural progression and indeed the calm, quiet atmosphere of scholarly activity was a really nice balance for all that boisterous playing that they loved doing.

    Additionally, it was a really lovely way for them to develop their relationship with me because they enjoyed seeing how proud I was of their ability to concentrate and work hard on something that required a fair bit of effort.

    I just wonder when someone will commision a sensible, statistically significant study into the use of direct instruction in early years without there being a huge amount of histrionics? I think a good foundation for getting permission for this study would be a large-scale anonymised survey of parents asking how they managed to get their children reading, writing and adding up before they entered the EYFS classroom, and whether they had to continue ‘tutoring’ even when their children had started school. I think such a study would yield very interesting results. I think a lot of parents naturally ‘do’ direct instruction for all sorts of things.

  4. I had a conversation with my father-in-law about the antagonism regards direct instruction. He’s a joiner. I asked him how he’d teach someone to use, for example, a plane on a piece of timber. “I’d show them what needs to be done, then I’d let them have a go, then I’d give them advice on how to do it better, and carry this on until they were good enough where I could leave them to it”.

    Then I explained to him that in education presently, that style of teaching is frowned upon. His response (edited to remove the swearing): “Well how the bloody hell will they know what’s the right way to do something?”.

    Exactly.

    I do not agree that we should always look to the past for the answers to current educational problems, but we should look to evolving those methods to suit the widest possible cohort of students, rather than look to the complete antithesis.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s