A few years ago I read a serious book on early reading instruction that stated, without question, that there was strong scientific evidence that direct instruction methods, used even at nursery level, lead to increased criminality among adults. It seemed implausible. How could the teaching style of your nursery school, in some cases over only one year, have such a significant impact that it could be measurable 20 years later? However, I then came across the claim in numerous respectable publications. Greg Ashman drew my attention to another in Psychology Today recently. Initially I felt I had to accept the claims – but should just investigate properly first. I soon discovered that all these claims pretty much originate
‘… from about half a dozen articles in the series written by Schweinhart and Weikart, in which they claim to compare (and apparently believe that they demonstrate the superiority of) their High/Scope pre-school curriculum with (1) a preschool that used Direct Instruction in reading and language for about an hour a day for one year, and (2) a traditional nursery school [Kozloff DI creates felons, but literate ones. Contribution to the DI Listserve, University of Oregon, 31 December, 2011.]
In this post I will explain how I discovered that it is no exaggeration to state that the influence of this and another related study on education policy around the world has been enormous. You will find this study as a central plank of evidence in swathes of papers on a diverse range of education related issues.
What was the approach of the apparently markedly superior Perry High/Scope approach?
The curriculum was based on the principle of active participatory learning, in which children and adults are treated as equal partners in the learning process, and children engage with objects, people, events, and ideas. Abilities to plan, execute, and evaluate tasks were fostered, as were social skills, including cooperation with others and resolution of interpersonal conflicts. The Perry curriculum has been interpreted as implementing the theories of Vygotsky (1986) in teaching self-control and sociability. [Heckman et al 2013]
What was the quality of the research in this study?
- In this study 18 subjects had been taught using Direct Instruction at their preschool for about an hour a day and 14 that had been taught using High/Scope child led methods and 16 subjects in a ‘traditional’ preschool. All were low IQ.
- These were about a third of the original subjects, the ones who could be traced 20 years later.
- Far more of the adults who had been in the Direct Instruction preschool had been reared by single, working mothers whose income was about half that of households in the High/Scope group.
- Eight of the 18 original Direct Instruction group had only one year of preschool while all the High/Scope subjects had two years of preschool.
- Their gender balance was greatly different, with the High/Scope group having nearly two thirds participants female. This was not accounted for in the study.
- Attendance at the local high school was 84% for the DI group and 64% for the High/Scope- and again this was not considered.
- Differences in between groups actually amount to differences in the activities of only one or two persons.
- The early results of project Follow Through (with 9000 children assigned to nine early childhood curricula) showed that disadvantaged children who received Direct Instruction went from the 20th to about the 50th percentile on the Metropolitan Achievement Test. Children who received the Perry High/Scope curriculum did not do as well. They fell from the 20th percentile to the 11th.
As Kozloff explains. What is “…just plain bizarre, is that these two writers barely entertain the possibility that: (1) a dozen years of school experience; (2) area of residence; (3) family background; (4) the influence of gangs; and (5) differential economic opportunity, had anything to do with adolescent development and adult behavior.
In fact a much larger study then found no correlation between preschool methods and criminality. It seems ridiculous that the outcomes of 18 twenty three year olds, should ever have been taken very seriously… However, this and another highly problematic study by Schweinhart and Weikhart (outlined below) have been cited 2444 times.
It sounds mad and I can’t quite understand it myself but you will find these studies as a central plank of evidence in swathes of papers on a diverse range of education related issues:
Other areas the Schweinhart and Weikart papers have had influence:
I was preparing for a panel debate on character education recently and started to look into the evidence in favour of teaching character attributes as a skill. Character can mean so many things that there is actually a vast body of relevant research but I began with a very glossy publication by the OECD. On p40 I found a Schweinhart and Weikart study was being used as evidence. Apparently their Perry preschool programme:
“significantly enhanced adult outcomes including education, employment, earnings, marriage, health, and participation in healthy behaviors, and reduced participation in crime [Heckman et al 2013]. “
The OECD report claimed that the evidence of this study provides:
“some of the most compelling evidence that non-cognitive skills can be boosted in ways that produce adult success… Arthur Jensen’s (1969) discussion of IQ fadeout in Head Start and other compensatory programmes promoted the widespread embrace of the notion that intervention efforts are ineffective and that intelligence is genetically determined. His uncritical reliance on intelligence test scores illustrates the fallacy of relying on mono-dimensional measurements of human skills. The Perry intervention provides an effective rebuttal to these arguments. The programme greatly improved outcomes for both participating boys and girls, resulting in a statistically significant rate of return around 7%-10% per annum for both genders (see Heckman et al., 2010a). ”
What was the approach of this second Schweinhart and Weikart study?
This study had 123 participants, all low IQ and they were divided between a treatment group receiving the ‘Active Participatory Learning’ (as outlined above) in classes of 13 students with two highly qualifies teachers for each group. The control group did not receiving any pre-schooling. “Sessions lasted 2.5 hours and were held five days a week during the school year. Teachers in the program, all of whom had bachelor’s degrees (or higher) in education, made weekly 1.5-hour home visits to treatment group mothers with the aim of involving them in the socio-emotional development of their children. The control group had no contact with the Perry program other than through annual testing and assessment (Weikart, Bond, and McNeil 1978).”
Concern 1: The research findings regarding delinquency have been disconfirmed
Crime reduction in adult life is argued by Heckman, (one of the OECD report authors) to be one of the major benefits of the Perry programme. However, there have been other studies that suggest the Perry programme was not especially effective. For example this from Mills, Cole, Jenkin and Dale.
“In a previous study of the differential effects of contrasting early intervention programs on later social behavior (Mills, Cole, Jenkins, & Dale, 2002), we found no differences in self-report of juvenile delinquency at age 15 for children enrolled in direct instruction and child-directed models. These results disconfirmed the conclusion of Schweinhart, Weikart, and Larner (1986b) that direct instruction was linked to higher rates of juvenile delinquency and other social differences. Our previous study was limited to self-report of juvenile delinquency, a very coarse measure of social development, in an attempt to replicate the key finding of Schweinhart et al. (1986b). In the present study, we examine additional measures of social development, which might be more sensitive to subtle program differences, including school satisfaction, loneliness, and depression. We administered a battery of social development measures to 174 children at age 15 who had been randomly assigned at preschool age to the two early childhood models. We found no differences on any social outcome for program type. Across a wide range of social behaviors at age 15, there is no evidence that type of early intervention program differentially influences subsequent adolescent social behavior.”
Concern 2: Small sample size
Heckman an author of the OECD report has become a leading player in character education research. Fascinatingly this is on the back of his examination of this study. In his paper on the work he has done with Schweinhart’s and Weikart’s he acknowledges:
“The small sample size of the Perry experiment (123 participants) has led some researchers to question the validity and relevance of its findings (e.g., Herrnstein and Murray 1994; Hanushek and Lindseth 2009). Heckman et al. (2010a) use a method of exact inference that is valid in small samples. They find that Perry treatment effects remain statistically significant even after accounting for multiple hypothesis testing and compromised randomization.
However, for the biggest effects claimed, differences between groups amount to the differences between the activities of about 20 people. I can’t fathom how this sort of data can possibly provide some of “the most compelling evidence that non cognitive skills can be boosted in ways that produce adult success.” Heckman is a Nobel prize winning economist. It would certainly take a kind of brilliance beyond any I can imagine to be able to justify the enormous influence this study has had over education and public policy.
Concern 3: Scaling up
A critic of the continued reliance on this study by education policy makers is Russ Whitehead who argues that this research was:
“From a time when very little of today’s safety net for the poor was in place…Further, [Perry was a]small single-site program run by [its] developers. Concluding that findings from these studies demonstrate that current and contemplated state pre-k programs will have similar effects is akin to believing that an expansion of the number of U.S. post offices today will spur economic development because there is some evidence that constructing post offices 50 years ago had that effect.”
There are many studies that examine the impact of preschool education but the likes of Heckman continue to rely upon the outcomes of a small single site ’boutique’ programme from 40-50 years ago. Heckman himself suggests that positive effects of the Perry program have become a cornerstone of the argument for preschool programs (e.g., Obama 2013). “Currently, about 30 percent of all Head Start centers nationwide [across America] offer a version of the Perry curriculum (ICPSR 2010).”
My next post looks at the second study in more detail and, among other things, I will look at the influence of this study on public policy regarding preschool education.
n.b. This post has been updated to make it clear that there are two separate studies being referenced.