Is grouping pupils by ability ‘iniquitous’?

Setting, Differentiation by Task or Mixed Ability?

There isn’t a ‘right answer’ to every question in teaching. Often there are simply decisions and those decisions have consequences, both positive and negative. This is especially true with the debate over differentiation, setting and mixed ability teaching.

Today I read this persuasive and angry piece by Mary Mered on the iniquities of ability grouping in primary schools. I fully sympathised with all her nuanced arguments. I have seen the way ability based table grouping can hold children back at primary level and I know there is research showing mixed ability teaching is (can be?) more successful. However, I think Mary is wrong to presume the problems with mixed ability teaching are particularly surmountable. It has its own distinct drawbacks, as any teacher allocated a massively mixed ability class knows full well. So while I fully sympathise with Mary’s concerns I think we need to openly acknowledge the weaknesses of EACH approach, weigh up the relative advantages and disadvantages given the children we need to teach and the type of subject matter. Then, having chosen the least of the possible evils in terms of the grouping rationale we plump for, we must then do our best to mitigate against the known weaknesses of that approach.

When deciding whether to set or go mixed ability I wish decision makers would stop following mindless blanket rules and instead take account of the following (rather obvious) considerations:

How wide is the ability range that will need to be catered for?

Mixed ability teaching means compromising. The explanation I would give in history if only A* students were present is necessarily different from what I actually say when I know B or C or even D ability students are also in the classroom. However, if your ‘mixed ability’ class does not have a big ability range, if the teacher can give whole class explanations and generally set the same tasks for all, then mixed ability teaching is probably the best bet. Children avoid being pigeon holed in terms of expectations – the big danger of setting – but can still be taught as a class.

Is the subject matter hierarchical?

In subjects such as maths and languages a very wide ability range can only be catered for by running what are in fact four or five mini classes each lesson because there is limited similarity between the subject matter different ability groups are handling. This significantly dilutes the ability of the teacher to provide strong explanations, respond to difficulties and assess learning. In humanities it is possible to teach a wider ability range as explanations can more easily be given to the whole class.

Is the workload feasible?

It is enormously hard work for the teacher who has to ‘differentiate each lesson three ways/four ways/for each individual…’ or whatever the latest unreasonable ‘best practice’ demands are on the conscientious or over-managed. There is also the danger that many tasks are differentiated because they have to be and thus not in ways that are going to ensure good progress. For example, Mary Mered wrote that her daughter on the low ability table was only required to cut and stick ready labelled parts of the human ear when high ability groups got to draw and label the ear themselves. Why on earth should the less able have less practice drawing and labelling? If they actually had poorer writing or presentation skills surely they need more practice? In fact there seems no reason why virtually all students couldn’t have done the same task in this case. Mandatory differentiation is daft as in many subjects it is difficult to devise a range of different tasks that exactly meet the needs of different groups and it is massively time consuming.

Is differentiation REALLY going to be better than setting?

Lessons where there is generally differentiation by task are worse than setting. You have all the drawbacks of setting – the fixed expectations and possible psychological hit but few of the advantages. The teacher of a set has a more manageable workload, can hone great explanations and spot and deal with misconceptions more easily. The teacher delivering differentiated lessons is less able to keep all groups on task and more assistants sometimes become necessary to provide extra support, which is expensive.

If setting is not practical but the ability range is wide can you avoid blanket rules about task differentiation?

Yes! Schools can avoid blanket rules and teachers must be allowed to be free to judge what differentiation is actually necessary, to avoid an outrageous workload and counterproductive or pointless extra task creation.

Conclusions:

If you choose mixed ability grouping then at least be aware that your most able are likely to find the explanations easy, tasks unchallenging or resent spending their time over the years supporting the weakest (who may feel lost and lack support).

If you think differentiation is the only option then be aware that there will be significant work load implications and the teacher’s ability to explain ideas and monitor and supervise each group will be weakened. You also need to be aware of the problems pigeon holing and low expectations.

If you opt for setting be hyper vigilant about the problems with pigeon holing and low expectations.

In an ideal world (otherwise known as Finland) there would be an expectation that the weakest students could to some extent ‘catch up’ with a focus on providing genuinely effective remedial help to that end, so that mixed ability teaching was more practicable in the long term.

In the meantime I tend towards setting as the best option when faced with wide ability ranges. This is because it is possible with good management to overcome some of the problems created by low expectations. A school can create a culture in which they expect weaker students to be given help to genuinely catch up, rather than applying developmentalist assumptions that will stop children having the opportunities to make more rapid progress than their peers. By comparison, the problems with mixed ability teaching or differentiating by task are innate in the model and less surmountable.

To those that want to believe that you can have the best of all worlds – sorry. There really isn’t such a thing as magic. There are simply decisions, each with pros and cons. By demanding that teachers deliver a ‘best of all worlds’ product you are simply piling unreasonable expectations on teachers and making them secretly feel like failures.

Advertisements

The Hydra Part 2

The Hydra Part 2

or ‘Weikart and Scweinhart’s [Perry] High/Scope Preschool Curriculum comparison Study Through Age 23’ and Lifetime Effects: The HighScope Perry Preschool Study Through Age 40 (2005) For details of these studies see Part 1

This post begins with the story of two preschool approaches and their fates. One approach was ‘teacher led’ Direct Instruction and the other ‘child led’ High/Scope Perry preschool, already discussed at length in my previous post. As the NIFDI website explains:

There was an enormous educational experiment beginning in 1968 comparing these two approaches called Project Follow Through. It was the most extensive educational experiment ever conducted. Beginning in 1968 under the sponsorship of the American federal government, it was charged with determining the best way of teaching at-risk children from kindergarten through grade 3. Over 200,000 children in 178 communities were included in the study, and 22 different models of instruction were compared.

Project-Follow-Through-Chart

Evaluation of the project occurred in 1977, nine years after the project began. The results were strong and clear. Students who received Direct Instruction had significantly higher academic achievement than students in any of the other programs. They also had higher self esteem and self-confidence. No other program had results that approached the positive impact of Direct Instruction. Subsequent research found that the DI students continued to outperform their peers and were more likely to finish high school and pursue higher education. The Perry High/Scope approach is on the graph above. It is the ‘Cognitive Curriculum’ approach (second from the end). You can see it wasn’t quite as successful…

So which preschool method was the winner? The answer might seem obvious from the table above but you would be mistaken. The approach that now dominates is the High/Scope Perry preschool approach – and this is to some extent on the basis of two very small and problematic studies.

In my last blog I outlined the problems with these two small studies from 40-50 years ago by Schweinhart and Weikart that examined the impact of their Perry High/Scope preschool methods on the participants into adulthood. I began to explain the staggering influence over education policy these two studies have had. To find out the details of these two studies click back to my last blog but to give you the gist here is Kozloff’s summary some of the problems with the way these studies have been interpreted:

 What is “…just plain bizarre, is that [Schweinhart and Weikart] barely entertain the possibility that: (1) a dozen years of school experience; (2) area of residence; (3) family background; (4) the influence of gangs; and (5) differential economic opportunity, had anything to do with adolescent development and adult behavior.

In this post I will look at the impact of these studies on early years education.

First: These studies have been crucial in building a case for the importance of preschool education in the early years of childhood.

The National Audit Office commissioned a summary of the evidence on the impact of early years’ provision on young children with emphasis given to children from disadvantaged backgrounds in 2004. Such a paper offers a good review of the key research literature that has been influencing public policy.

The evidence of the biggest ever educational experiment “Project Follow Through” does not feature but both tiny Perry preschool studies feature heavily in the report. The second study of 123 subjects is one of six randomised controlled trials cited to provide evidence of the effectiveness of preschool programmes for disadvantaged children. These trials were all small scale and to an extent have contradictory findings. Some found reductions in antisocial behaviour but not academic gains and others had the opposite findings. None of the trials seemed to offer better evidence than the problematic second Perry study of 123 subjects.

The report then goes onto look at the evidence of preschool having an impact on the general population, rather than studies that only focus on those children that are highly disadvantaged. Perhaps it would be reasonable to argue that the majority of this research had positive findings, either for social development, academic development or both. However, there were still many contradictory findings and what seemed to be quite low effect sizes. Often, the preschool methods examined provide limited academic advantage but show positive social effects later in life. Having looked at the literature, I do begin to wonder if the commenter on this American website has a point:

“As the authors note, it is indeed quite a puzzle how pre-school education could possibly not show positive effects during schooling, yet have dramatically positive effects in adulthood. But if you look at the studies that find no effect during early schooling, you’ll find them very dense with objective facts such as testing results, etc. But if you look at those handful of studies purporting to show dramatic adulthood outcomes, you don’t find a lot of such data. In fact these latter studies aren’t scientific – they’re advocacy. They didn’t come up with rigorously selected criteria prior to pre-school to evaluate the outcomes, but instead retrospectively identified metrics to compare the control groups, leaving much room for post-hoc cherry-picking. The most parsimonious explanation of the paradox is that the adulthood-effects studies are flawed, and aren’t actually showing any real positive outcomes from pre-school.”

I’d need to do much more research to comment further. What is very interesting to me is that that there is no doubt the much publicised benefits of preschool education are built on shakier foundations than advocates would like policy makers to think. If you are interested in forming an opinion, this post , this and this post and this riposte are a great starting point.

There is a very concerning reliance on the Schweinhart and Weikart Perry preschool studies in the National Audit Office report.

1. The shockingly ‘dodgy’ first study (see my previous post) is relied upon to define ‘high quality’ child care.

I’ll explain further. One noticeable feature of the research on preschool effectiveness is the reliance on the idea that the reason some studies showed no effect was because the preschool programme was not ‘high quality’. On one level that is sensible as there must be huge variation in the quality of preschool provision but it is also a way of arguing that we should dismiss all studies with weak or no effects, presuming they are not ‘high quality’. I had noticed that the term ‘high quality’ is used frequently in the literature and repeatedly by policy makers and so I was interested to see how researchers had reached a decision on what constituted ‘high quality’. This is what the National Audit Office report had to say (I’ll highlight the key passage but thought I should include the full extract):

“In pre-school education (3+ years), quality is most often associated with the concept of developmentally appropriate practice. Bryant et al. (1994) report on several studies that illustrate the relationship between developmentally appropriate practice and child outcomes. The High/Scope study [Schweinhart’s and Weikart’s] shows that children who attend a developmentally appropriate, child-centred programme are better adjusted socially than similar children who attend a teacher-directed programme implementing a direct-instruction curriculum (Schweinhart, Weikart, and Larner 1986). In North Carolina, Bryant, Peisner-Feinberg, and Clifford (1993) found that children’s communication and language development were positively associated with appropriate care giving. Burts et al. (1992) and Hart & Todd (1995) show that children’s attendance in developmentally appropriate kindergartens is associated with fewer stress behaviours. Educational content is also important for this age group. Jowett & Sylva (1986) found nursery education graduates did better in primary school than playgroup graduates, suggesting the value of an educationally orientated pre-school. The research demonstrates that the following aspects of pre-school quality are most important for enhancing children’s development: Well-trained staff who are committed to their work with children, facilities that are safe and sanitary and accessible to parents, ratios and group sizes that allow staff to interact appropriately with children, supervision that maintains consistency, staff development that ensures continuity, stability and improving quality and a developmentally appropriate curriculum with educational content

  • Oh my goodness! How can the study referred to possibly support the weight being placed on it (see first half of my previous post)? The National Audit Office report writer considers it a central plank in research used to define ‘high quality’ child care when it is hopelessly flawed.

 

  • Not only this, there is good research demonstrating the academic advantage of preschool methods with approaches which are contradictory approaches to the Perry preschool methods. Why does the definition of ‘high quality’ actually exclude these successful alternative methods? The Perry Preschool method endorsed ‘developmentally appropriate practice’ which means ‘child led’ experiential curriculum rather than teacher led instruction. The National Audit Office report actually mentions the success of a very teacher led approach, used widely in France:

“Studies of children in the French Ecoles Maternelle programme (Bergmann 1996) show that this programme enhances performance in the school system for children from all social classes and that the earlier the children entered the pre-school program, the better their outcome.”

The early results of project Follow Through (with 9000 children assigned to nine early childhood curricula) showed that disadvantaged children who received Direct Instruction (anathema to those advocating child led approaches) went from the 20th to about the 50th percentile on the Metropolitan Achievement Test. Children who received the Perry High/Scope curriculum did not do as well. They fell from the 20th percentile to the 11th percentile.

2. It is a concern that the second Schweinhart and Weikart study is used to demonstrate the cost effectiveness of preschool education in the National Audit Office report.

“The Perry Pre-school Project is the most cited study in this area and its benefits are well reported (e.g. Schweinhart et al. 1993). The cost-benefit analysis for this project (Barnett 1996) is worthy of some consideration as its findings are extensively used for justifying expenditure on pre-school education and care.

This is troubling given the small size and context of the original study. It can’t possibly support these inferences.

Second: This research on impact of education in children’s early years has been used to justify our statutory Early Years Foundation Stage.

The EPPE study was an enormously significant longitudinal Study funded by the DfES from 1997 – 2004 and with findings in support of our ‘child led’ EYFS curriculum. It actually cites the first highly flawed Schweinhart and Weikart study where, among other problems, findings were the result of changes in outcomes of two or three people. This  is what it says of a study that should never have been taken seriously:

Previous Research on the Effectiveness of Pre-School Education and Care: The vast majority of longitudinal research on early education has been carried out in the U.S.Two of the studies cited most often are the Abecedarian Project and the Perry Pre-school Programme (Ramey & Ramey, 1998; Schweinhart and Weikart, 1997). Both used randomised control trial methods to demonstrate the lasting effects of high quality early intervention. These landmark studies, begun in the 1970s, have been followed by further small scale ‘experiments’ (see the Early Headstart, Love et al., 2001) and larger cohort studies (See Brooks-Gunn, 2003; Melhuish, 2004a, for reviews). This huge body of literature points to the many positive effects of centre-based care and education.”

The EPPE refers to the first Perry study as having been ‘admired for decades for its internal validity.’ I am not sure the writer has actually looked at the study. Amusingly, the EPPE report writer seems bemused when their own findings contradict those of the Perry study:

“ It appears therefore that the beneficial impact of pre-school on cognitive attainment is more long lasting than that on social behaviour. Social/behavioural outcomes may be more influenced than cognitive outcomes by the primary school peer group. Still this finding is at odds with the Perry Pre-school study, which indicated that the social outcomes of pre-school were more salient than the cognitive ones by adolescence. Data from the EPPE continuation study, will follow children into adolescence, to shed light on this.”

The influence the Perry study has had on understanding of what might be meant by ‘high quality’ provision is quite concerning given these contradictory findings.  Our national EYFS curriculum presumes ‘high quality’ provision means:

“…all areas of learning [are] to be delivered through planned, purposeful play, with a balance of adult-led and child-initiated activities… Professionals should therefore adopt a flexible, fluid approach to teaching, based on the level of development of each child. Research also confirms that the quality of teaching is a key factor in a child’s learning in the early years. High quality teaching entails high levels of interaction, warmth, trust, creativity and sensitivity. Practitioners and children work together to clarify an idea, solve a problem, express opinions and develop narratives.

However this is problematic:

1. We know the first flawed Schweinhart and Weikart Perry study contributed towards this idea of ‘high quality’ provision but it did not actually improve academic outcomes for its young participants. Evidence such as Follow Through (from the same era is) not even mentioned which did improve academic outcomes. The Perry programme was viewed as worthwhile because it was believed this intervention limited adult anti-social behaviour.

“The Perry program initially boosted IQs. However, this effect faded within a few years after the end of the two-year program, with no statistically significant effect remaining for males, and only a borderline significant effect remaining for females.

2. I believe good parenting makes a difference to children and so, although the studies seem unconvincing it is not outside the realms of possibility that the committed teachers involved in the second Schweinhart and Weikart study had some positive impact on their pupils. They were a highly committed team of extremely well qualified teachers. They must have involved themselves deeply in the lives of their very disadvantaged, very low IQ pupils, given they made 90 minutes visits every week to their homes as well as teaching them. Such a scheme is not really more widely replicable though.

Also, given the range of possible benefits the children experienced it is quite a leap to pin point the child-led learning as a crucial factor and suggest it provides a model of ‘high quality’ pre-schooling for all children today.

However some sort of  model is what the Perry preschool at Ypsilanti Michigan seems to be:

3. The use of these studies to justify ‘developmentally appropriate’, ‘child led’ practices is concerning. If children are not highly disadvantaged or at any great risk of engaging in felonies (most children) it is hard to see how these studies can justify the use of such approaches. This is particularly given the success of other methods. [The results of project Follow Through (with 9000 children assigned to nine early childhood curricula) showed that all groups made greater academic gains than using the Perry High/Scope approaches.]

Some of the many subsequent studies on the effectiveness of preschool education record academic gains and others don’t. Some record social gains and others don’t. However, partly thanks to the Schweinhart and Weikart studies the importance of the ‘high quality’ preschool education is a mantra repeated by all politicians. The Perry approach has become a model for ‘high quality’ preschool education around the world and currently, about 30 percent of all Head Start centres in America offer a version of the Perry curriculum (ICPSR 2010). For those, like myself, concerned that ‘child-led’ approaches are not the most efficacious, the impact of these studies is an enormous cause for concern.

So why are two small, context dependent flawed studies still widely cited? Why are the results of the largest ever educational experiment from the same era ignored in early years research literature? There is only one possible explanation. The small studies said what educationalists wanted to hear, that a child led curriculum could be proved to effect life outcomes.  The enormous Project Follow Through had more uncomfortable findings – so it was ignored.

I examine the evidence base for what is deemed ‘high quality’ pre school education here: https://heatherfblog.wordpress.com/2015/07/09/a-truism-that-needs-questioning/?relatedposts_hit=1&relatedposts_origin=626&relatedposts_position=2

 

The Hydra

‘The Hydra’

or ‘Weikart and Scweinhart’s [Perry] High/Scope Preschool Curriculum comparison Study Through Age 23’ and Lifetime Effects: The HighScope Perry Preschool Study Through Age 40 (2005)

A few years ago I read a serious book on early reading instruction that stated, without question, that there was strong scientific evidence that direct instruction methods, used even at nursery level, lead to increased criminality among adults. It seemed implausible. How could the teaching style of your nursery school, in some cases over only one year, have such a significant impact that it could be measurable 20 years later? However, I then came across the claim in numerous respectable publications. Greg Ashman  drew my attention to another in Psychology Today recently. Initially I felt I had to accept the claims – but should just investigate properly first. I soon discovered that all these claims pretty much originate

‘… from about half a dozen articles in the series written by Schweinhart and Weikart, in which they claim to compare (and apparently believe that they demonstrate the superiority of) their High/Scope pre-school curriculum with (1) a preschool that used Direct Instruction in reading and language for about an hour a day for one year, and (2) a traditional nursery school [Kozloff DI creates felons, but literate ones. Contribution to the DI Listserve, University of Oregon, 31 December, 2011.]

In this post I will explain how I discovered that it is no exaggeration to state that the influence of this and another related study on education policy around the world has been enormous. You will find this study as a central plank of evidence in swathes of papers on a diverse range of education related issues.

What was the approach of the apparently markedly superior Perry High/Scope approach?

The curriculum was based on the principle of active participatory learning, in which children and adults are treated as equal partners in the learning process, and children engage with objects, people, events, and ideas. Abilities to plan, execute, and evaluate tasks were fostered, as were social skills, including cooperation with others and resolution of interpersonal conflicts. The Perry curriculum has been interpreted as implementing the theories of Vygotsky (1986) in teaching self-control and sociability. [Heckman et al 2013]

What was the quality of the research in this study?

I soon discovered that to say this Schweinhart and Weikart study is problematic is an almighty understatement – as outlined by Bereiter, Kozloff (thanks to Greg Ashman for this) and Engelmann.

    • In this study 18 subjects had been taught using Direct Instruction at their preschool for about an hour a day and 14 that had been taught using High/Scope child led methods and 16 subjects in a ‘traditional’ preschool. All were low IQ.
    • These were about a third of the original subjects, the ones who could be traced 20 years later.
    • Far more of the adults who had been in the Direct Instruction preschool had been reared by single, working mothers whose income was about half that of households in the High/Scope group.
    • Eight of the 18 original Direct Instruction group had only one year of preschool while all the High/Scope subjects had two years of preschool.
    • Their gender balance was greatly different, with the High/Scope group having nearly two thirds participants female. This was not accounted for in the study.
    • Attendance at the local high school was 84% for the DI group and 64% for the High/Scope- and again this was not considered.
    • Differences in between groups actually amount to differences in the activities of only one or two persons.
    • The early results of project Follow Through (with 9000 children assigned to nine early childhood curricula) showed that disadvantaged children who received Direct Instruction went from the 20th to about the 50th percentile on the Metropolitan Achievement Test. Children who received the Perry High/Scope curriculum did not do as well. They fell from the 20th percentile to the 11th.

As Kozloff explains. What is “…just plain bizarre, is that these two writers barely entertain the possibility that: (1) a dozen years of school experience; (2) area of residence; (3) family background; (4) the influence of gangs; and (5) differential economic opportunity, had anything to do with adolescent development and adult behavior.

In fact a much larger study then found no correlation between preschool methods and criminality. It seems ridiculous that the outcomes of 18 twenty three year olds, should ever have been taken very seriously… However, this and another highly problematic study by Schweinhart  and Weikhart  (outlined below) have been cited 2444 times.

It sounds mad and I can’t quite understand it myself but  you will find these studies as a central plank of evidence in swathes of papers on a diverse range of education related issues:

Other areas the Schweinhart and Weikart papers have had influence:

Character Education

I was preparing for a panel debate on character education recently and started to look into the evidence in favour of teaching character attributes as a skill. Character can mean so many things that there is actually a vast body of relevant research but I began with a very glossy publication by the OECD. On p40 I found a Schweinhart and Weikart study was being used as evidence. Apparently their Perry preschool programme:

“significantly enhanced adult outcomes including education, employment, earnings, marriage, health, and participation in healthy behaviors, and reduced participation in crime [Heckman et al 2013]. “

The OECD report claimed that the evidence of this study provides:

some of the most compelling evidence that non-cognitive skills can be boosted in ways that produce adult success… Arthur Jensen’s (1969) discussion of IQ fadeout in Head Start and other compensatory programmes promoted the widespread embrace of the notion that intervention efforts are ineffective and that intelligence is genetically determined. His uncritical reliance on intelligence test scores illustrates the fallacy of relying on mono-dimensional measurements of human skills. The Perry intervention provides an effective rebuttal to these arguments. The programme greatly improved outcomes for both participating boys and girls, resulting in a statistically significant rate of return around 7%-10% per annum for both genders (see Heckman et al., 2010a). ”

What was the approach of this  second Schweinhart and Weikart study?

This study had 123 participants, all low IQ and they were divided between a treatment group receiving the ‘Active Participatory Learning’ (as outlined above) in classes of 13 students with two highly qualifies teachers for each group. The control group did not receiving any pre-schooling. “Sessions lasted 2.5 hours and were held five days a week during the school year. Teachers in the program, all of whom had bachelor’s degrees (or higher) in education, made weekly 1.5-hour home visits to treatment group mothers with the aim of involving them in the socio-emotional development of their children. The control group had no contact with the Perry program other than through annual testing and assessment (Weikart, Bond, and McNeil 1978).”

Concern 1: The research findings regarding delinquency have been disconfirmed

Crime reduction in adult life is argued by Heckman, (one of the OECD report authors) to be one of the major benefits of the Perry programme. However, there have been other studies that suggest the Perry programme was not especially effective.  For example this from Mills, Cole, Jenkin and Dale.

“In a previous study of the differential effects of contrasting early intervention programs on later social behavior (Mills, Cole, Jenkins, & Dale, 2002), we found no differences in self-report of juvenile delinquency at age 15 for children enrolled in direct instruction and child-directed models. These results disconfirmed the conclusion of Schweinhart, Weikart, and Larner (1986b) that direct instruction was linked to higher rates of juvenile delinquency and other social differences. Our previous study was limited to self-report of juvenile delinquency, a very coarse measure of social development, in an attempt to replicate the key finding of Schweinhart et al. (1986b). In the present study, we examine additional measures of social development, which might be more sensitive to subtle program differences, including school satisfaction, loneliness, and depression. We administered a battery of social development measures to 174 children at age 15 who had been randomly assigned at preschool age to the two early childhood models. We found no differences on any social outcome for program type. Across a wide range of social behaviors at age 15, there is no evidence that type of early intervention program differentially influences subsequent adolescent social behavior.”

Concern 2: Small sample size

Heckman an author of the OECD report has become a leading player in character education research. Fascinatingly this is on the back of his examination of this study. In his paper on the work he has done with Schweinhart’s and Weikart’s he acknowledges:

“The small sample size of the Perry experiment (123 participants) has led some researchers to question the validity and relevance of its findings (e.g., Herrnstein and Murray 1994; Hanushek and Lindseth 2009). Heckman et al. (2010a) use a method of exact inference that is valid in small samples. They find that Perry treatment effects remain statistically significant even after accounting for multiple hypothesis testing and compromised randomization.

However, for the biggest effects claimed, differences between groups amount to the differences between the activities of about 20 people. I can’t fathom how this sort of data can possibly provide some of  “the most compelling evidence that non cognitive skills can be boosted in ways that produce adult success.” Heckman is a Nobel prize winning economist. It would certainly take a kind of brilliance beyond any I can imagine to be able to justify the enormous influence this study has had over education and public policy.

Concern 3: Scaling up

A critic of the continued reliance on this study by education policy makers is Russ Whitehead who argues that this research was:

“From a time when very little of today’s safety net for the poor was in place…Further, [Perry was a]small single-site program run by [its] developers.  Concluding that findings from these studies demonstrate that current and contemplated state pre-k programs will have similar effects is akin to believing that an expansion of the number of U.S. post offices today will spur economic development because there is some evidence that constructing post offices 50 years ago had that effect.”

There are many studies that examine the impact of preschool education but the likes of Heckman continue to rely upon the outcomes of a small single site ’boutique’ programme from 40-50 years ago. Heckman himself suggests that positive effects of the Perry program have become a cornerstone of the argument for preschool programs (e.g., Obama 2013). “Currently, about 30 percent of all Head Start centers nationwide [across America] offer a version of the Perry curriculum (ICPSR 2010).”

My next post looks at the second study in more detail and, among other things,  I will look at the influence of this study on public policy regarding preschool education.

n.b. This post has been updated to make it clear that there are two separate studies being referenced.