One of yesterday’s holiday jobs was to look in more detail at the new history A level specification from OCR. It made me depressed. I even woke up realising that I had been worrying about the blasted spec in my sleep. I’ve never had an exam specification themed nightmare before. However hope also ‘springs eternal’ and perhaps my initial reaction was overly negative. There is much to be welcomed in the new OCR specification.
Content: I am actually very happy with the content offered. I like OCR’s specification because it doesn’t have any prescribed topics and I want students to learn some early modern history with one teacher over two years, while doing modern history with their other teacher. While the ruling that all history A levels must cover over a 200 year span might mean less reinforcement between topics, there is sound justifiable rationale behind it. Most teachers I speak to don’t seem to look further than the content options when choosing a spec but those who have read my previous blogs will know I am concerned that the actual experience of teaching and learning the material is significantly influenced by the mode of assessment.
Assessment: I think my emotional reaction was because I had been really happy and hopeful about the assessment of the new A level. I have written previously about the way that assessment objectives that split knowledge and skills lead to ‘skills based mark schemes’ which distort the teaching of history (and my other subject, politics) and make marking unreliable. I was absolutely over the moon when I found out that this irrational split no longer exists in the assessment objectives of the new history A level. Instead AO1 acknowledges that good analysis can’t be judged separately from the understanding necessary to deploy well-chosen knowledge. It looks as if essays will no longer have this artificial divide when marked which is great news.
The new A level assessment objectives are below but briefly AO1 is for analysis, AO2 for analysing primary sources and AO3 for analysing interpretations:
AO1 Demonstrate, organise and communicate knowledge and understanding to analyse and evaluate the key features related to the periods studied, making substantiated judgements and exploring concepts, as relevant, of cause, consequence, change, continuity, similarity, difference and significance.
AO2 Analyse and evaluate appropriate source materials, primary and/or contemporary to the period, within its historical context.
AO3 Analyse and evaluate, in relation to the historical context, different ways in which aspects of the past have been interpreted.
The importance of high quality mark schemes: There is a very important reason why assessment objectives MUST be coherent. Currently they are used to make mark schemes. Mark schemes dictate how students must write answers and thus what you teach and also a good mark scheme is utterly crucial for sound marking. Testing whether marking is reliable, when the mark scheme is actually unsound and does not effectively describe progression, is like testing whether a water drainage pipe has any cracks when the pipe is made of paper. Mark schemes need to correctly identify progression. Recent mark schemes have been written to assess individual assessment objectives rather than allowing the examiner to make a holistic ‘best fit’ judgement. I see pros and cons to splitting your mark scheme into separate strands to assess different objectives directly:
• If students frequently perform better in one aspect of a task than another, split strands to the mark scheme can be fairer.
• They also make marking judgements more uniform (I purposefully avoided the word ‘reliable’ there…) as the mark schemes are more specific about allocation of marks.
• Exam boards can demonstrate that each assessment objective is being assessed and given correct weighting in the assessment.
• Often apparently distinct areas of assessment are so inextricably linked that trying to assess them separately is an artificial exercise (as with assessing knowledge/understanding separately from analysis).
• When objectives can’t really be separated it can lead to unpredictable marking as markers need to be specifically told what ‘features of AO2’ might be as it might not be clear otherwise, even to a real pro. However, this means there is less exercise of professional judgement when marking.
• Once a mark scheme has three or more separate marks to be awarded it becomes impossible to make a holistic judgement of essay quality. Given the frequent overlap in what is actually being assessed by different assessment objectives this often means students are penalised for the same fault as they are marked in each strand. I think that is a reason why we so frequently see apparently good answers with bizzarely low marks because of one area of weakness and perhaps explains why they don’t go up when remarks are requested.
So you really want to avoid splitting mark schemes when assessment objectives actually overlap and to avoid making it impossible for the examiner to make a holistic judgement.
Eagle eyed readers may have spotted that all three AOs judge ‘analysis’, AO2 and AO3 just give a specific context when analysis might be used. Perhaps those readers will spot the problem this leads to with the published mark scheme for assessing the history A level coursework essay worth 20% of the A level. The mark scheme is copied at the bottom of this post – do scroll down if you are curious. For this assessment the student uses a selection of primary and secondary sources of evidence to inform their analysis as they answer an essay question
The problems with this mark scheme:
• There are three strands so the examiners can’t make a holistic judgement of quality. [n.b. If you want to see a real shocker AQAs split is far worse]
• It is unclear how ‘analysis’ in AO1 is qualitatively different from the analysis needed for sources AO2 and interpretations AO3. I think in practice AO1 could mean judging how well the student reaches their own conclusion when not handling the sources or interpretations. That would be O.K. as it is not so hard to assess how well students examine the sources/interpretations separately from how well they discuss other evidence to reach an overall conclusion. However if that is what is meant by AO1 it should explicitly say so in the mark scheme? Goodness knows if I am right and I am very worried if not. The loss of clarity here is a serious issue.
• There is just no need to have AO2 and AO3 as separate strands. I have never to my memory, in my whole career, heard a teacher make the distinction between a student’s ability to analyse primary versus secondary sources.
If the writers of our history assessment objectives HAD to have three AOs then the choices aren’t bad at all. However, it is very depressing that each AO must be separately assessed using a three strand mark scheme. In my dream world mark schemes would be written to clearly describe likely progression in a specific task. It is a brilliant improvement that it looks like ordinary essays do now have such mark schemes and rightly so. Surely the assessment objectives should simply dictate the nature of the task? Does someone somewhere actually believe that by making each AO worth a certain number of marks they have in any real way achieved correct weighting for each AO?
Hopefully/maybe/fingers crossed we have better assessment than previously but I wonder if that three strand split for coursework could mean the difference between an assessment that works well and one that is distinctly less reliable.
Top level (17-20 marks) of the mark scheme for new OCR A level coursework essays:
AO1: There is a consistent focus on the question throughout the answer. Detailed, accurate and fully relevant knowledge and understanding is used to effectively analyse and evaluate key features of the period studied in order to produce a clear and well-supported argument which reaches a convincing and substantiated judgement. 17–20 marks
AO2: The answer has excellent evaluation of a fully appropriate range of different sources that are primary and/or contemporary to the period. The answer demonstrates the candidate’s own full engagement with the sources, using detailed and accurate knowledge in order to produce a well-supported analysis of them within their historical context. 9–10 marks
AO3: The answer has excellent evaluation of a fully appropriate range of different interpretations of the historical issue chosen, using detailed and accurate knowledge of the historical context in order to produce a well-supported analysis of the interpretations and to locate them effectively within the wider historical debate on the issue. 9-10 marks