Sunday, 30 April 2017

Controlled Assessment: A Brief History!

A reminder of the rationale for the Languages GCSE 2010 Controlled Assessments in Speaking and Writing.

1. December 2006 - QCA 11-19 reform programme - 'The current system will be monitoring [sic] to ensure integrity, safety and a reduction in burden'.  The Government wanted to tighten up the GCSE.  'Coursework' for subjects was often carried out in 'uncontrolled' conditions' (i.e. could be at home, could be someone else was actually doing it, even though all had to sign it was their own work ..) so the new GCSEs (first taught 2008, first tested 2010) insisted that all subjects be tested by final exam except when there was a compelling case that it was impossible to test subject skills in this way.

2. As Part of this review, QCA commissioned an investigation into the assessment of speaking for GCSE modern foreign languages,  Here is the executive summary:

Executive summary
Assessment of GCSE speaking: the current position
The current method of assessing speaking in modern foreign languages (MFL) has changed little since the introduction of GCSE in 1988. The majority of students take one examination at the end of the two-year course. This test is usually composed of short structured role-plays and a prepared presentation followed by a conversation. The test is recorded. In most cases the recordings are sent to the awarding body for marking, but some awarding bodies give centres the option to mark the tests and for their marks to be moderated by the awarding body.

The nature of this test can lead to formulaic responses and heavy reliance on a limited range of memorised language. This in turn has a negative impact on teaching and learning, with little opportunity for students to demonstrate and apply their knowledge and skills and to develop independence.

 3. At the same time, Lord Dearing was conducting a review into languages. (Published March 2007)
His understanding of the problems at GCSE are in section 3 of his report.
In the light of the evidence from teachers, he made recommendations which included the two following aspects:

(1)   Subject matter for writing and speaking: The content needed to be flexible to suit the individual motivations of the students.   [See 3.17 ' It is particularly in these years that the context of the learning needs to be stimulating to pupils and to engage them in discussion, debates and writing about subjects that are of concern and interest to teenagers.' ]
(2)   Speaking test: It was deemed that a short final test was stressful and unreliable.    [See 3.22 . "We also proposed a new approach to the assessment of speaking and listening, which rightly account for half the marks in the GCSE, on the grounds that the present method is too stressful and too short to be a reliable way of assessing what the candidates can do. It is interesting that when people spoke about the oral test, that however long ago it may have been, it is often remembered as a stressful experience. We therefore proposed that these parts of the examination should be over a period through moderated teacher assessment."

4. QCA (as Ofqual was then) required the Awarding Organisations (AOs) to take account of the Dearing review  (informed by teachers).  As part of the overall 'across all subjects' requirements, QCA  stipulated that the controlled assessment could only be 0%, 25% or 60% (and nothing else.)  This led inexorably to the decision to require 60% of the content to be assessed by 'controlled assessment'.   This then allowed for (1) subject matter for the writing and speaking to be determined by the teacher  in response to pupil preferences rather than being quesitons common to all set by the AO in a terminal exam and  (2)  the speaking skill to be tested over time rather than as a final short exam.


Monday, 10 April 2017

GCSE 9-1 grading

This blogpost is focussed particularly on ML in the context of the reformed GCSE currently being studied by Year 10 but the principles and messages are exactly the same for
(a)    Current Y11 in maths and English
(b)   Other reformed GCSE subjects for current Year 10

People are asking how they can use the results of mock exams being taken by current year 10 pupils to estimate final GCSE grades.

Being fair to all the pupils of all abilities at a time of major transition and change (and by extension to their teachers) taking a wide range of GCSEs  in a real world of historical legacy and political realities, not starting from scratch, we need to be sophisticated and thoughtful and avoid simplistic solutions.

It's a complex situation but there are a few key principles to help through this transition period to ensure fairness for pupils.

There is a range of information available from Ofqual itself (the body responsible for ensuring the quality of standards in qualifications which have been written by exam boards within the constraints set by DfE) and from ASCL (the Association of School and College Leaders).

There is also much rhetoric around which is confusing the situation and so it is important to unpick some of the political rhetoric, failed logic etc. etc.  For example, 'the content is more demanding'.  Yes, the specifications do appear more demanding.  They have stripped  out any mention of what pupils can be expected to know understand and do at lower levels.  However, grades will be awarded according to 'comparable outcomes' (see below), not according to any statement-related criteria.

At a time of transition, change and uncertainty this can open a floodgate to discussion (e.g the value of statement-related criteria).  This may be very interesting, but it can unnecessarily distract from the task at hand,  It’s important to focus on what is happening in order to ensure fairness to pupils during this transition.


The Ofqual Blog has some extremely useful posts.  Here are some examples:

(a)    a 9-1 campaign including a chart to show how grades 1, 4 and 7 will compare with current grades G, C and A

(b)   Grade boundaries: the problems with predictions 3/2/17  This blogpost explicitly says that it is unhelpful to estimate grade boundaries in mock exams.  Three reasons are given for this, and I copy and paste the third of  these below:

Statistics will play a key role in making sure this year’s students are not disadvantaged by being the first to sit these new GCSEs. Exam boards will use prior attainment at Key Stage 2 for the 16-year-old cohort to predict likely achievement at the key grades – 1, 4 and 7. The bottom of these grades will be aligned with the bottom of grades G, C and A respectively so the proportions of students achieving these grades or higher will be broadly similar to the previous year. We, and the exam boards, will have the full national picture; other organisations will only have a sub-set of the cohort, which may not be representative of the national situation.

(c)    Setting grade 9 in new GCSEs  This post sets out how the number of grade 9s will be caluclated

 2.      ASCL ADVICE

ASCL has provided advice to students, parents, teachers and  headteachers.  Here is a copy and paste of the first FAQ in the Headteacher paper:

1 Can I give my governors an estimate of how our GCSE grade distribution in the reformed GCSEs will be this year? What about grade boundaries?

ASCL’s advice is not to rely on any predictions of grade boundary marks for new GCSEs next summer. Statistics will play a key role in making sure this year’s students are not disadvantaged by being the first to sit these new GCSEs, and the setting of grade boundaries will need to take account of the national picture which will not be known until all the papers are marked in the summer. In the School Inspection Update March 2017, Ofsted inspectors have been advised not to press schools for predictions of outcomes this year. It is possible to give an indication of the likely grade distribution because of the statistical linkage specified by Ofqual. ASCL will be publishing more guidance on this in the coming months.

One of the most common issues being raised on ML fora relates to other teachers or managers either expecting to be able to convert raw marks to grades or having managers who are putting pressure on them to convert raw marks to grades as a way of telling pupils about their attainment / progress. What can you do about this?

You have to assess what is or is not within your control.  Staff are in a complicated situation.  The ideal is to have managers who understand the situation and are not making unrealistic demands.  

What is in your control …
Following tests, give  pupils specific advice on how to  improve - in effect using the assessment formatively e.g.  'you have shown you understand how to form the past tense, but you do not do this consistently accurately.  Learn all the verbs which take 'ĂȘtre' in the past tense.'

What is out of your control ...
There may be requirements set by your management at school, and you have to follow their direction.  However, you could refer them to the Ofqual blog and to ASCL's advice.

I hope this helps,.



I paste below some comments I have offered on the Facebook group Secondarymflmatters regarding using sample papers to judge pupil attainment within the 9-1 framework.

A possible approach to getting a rough idea of how your pupils are performing relative to their target grade.

Step 1: 
Imagine if the spec had not changed. Looking at the current cohort, what would you have estimated for
(a) A*-A 
(b) A*-C and 
(c) A*-G? 

[And I don't think you should have to give the an actual past paper to determine this .. it's what I think most schools do at the start of Year 10 .. an estimation ..and as a new teacher, that's where your colleagues / SMT will be helping, based on prior attainment]

Step 2: 
Work out the raw mark which gives you these percentages on whatever set of exams you administer .

This gives you your 'anchor points':
A = 7
C = 4
G = 1

Step 3: Nobody can calculate the intermediate grades as this depends on the raw mark distribution of all pupils taking the exam up and down the country. However, to get a feeling of what they may be, use the method which will be used, i.e.: rule of thumb:

9 =top one fifth of the combined A and A* grades
8 = half way between boundary 9 and boundary 7
6 = top third of the raw marks between a C/4 and an A/7
5 = middle third of the raw marks between a C/4 and an A/7
3 = top third of the raw marks between a G/1 and a C/4

2 = middle third of the raw marks between a G/1 and a C/4

Here are the percentages of candidates who gained the anchor grades in England last year:
French: 7/A 22.7%; 4/C: 68.8%; 1/G - 99.7%
German: 7/A 22.9%; 4/C 74.3%; 1/G 99.9%
Spanish: 7/A 27.1%; 4/C 69.8%; 1/G 99.4%
Obviously you cannot possibly use these percentages and apply them to your cohort, as every cohort will have different prior attainment. The pattern will be especially different if you are in a school where languages are still compulsory. (Not sure there are many of us left in the state sector!).
From this info, last year in England you can see that there would have been differences in the percentage awarded a 9 across the languages.

Out of interest, the percentages needed in tiered maths papers last year were as follows:
AQA Maths higher,
A = 52% of the total marks available (so feel good if you manage to understand half of what you are faced with!) ,
C = 19% of total marks available. (the argument from Ofqual being that those who are of this standard should really be entered for a foundation paper)
AQA Maths Foundation:
C/4 = 52% of the total marks available and
G/1 = 11% of the total marks available.
Edexcel Maths higher
A = 52% of the total marks available (as AQA)
C = 17% of total marks available. (slightly lower than AQA)
Edexcel Maths Foundation:
C/4 = 51% of the total marks available and

G/1 = 11% of the total marks available.