My response to others' responses to the proposed GCSE 2016 criteria for MFL

From reading some initial responses from other people to the proposed GCSE ML subject content and assessment objectives criteria , I fear that there is a confusion between subject criteria for content and assessment objectives,  and regulations about assessment. 

 

It would be extremely helpful to make responses with reference to the previous criteria and not with reference to the exams which the awarding organisations produced.

The features which many people do not like about the current exams (the controlled assessment, the choice that teachers have (understandably) made to encourage rote learning) are to do with regulations about the way the criteria are tested (i.e. the requirement to test the practical elements of writing and speaking by controlled assessment, the constraint that this has to be 60%, and the conditions for the assessment which are 'global' conditions for all GCSEs); they are nothing to do with the criteria for subject content and assessment objectives .

 

The subject content and objectives of 2008 give greater freedom to teachers and awarding bodies than they have ever had, and certainly much more than the proposed 2016 criteria.  This was a particular requirement from the Dearing report - and it was this which drove the QCA and exam boards to require writing and speaking to be tested as CAs. The need to encourage CLIL and imaginative and creative contexts was a key element in the 2008 criteria.

I very much hope that responses will make this distinction, and address the way in which the content is tested when Ofqual launches the relevant consultation.

My response to the Consultation on GCSE subject content and assessment objectives: Modern Languages

Having carried out a thorough analysis of the proposed changes to the subject content and assessment objectives for Modern Languages downloadable in a document here or viewable on a web page here, I have now submitted my response to the 5 questions.

I would really urge people to respond to the consultation here by the closing date (Tuesday 20th August) and to draw attention to the problems, doing so in the light of knowing what the previous criteria were.

Please note also that this is a consultation about what is to be tested, not how it is to be tested.  The Ofqual consultation on how it is to be tested will be taking place later.

 My main concerns are as follows:

(1) lack of definition to allow for recognising full ability range following the GCSE course 

(2) omission of objectives (derive enjoyment and benefit from language learning;  provide basis for practical use of the language)

(3) Lack of differentiation according to language being learnt e.g. languages with non-Roman scripts should have different demands.

(4) Confusion over contexts and settings for different skills

(5) Lack of clarity about the breadth of vocabulary expected and the exact nature of themes

(6) Inclusion of how the subejct should  be tested (this is Ofqual's remit) e.g. mixed skill testing implied by the wording of the assessment obejctives, and the expectation that the target language should be used in testing

The criteria as they stand would make it extremely difficult for Awarding Organisations and Ofqual to ensure valid and reliable assessment in this extremely high-stakes public examination.

Here is my response:

Question 1: Do the proposed content and assessment objectives cover the appropriate knowledge and understanding for GCSE in this subject?

NO

In summary, I give this response for three reasons which I develop at length below.

(1) lack of definition to allow for recognising full ability range following the GCSE course 

(2) omission of objectives (derive enjoyment and benefit from language learning provide basis for practical use of the language)

(3) Lack of differentiation according to language being learnt e.g. languages with non-Roman scripts should have different demands.

 

The purpose of The General Certificate of Secondary Education is to recognise the knowledge skills and understanding of the 16 year olds who take it.

A broad range of ability profile takes the GCSE.

 The proposed content and assessment objectives as they stand describe what might be expected of native speakers.  There are very few instances where allowance appears to be made for the fact that the content and objectives are for people at different stages of learning the language.

One example of a section which makes allowance for non-native speakers is the description of the nature of the speech which is to be understood: 'clear standard speech spoken at normal speed using familiar language across a range of contexts'. ( I assume a native speaker would be expected to understand obscure speech spoken quickly.)  However, this description is at a high level, as for most interactions with a 16 year old, (and indeed anyone who is clearly not a native speaker), native speakers would usually 'slow down' in order to aid comprehensions.   

The lack of guidance for lower levels will be problematic for Ofqual and the Awarding Organisations preparing assessments which still have to cater for the wide range of ability and presumably will have to extrapolate progression towards the goals and define grades for those who do not match the upper boundary descriptions.

 It is essential to value knowledge understanding and skills appropriate to the abilities or experience of the candidates and to have expectations of what can realistically be achieved by the end of a GCSE course.  If this does not happen, the qualification will only attract the very highest ability pupils.

 I would expect the subject content to be described as a 'continuum', to provide indicators to those responsible for assessing the content as to what boundaries might be considered reasonable for a candidate who has followed a GCSE course.

It is essential to be able to describe a broader range of outcomes in a way that can be useful for distinguishing different levels of performance.

I suggest the following changes for each of the sections ( (a) Subject aims and learning outcomes,  (b) contexts and purposes, and (c) scope of study) in order to ensure that the qualification is attainable by a broad ability range of students (thereby satisfying the government expectation that GCSE should still be the standard qualification).

 (a)   Subject aims and learning outcomes

 Aims and learning outcomes can be expressed in a way that makes them realistic for the full ability range.

Many of the aims and learning outcomes are the same in 2008 and 2016. 

Additional in the 2008 criteria and omitted in the 2016:

·         derive enjoyment and benefit from language learning

·         provide basis for practical use of the language

 I strongly recommend that these aims be re-instated - especially the 'practical use of language learning'.

The 2008 aims and learning outcomes (points 6 and 7) are ordered logically (an introductory paragraph summarising overall aims followed by a list of specific skills), are worded clearly and succinctly, and are achievable by pupils of any prior ability.

 The 2016 aims and learning outcomes mix aims and outcomes.  The opening paragraph (which presumably is to describe aims) omits language learning skills, preparation for future study and work, and being able to provide a basis for 'practical' use of language.

 The bullet points describing outcomes describe skills, but by attaching them to specific examples of aims, purposes and contexts this makes them more limited in scope e.g. language learning skills is referenced only to future contexts 'to prepare them for  …' (I would argue for example that language learning skills are useful for current study and for practical purposes).

More importantly the skills are also limited in scope  by reference to degree of accuracy, fluency, range of language, range of contexts, which makes the outcome less achievable by all e.g. listening and reading is described as  'acquire new knowledge, skills and ways of thinking through their ability to understand and respond to a rich range of authentic spoken and written material, including literary texts'  (I would suggest a reasonable outcome for all is to develop understanding in a variety of contexts)

 (b)   Contexts and purposes

Different contexts and purposes appear to be cited for different skills.  There may be a rationale for this, but I cannot detect it.

It will be essential to be clear about the range and nature of contexts and purposes in order to allow for comparability across examining boards and fairness.  Previous criteria have required boards to set a defined content, with a number of words and indication as to 'level' and a percentage of words which can be allowed beyond the defined content.  It seems fair to have some sort of boundary set rather than expecting an infinite number of words to be known and learnt.

It may be helpful to organise the description in a way which spells out which skills relate to which purposes, or which registers of language.  And if the contexts are applicable to all skills, then this could be made clearer.

e.g.

 Which skills are to be used (Point 10 2008)

GCSE specifications in modern foreign languages must require learners to:

• listen and respond to different types of spoken language

• communicate in speech for a variety of purposes

• read and respond to different types of written language

• communicate in writing for a variety of purposes

• use and understand a range of vocabulary and structures

• understand and apply the grammar of the language, as detailed in the specification.

Contexts (for all skills - not confined to specific skills ..): 

(1)   appropriate to their age, interests and maturity levels (so contexts which a 16 year-old might reasonably be expected to operate e.g. when entertaining a TL speaker in this country, when visiting the TL country,

(2)   appropriate for a scholar - when learning about languages and the culture of the TL country and when deepening knowledge and understanding of other subjects studied

(3)   appropriate for someone preparing for the world of future employment 

 The following settings must be included for all skills

·         personal

·         social

·         public

·         academic

·         employment-related

 Skill-specific task requirements (according to the 2016 criteria)

Listening tasks must include response to the following sources:

·         recorded material from authentic sources, including the media [can it be abridged / amended as for literary texts?]

Speaking tasks must include the requirement to give information and ask questions in an unrehearsed situation (e.g. 10 minutes with access to a dictionary to prepare a response to a task)

Reading tasks must include response to the following sources: [can they all be abridged / amended?  or just the literary texts?]

·         personal communication (e.g. letter, emails, texts)

·         public information (e.g. timetables, brochures, newspapers, websites)

·         factual (non fiction?) texts (e.g. leaflet, brochures, magazines, newspapers

·         literary texts: poems, letters, short stories, essays, novels,  plays from any period.  (e.g. … give specific examples of authentic literary texts which would be deemed appropriate for this level (NB may be difficult in French - past historic - even Mr Men books are at a very high level ….)

 Writing tasks must include the requirement to translate text from English into French.

 It would be helpful to clarify the meaning of 'language contexts will be organised in a specified number of themes'

Will the themes only relate to some contexts?

Will all contexts be used for all themes?

(I note that other subjects e.g. science, maths, history are much more specific).

Perhaps it means that themes will be given, and for each theme there will be a series of defined language contexts (e.g. as with 'Language tasks' listed by AQA 2004 linear specification under each theme).    This would then include the 'functions' needed for each Theme/ context.

 

e.g. Theme 1 - My World (identity and culture, lifestyle, culture, beliefs)

 ·         exchange info about self, family, friends and pets: name, age & birthday, nationality, likes & dislikes, appearance, jobs

·         spell out own name, street, town

·         greet someone

·         etc. etc.

 It would be useful to have an example of what is meant by this.

I note that the following contexts from 2008 are omitted: 'GCSE specifications in modern foreign languages must set out contexts and purposes that: 

•         reflect, and are appropriate to, the culture of countries and communities where the language is spoken

•         relate, where appropriate, to other areas of the curriculum'

I believe that the spirit of the 2016 criteria is to maintain the emphasis on the use of 'authentic' sources, and therefore suggest that this is made explicit in this section.

(c)    Scope of study

 The approach seems to be to write bullet points for each skill which are loosely 'hierarchical' in describing the tasks demanded.  This is not explicit and it not a consistent approach across the skills:

 

•                     Some skills have no examples of lower levels of achievement (e.g. writing)

•                     All skills have a majority of statements describing the highest level of achievement.

•                     Few skills describe any 'limits' which might reasonably be expected of a learner (e.g. not totally accurate)

•                     Some skills are associated with a particular task/text type, others are not

There is inconsistency in the way the skills are presented and at the very least, some revision needs to be made to make it clear as to whether people would be expected to demonstrate all these skills in order to be awarded a GCSE.  (If this is the case, there will not be many entrants or certificates awarded).  As it stands, it appears that these are 'requirements' of all candidates.

 Overall, the descriptions are mainly of what you would expect a very high ability pupil to achieve. It may be helpful to give an indication of the continuum / progression of what can be reasonably expected of a 16 year old.

 In the absence of grade descriptors which defined and A C and F grade and provided 'anchors' for those writing a specification and some sort of means of comparison, it is necessary to find some other means of describing the continuum.

 I note that in the mathematics 2016 criteria (maths a similar 'linear' subject) higher levels are defined and denoted by emboldened text.

I have outlined three possible approaches below.

APPROACH 1.  Describe the lowest level, then have an over-arching description of what can be expected at higher levels:

 Learners should have the opportunity to demonstrate the level which they have reached with respect to what they know, understand and can do in different skills.

 For all skills, at higher levels they will be able to

·         cope with a greater degree of unpredictability,

·         operate with less support

·         understand a wider range of vocabulary and  structures, including some unfamiliar language, and

·         operate in a more abstract context

 APPROACH 2 Define the lowest (intermediate?) and highest performance expected for each element which contributes to the skill:

 Listening

 See above for required sources

 GCSE specification should require students to demonstrate:

·         the highest level of response they can achieve (ranging from identifying main points and extracting some details through to identifying details, opinions, and drawing conclusions, deducing meaning/inference, synthesis)

·         the broadest range of contexts they understand (ranging from defined, familiar contexts to a wide range of defined contexts and a few unfamiliar contexts

·         the highest level of complexity of language they understand (ranging from simple language spoken deliberately clearly by a sympathetic native speaker to complex language spoken by native speakers, including adapted authentic media)

 Speaking

GCSE specification should require students to demonstrate:

·         the broadest range of contexts in which they can operate (ranging from defined, familiar contexts to a wide range of defined contexts and a few unfamiliar contexts)

·         the highest level of independent communication they can perform (ranging from giving simple information in highly predictable situations through to initiating, developing and sustaining conversations and discussions in situations which have some unpredictable elements)

·         the highest level of complexity of language they can use (ranging from using simple language in a limited range of topics through to using complex language and extended sequences of speech using a variety of vocabulary across  in a broad range of topics) (Complex is defined in the higher grammar section, and includes a wide range of tenses and moods; range could be defined by the guidance on themes)

·         the highest level of grammatical accuracy they can achieve  (ranging from being usually understandable despite inaccuracies through to being clear despite errors with more complex structures)

·         the highest level of accuracy in pronunciation and intonation they can achieve (ranging from being understandable by a sympathetic native speaker to having reasonably accurate pronunciation and intonation)

·         the highest level of fluency they can achieve (ranging from frequent hesitations to minimal hesitation through use of re-phrasing and repair strategies) (I like the phrase re-phrase and repair!!)

·         the highest level of appropriate use of register (ranging from some awareness and skill in appropriate register to full awareness and skill)

Reading

See above for required sources

 GCSE specification should require students to demonstrate:

·         the highest level of response they can achieve (ranging from identifying main points and extracting some details through to identifying details, opinions, and drawing conclusions inference, synthesis)

·         the broadest range of contexts they understand (ranging from defined, familiar contexts to a wide range of defined contexts and a few unfamiliar contexts)

·         the highest level of complexity of language they understand (ranging from simple, short  text with simple structures deliberately designed to be very clear (e.g. public notices, signs, adapted for learners) through to extended text using complex language (e.g. adapted literary texts)

Throughout: complex is defined with reference to the higher levels specified in the grammar list

Writing

GCSE specification should require students to demonstrate:

·         the highest level of independent communication they can perform (ranging from giving simple information in highly predictable situations through to giving detailed information in a situation which has some unpredictable elements)

·          the broadest range of contexts they can write about (ranging from using vocabulary and structures from a limited, defined prescribed list to a variety of vocabulary and structures form an extended defined list)

·         the highest level of complexity of language they can use (ranging from using simple language through to using complex language and extended sequences of speech) (Complex is defined in the higher grammar section, and includes a wide range of tenses and moods; range is defined by the guidance on themes???)

·         the highest level of grammatical accuracy they can achieve (ranging from being usually understandable despite inaccuracies through to being clear despite errors with more complex structures)

·         the highest level of accuracy in pronunciation and intonation they can achieve (ranging from being understandable by a sympathetic native speaker to having reasonably accurate pronunciation and intonation)

·         the highest level of understanding of register (ranging from some awareness and skill in appropriate register to full awareness and skill)

Language learning skills

GCSE specification should require students to demonstrate use of reference materials e.g. dictionary and grammar book (ranging from finding words to making informed use of verb tables)  [This gives a good justification for including translation skills]

I note that scope of study does not make specific reference to understanding of culture, and I approve of it not being specified in the scope of study. 

APPROACH 3

 (closest to the 2016 approach I think)

Hierarchical statements combining all the elements: (note, these become very long, so it may be worth breaking them down into separate bullet points, but if this is done, it is still essential to indicate what describes each level.  Below I suggest what may be the lowest and the highest, to match the heading of 'scope')

Listening

 •                     Identify main points and extract some details from defined, familiar contexts with simple language spoken deliberately clearly by a sympathetic native speaker (lowest)

•                     Identify details and opinions, draw conclusions, infer and synthesis meaning from a wide range of defined contexts and a few unfamiliar contexts with language spoken by native speakers, including adapted authentic media. (highest)

Speaking

•                     Give simple information in in highly predictable situations covering a limited range of topics using simple language and conveying the message in a way which is understandable to the sympathetic native speaker despite inaccuracies

•                     Initiate, develop and sustain conversations and discussions in formal and informal situations which have some unpredictable elements, using complex language, extended sequences of speech and a variety of vocabulary across  in a broad range of topics, being clear despite errors with more complex structures, and speaking reasonably fluently with a reasonably accurate pronunciation and intonation.

 

Reading

•                     Identify main points and extract some details from defined, familiar contexts with simple language (lowest)

•                     Identify details and opinions, draw conclusions, infer and synthesis meaning from a wide range of defined contexts and a few unfamiliar contexts including adapted authentic literary texts. (highest)

Writing

 

•                     Give simple information in in highly predictable situations covering a limited range of topics using simple language and conveying the message in a way which is understandable to the sympathetic native speaker despite inaccuracies

•                     Give detailed information in formal and informal situations which have some unpredictable elements, using complex language, extended text and a variety of vocabulary across in a broad range of topics, being clear despite errors with more complex structures..

 

Question 2: Is the relative weighting of assessment objectives right?

 I am in agreement with the principle of having a fixed rather than flexible weighting for each skill, as this should lead to a fairer comparison across examination boards. 

 

Note that 2008 had a constraint of 25%/60% rule for practical skills and the Dearing recommendation about Speaking and Writing led to requirement that writing and speaking should be tested under controlled assessment conditions.  There was an expectation that boards  might take advantage of  flexibility to introduce 'mixed skill' testing (e.g. speaking test where 20% - speaking, 10% - listening) however this did not occur and they all offered the identical weighting of 30% speaking, 30% writing, 20% listening, 20% reading.

 There is a case for making the weighting for listening and reading relatively greater, as these skills are the most commonly used and the most useful in all the contexts given. However, speaking and writing are the skills which test 'grammatical' understanding an knowledge.  Ideally I would suggest 20% speaking, 20% writing, 30% reading and 30% listening.

 My main concern here is that this section carries assumptions / instructions about the way in which assessments will be carried out.  I believe that this is Ofqual's remit.

 Firstly, the wording of the assessment objectives implies mixed skills tasks and testing:  'read/listen and respond to, speak/write and interact'. 

 The objectives need to be stated simply.

There is no problem in carrying out mixed skill tasks in teaching and learning activities, but they pose enormous difficulties in assessment situations.  They seriously constrain the means, validity and reliability of the assessment in a test situation and can lead to a double penalty.  (For example, if they hear a text and understand it, but do not understand the target language question and cannot write the answer in the target language, they will not be credited for their listening comprehension. 

Secondly I note that the description of assessment objectives specifies the level reached in the skills  'understand and respond to different types of language / communicate and interact effectively'

 I strongly recommend that the objectives be stated simply as an objective to assess the skills of listening / speaking / reading / writing

 
Thirdly, I note that instructions are given about the means of assessment.  'Assessment objectives will be fully assessed through external assessment: awarding organisations can require teachers to conduct the oral exams, and then send recordings to the awarding organisations for marking. They are internally conducted, but not internally assessed.'

 The decision about the means of testing in the most valid and reliable way is Ofqual's remit.

I am extremely concerned about the prospect of a return to target language testing, even though I am aware that this is a controversial area.  Target language testing leads to convoluted, artificial test types which confuse the candidates in high pressure test situations, and do not allow them to demonstrate what they know, understand and can do with respect to the assessment objectives.  Valid reliable forms of testing leading to fair judgements of candidates' skills are vital for this high-stakes examination.  An examination should not be used to promote a pedagogical ideal. I know that this view is shared by significant people working for awarding bodies who have had the task of setting exam questions and who in the past had to spend inordinate amounts of time and effort finding pictures to represent lexical items clearly in order to avoid the use of English.

 I see no advantage in terms of assessment for using the target language for testing.

Separately, I note that exceptions are given for Languages using logographic systems or characters such as Mandarin Chinese and Japanese.  Given the acknowledgment of relative difficulty in these languages, should not other aspects of the criteria be similarly adapted?  (Allowance made for this in 2008)

Grammatical expectations for French, German and Spanish (p.8)

I note that there is no change to the grammar requirements

 The 2008 specification grammar list categorises and distinguishes core / more complex grammar, though requirement to do so not specified in its criteria. I think it is a good idea to include as a requirement.

 

Question 3: Do the proposed subject content and assessment objectives provide assurance that the essential knowledge taught at the earlier key stages is built upon and represented adequately?

 Yes. 

Question 4: Will the proposed qualifications secure sound progression for the purposes of further academic and vocational study?

No

 As the criteria stand, only the most able will achieve them, and the qualification is therefore only suitable as a preparation for advanced qualifications.

Question 5: Do any of the proposals have potential to have a disproportionate impact, positive or negative, on specific pupil groups, in particular 'protected characteristic groups'? (disability, gender reassignment, pregnancy and maternity, race, religion, belief, sex and sexual orientation); if they have a potential for adverse impact, how can this be reduced?

 Yes.

If Awarding Organisations were forced to implement mixed skill testing (as implied by testing in the target language, and the description of the assessment objectives 'speak/write and interact … listen/read and respond) this would disadvantage those with visual impairment or hearing impairment.

Reformed GCSE subject content consultation June 2013

The Outcome!

I have just completed a first draft of my analysis of the proposed Modern Languages GCSE subject content and assessment objectives for teaching 2016, testing 2018.  It can be viewed  or downloaded from this page  on the ALL London website which I maintain.   May I stress that it contains my own views, not those of the ALL.

The Background.

The government plans to introduce new GCSE criteria for Modern Languages in 2016, for first testing in 2018.

The DfE has published draft criteria for consultation.  The consultation period ends 20th August.

Ofqual will consult in due course on the assessment regulations with respect to ML.  It is currently consulting on other E-Bac subjects who are in the first phase.  (2015 for first testing in 2017)

Previously there was one body responsible for both curriculum and assessment (QCA)  This has been disbanded.  A  really important process difference this time from previous changes is the very clear and distinct separation between the description of the subject content (DfE) and the way in which it is to be assessed (Ofqual).  While looking at the subject content we need to ask ourselves:

 'Is it subject content which is being described, or is it an assumption about how it is to be assessed?' e.g. it should not include a statement about whether or not the Target language is to be used in assessment.

 

Everyone needs to be really clear about the fact that since ML is in the second phase, the assessment aspect is not being consulted on currently for languages.  I think that this is actually very helpful because it allows us to enter into a constructive dialogue while the thinking is still taking place.    

It is obviously critical to analyse the proposals for subject content and assessment objectives in detail to ensure that they can be easily interpreted by Ofqual and Awarding Bodies such that valid and reliable assessments can be designed  which fairly test what candidates know, understand and can do. 

Last weekend, I pored over the proposed 2016 criteria, comparing it with the 2008 criteria, and trying to analyse the rationale for and potential impact of the proposed changes.

The result is 21 pages of analysis, commentary and questions. I have also included an 'appendix' which contains all my initial cross referencing and comments, and this may be useful for anyone else who wants to undertake a similar exercise.

I would really welcome any comments  about my interpretation / comments / questions.  I will publish my formal response to the consultation in due course!