Ofqual consultation - MFL GCSE assessment: Urgent need for response from as many people as possible - especially Question 10!
The Ofqual consultation regarding assessment of GCSE languages for the GCSE subject content to be first taught 2016 and first tested 2018 is now published and responses are needed by
Friday 23rd May 2014.
I think it is a well constructed document stating clear rationale for its proposals. After each explanatory section, there are clear statements with option of responding 'strongly agree, agree, disagree, strongly disagree and don't know/no opinion' or 'Yes/No'. It is optional to add text.
Go to page 40 to see the response section Pages 40-52 containing the 26 questions which you can fill in and send off if you prefer not to do it online.
I urge you to register your response so that you can support good decisions or challenge poor decisions. It may seem daunting, but it would be sufficient just to indicate the extent to which you agree to 26 statements. (So possibly just 15 minutes). In my experience, organisations DO take notice of us when enough of us make the point.
I have written a response on behalf of our school and there is a copy to view / download here (stored on my google drive). I have also copied and pasted it below.
Question 10 - An end to mixed tier entry?
To me, the most important element to challenge is the proposal that students should be required to enter for EITHER higher OR foundation-tier assessments, but not a combination of the two. So it would no longer be possible to enter, say, speaking and writing foundation, and reading and listening higher level. This would be detrimental for a significant number of 'mid-range' students, and especially punishing for those with dyslexia. Please communicate this to Ofqual by filling in the consultation, and 'strongly disagree' with this proposal which is in Question 10.There is an opportunity to give reasons for your responses. I have taken the opportunity to reiterate points made to Ofqual in the past on Marking criteria in the context of MFL, [download word doc here] and I did a quick analysis of previous GCSE and O level 'test types' to help me answer the section on validity, reliability and fairness. You can view/download it here: analysis of the relative validity, reliability and fairness of current and former GCSE test types
Background
The DfE consulted on GCSE subject content and assessment objectives for Modern Languages in June 2013. I gave a full response in my blogpost here.The DfE published the GCSE subject content and assessment objectives for Modern Languages in April 2014 with the following explanation on the front cover: 'Final content for this subject will be puiblished when Ofqual has consulted on and made decisions about assessment arrangements for GCSE languages'. I have created a 9-page word document which compares directly the June 2013 with the April 2014 criteria [Download here]
RESPONSE COPIED AND PASTED BELOW: (26 questions highlighted to aid skimming!)
1.
Reading
skills should be assessed using exams set and marked by the exam boards. To
what extent do you agree or disagree this statement?
(x ) Strongly agree
( ) Agree
( ) Disagree
( ) Strongly disagree
( ) Don’t know/no opinion
Please give reasons for your answer:
Setting valid and reliable
reading exams is a highly skilled job. This is the most straightforward and
fair way of ensuring fair comparison across the cohort.
In order to make the test a
valid test of reading, the questions need to be in English. Testing using the target language makes the
test of a discrete skill invalid (e.g. a candidate may understand what is
written in a text, but because they may not understand or misunderstand the
rubric, they may fail to demonstrate this OR a candidate may make a guess at
the answer and write it in the target language without understanding its
meaning. It is vital to have questions
in English (as for the 'harder language' exams)
Having said this, we assume
that the DfE criteria cannot be changed now, so would urge consideration of the
following principles when applying the requirement to use a percentage of
target language testing:
(1) Give great thought as to
which type of questions can maintain validity the closest e.g. ensure that
questions set are of a lower language complexity than the language being tested
so that it is totally reasonable to expect that candidates can understand the
question and know what they have to show they can do.
(2) Given that the decision has
been made not to have mixed skills testing, you need to be constantly asking
'is this question testing skills other than the one we are meant to be
testing?' Do not penalise for the quality of written response. Test only for evidence that the text has been
understood.
2 Writing skills should be assessed using exams set
and marked by the exam boards. To what extent do you agree or disagree this
statement?
( x) Strongly agree
( ) Agree
( ) Disagree
( ) Strongly disagree
( ) Don’t know/no opinion
Please give reasons for your answer…
Taking the task in
examination conditions with a high level of control (invigilators, silence)
ensures fairness. Making comparative
judgements about quality of writing is a highly skilled job.
Although we do not
recommend target language testing, we are pleased that the phrasing used with
respect to the written element seems to comply with the principle that pupils
need to be able to understand very clearly what they are expected to do.
3 Listening skills should be assessed using exams set
and marked by the exam boards. To what extent do you agree or disagree with
this statement?
( x) Strongly agree
( ) Agree
( ) Disagree
( ) Strongly disagree
( ) Don’t know/no opinion
Please give reasons for your answer…
Setting valid and reliable
reading exams is a highly skilled job. This is the most straightforward and fair
way of ensuring fair comparison across the cohort.
In order to make the test a
valid test of listening, the questions need to be in English. Testing using the target language makes the
test of a discrete skill invalid (e.g. a candidate may understand what is
written in a text, but because they may not understand or misunderstand the
rubric, they may fail to demonstrate this OR a candidate may make a guess at
the answer and write it in the target language without understanding its
meaning. It is vital to have questions
in English (as for the 'harder language' exams)
Having said this, we assume
that the DfE criteria cannot be changed now, so would urge consideration of the
following principles when applying the requirement to use a percentage of target
language testing:
(1) Give great thought as to
which type of questions can maintain validity the closest e.g. ensure that
questions set are of a lower language complexity than the language being tested
so that it is totally reasonable to expect that candidates can understand the
question and know what they have to show they can do.
(2) Given that the decision has
been made not to have mixed skills testing, you need to be constantly asking
'is this question testing skills other than the one we are meant to be testing?'
Do not penalise for the quality of written response. Test only for evidence that the text has been
understood.
4 Speaking skills should be assessed by non-exam
assessments, using tasks set and marked by the exam board. To what extent do
you agree or disagree this statement?
(x ) Strongly agree
( ) Agree
( ) Disagree
( ) Strongly disagree
( ) Don’t know/no opinion
Please give reasons for your answer…
External examining would
ensure the highest level of control, but I appreciate that this would be extremely
costly.
5 What considerations need to be taken into account
to make sure students’ speaking skills are assessed in a way that is:
Please see appendix for
detailed analysis of the extent to which current and past assessment types for
all skills are valid, reliable or fair. We
summarise principles below.
a)
valid
· Ensure that assessments
test what they are meant to be testing and allow all pupils of all abilities to
demonstrate what they know, understand and can do, across the full range of
speaking skills (communication, independent interaction, pronunciation &
intonation, fluency/confidence, accuracy), across the full range of authentic
situations (e.g. ones which are highly predictable to ones which are not
predictable) and across the full range of authentic 'support' which can be
given (e.g. glossaries of phrases/words, wordlists/dictionaries, prompting from
a sympathetic native speaker through to
surviving with minimal support)
· Design range of test types
to suit the aspect of the skill being tested e.g. supplying a common text to be
read aloud, thereby testing pronunciation only.
· Recognise that the ability
to understand a question is not a test of speaking ability, so ensure that
tasks and assessment criteria do not penalise candidates unduly for this (e.g.
allow re-wording, prompting, quickly moving on to another question where
candidate can demonstrate abilities, do not penalise hesitation when a
candidate has not instantly understood a question).
· Allow a reasonable amount
of time / number of opportunities for candidates to show what they know
understand and can do. [Try to avoid a
high-stakes '7 minutes' at the end of the course if at all possible]
………………………………………………………...………………………………….
b)
reliable,
Common shared criteria for all exam boards.
(See appendix on marking criteria - response to Ofqual questions).
·
Establish very
clear common subject criteria with interpretation shared by exam boards
·
Ensure there is a clear and explicit reference to a benchmark (e.g. the
GCSE core subject specification), that they reflect the subject criteria and
content fairly (do not include descriptors which describe AS/A2 performance)
·
Ensure that criteria are clear, transparent and as objective as possible
to avoid a subjective response (include full descriptions of all strands for
each 'band' - do not have gaps; be very clear about how apparently 'subjective'
qualities are tested (e.g. 'spontaneity', ''creativity' , 'independence') and
if they cannot be defined, do not include them.
·
Indicative
content (samples) to demonstrate meaning of quantitative and qualitative words
(e.g. long/short, basic/ higher, complex / simple / original; if samples cannot
be given, omit the words.
·
Ensure consistency and lack of contradiction in descriptions of bands of
performance
·
Consider how to
deal with language which may not conform to the definitions of a scale of levels .. e.g. someone may have grasped
something which may be seen as 'complex' (e.g. perfect tense) but may make
mistakes in something defined as 'simple language' (e.g. basic present tense) Recommendation: Take a
positive approach. Reward what is demonstrated, allow the benefit
of the doubt (e.g. do not penalise for incorrect ambitious language or
incorrect understanding of a question (listening comprehension).
·
Identify what range
of language, grammar and structures constitutes language worthy of 'higher
levels' (avoid obsession with tenses / opinions and justifications which can
lead to an extremely 'formulaic' approach in order to 'tick the boxes') [Note
that the 2008 criteria specifically did not 'require' three tenses for every piece of written work
in order to qualify for higher grade. Higher
level language can be represented in more sophisticated ways than just using
three tenses, as evidenced in authentic literature]
·
Require a field
test to establish how fairly the mark scheme can be applied and a moderation
process which ensures that if 2 people mark the same piece of work
independently they reach exactly the same / agreed tolerance of x points
·
Be aware of the
risk of a marking scheme which prevents candidates form getting top marks e.g
where 4 sub-criteria of a skill are each marked out of 5, and where examiners
are reluctant to award the top band, there will be a 'bunching' of marks around
16/20 [This has been identified as one
of the possible reasons for the relative lack of A* at A level]
·
Ensure a common approach across examining boards to any
'limiting' factors (e.g. to what extent is accuracy penalised if communication
is weaker ) [This has been an issue at A level, where one board will not award
marks from the top band for accuracy if the top band for content is not reached,
whereas another board does allow this]
·
Ensure a common
approach to how to deal with pupils who are over-ambitious, e.g. how can you give equal credit for what they
know understand and can do and avoid penalising them for attempts to be more 'original' which may
lead to a higher proportion of inaccuracy compared with the work fo a candidate
who has adopted a more 'standard' approach
·
Ensure that tasks match the criteria
………………………………………………………...………………………………….
c)
fair?
Ensure equal conditions for all candidates (same rules, whatever the board,
with respect to predictability and support; system for checking schools'
compliance with rules when teacher is examiner (e.g. opening of test papers, degree
of support)
Ensure equal standard of marking, whatever the board (e.g. through
having common criteria and cross-board moderation checking, including teachers
in standardisation meetings) - see above
………………………………………………………...…………………………………..
6 How might any aspects of the proposed assessment
requirements impact on:
a)
the costs,
and
Conditions requiring higher levels of control / supervision /
invigilation and a recording of all tasks increase staffing costs [cost of
cover when teachers need to administer tests simultaneously to protect
confidentiality, less flexible time slots mean that centres cannot take
advantage of specific periods when teachers have lower contact time]. Currently no invigilators needed for test and
only one task needs to be recorded formally so the cost is covering the teacher
lesson.
External examiners would be more costly to the exam board, however, this
method has been used in the past.
Teachers are currently 'unpaid' examiners for the exam boards.
……………………………………………………...……………………………
b)
likely
take-up of new modern foreign language GCSEs?
If all tasks are
unpredictable and have no support, they will be extremely difficult for lower attainers
who will not be able to demonstrate the lower order speaking skills they have e.g.
ability to communicate with use of a guide book / glossary, ability to
pronounce, ability to 'recall' and memorise (note that memorisation is a valid
skill)
There is a danger that we
revert to the situation pre-current GCSE when teachers were complaining
bitterly about candidates having to memorise 16 topics. This can be very off-putting for all
abilities
If the ML exam ends up
being more expensive than other exams, this could be a disincentive for
managers to encourage students to take languages
7 The outcome of the speaking component should
contribute to a student’s overall grade. To what extent do you agree or
disagree this statement?
( x) Strongly agree
( ) Agree
( ) Disagree
( ) Strongly disagree
( ) Don’t know/no opinion
1.1
Please give reasons for your answer
We strongly agree with point 2.14
"For modern foreign language GCSEs the ability to speak the language is a
key aspect of the qualification. Modern foreign language GCSEs are designed to
develop and recognise students’ skills in a second or additional language. This
is in contrast to English language GCSE which is designed to be taken by
English speakers. "
8 All assessments (reading, writing, listening and
speaking) in new modern foreign language GCSEs should be tiered. To what extent
do you agree or disagree this statement?
(x) Strongly agree
( ) Agree
( ) Disagree
( ) Strongly disagree
( ) Don’t know/no opinion
Please give reasons for your answer
Firstly, we must challenge
your assertion in 3.7. Much thought has
been given to the rationale for tiering. (Helen Myers was heavily involved in
the last GCSE revision as one of the two teacher representatives invited by QCA
to meetings with exam boards, and tiering
certainly was not adopted because of 'historical
practice'. It was based on considering
what was the fairest way of assessing candidates of all abilities.
A single exam covering all
grades lasting, say, 40 minutes would not give adequate opportunity for
students to demonstrate what they know understand and can do and would be
particularly damaging for the least able for whom under half would be
applicable. (However, the more able do
not suffer from 15 minutes of relatively easy questions)
The most valid and reliable
method for assessment would be to revert to the method in 1988. All take short foundation tier in each
skill. It is then optional to take a
further higher tier paper in any or all of the skills. From the candidate point of view there is no
'gamble' as to which level to take, and
no feeling of being 'pigeon-holed' at an early stage of the course. In our experience, higher level students are
not demotivated by sitting tests which they find easy, and are intelligent
enough to understand the reasons for the system when this is explained. If lower level students attempted the higher
and found it difficult, they were not demotivated as they had already taken the
foundation tier and succeeded in it.
From the exam board point of view, there is no problem about
comparability between foundation and higher levels to secure the same grade, and
obviously no need for an overlap section).
Thus, the approach we strongly recommend is to have a '2-part' test for
each skill. This would take up more time
and involve additional marking, but this is achievable (-it worked in 1988-)
and electronic means of marking could speed up the process of marking. The academic benefits far outweigh the organisational
disadvantages.
Failing this, we would strongly
support tiering, with an overlap between higher and foundation tiers. It would be essential for listening and
reading in order to give candidates the maximum opportunity to demonstrate what
they know understand and can do. (e.g. if equivalent grades G - C were tested
in just half the paper, say 20 minutes, this would not be a valid enough
'sample').
It is less essential in
speaking and writing where feasibly a more open-ended task could be set with
guidance on 'points which could be included'.
However, we would welcome the opportunity to have more focussed tasks
which are more accessible to the lower attainers and do not rely on an 'open-ended'
response and memorisation. Below we
summarise the ways in which different tiers could make tasks more relevant for
the ability profile by offering a range of predictability and support.
Foundation speaking: read aloud a pre-learnt passage; carry out a predictable role play with
support of a glossary; carry out a predictable conversation from a defined
range of topics with support of a glossary
Higher speaking: carry out a less predictable role play with limited glossary; carry out
a less predictable conversation/simulation from a wider range of topics with
limited / no glossary
Foundation writing: Write a list of notes (with support of glossary); reply to a formulaic
straightforward letter (with glossary) [All structures and topics required from
foundation list only]
Higher writing: Write answer to a more challenging letter (with limited glossary) frpm
a wider range of topics; write a document demonstrating higher level functions
and structures (to interest, persuade, inform, convince) and expressing
directed content (so in the form of 'translation')
Foundation reading: Tasks based on texts which reflect foundation topics and structures;
translation into English with support of glossary
Higher reading: Tasks based on texts which reflect higher topics and structures and
require more skills of inference; translation into English with support of
limited glossary
Foundation listening: Tasks based on passages which reflect foundation
topics and structures (defined); Opportunity to listen to texts more frequently
/ passages broken up more
Higher listening: Tasks based on passages which reflect higher topics and structures
(defined); Longer passages; Requirement to infer meaning
9 All available new modern foreign language GCSEs
should be tiered. To what extent do you agree or disagree with this statement?
( x) Strongly agree
( ) Agree
( ) Disagree
( ) Strongly disagree
( ) Don’t know/no opinion
Please give reasons for your answer
We agree with the rationale
you give in 3.19 We are providing a General Certificate of Secondary Education,
rewarding what people achieve through being at secondary school. We should be encouraging non-native speakers
to be learning community languages. (e.g.
especially those in the caring / service industries e.g. nursing)
10 Students should be required to enter for either
higher- or foundation-tier assessments but not a combination of the two. To
what extent do you agree or disagree this statement?
( ) Strongly agree
( ) Agree
( ) Disagree
( x) Strongly disagree
( ) Don’t know/no opinion
Please give reasons for your answer
A significant number of
students have SEN with respect to literacy and can perform significantly better
in speaking and listening tasks. They
would be heavily disadvantaged by such a system.
Para 3.29 is not necessarily
logical. Candidates can be very 'thrown'
by a paper where they do not understand very much, and can 'underperform' as
they 'give up'. In mock exams we
frequently find pupils who gain a grade D in the higher and a grade C in the
foundation.
Para 3.31 seems controversial. What evidence do you have to suggest that
Category 2 students are deterred from improving their reading and writing
skills because of the option to take a foundation paper?
If the proposal to remove
the option of mixed tier testing were taken forward, we believe it would have a
very serious impact on the popularity of and take-up of languages. It would not pose a problem for the more able
of-course, but would pose a further deterrent for the less able, who would be
forced to take foundation papers in all skills if they had relative weaknesses
in any one of the skills.
11 For the listening, reading and writing
assessments 20 per cent of marks, and for speaking 50 per cent of
marks, should be allocated to questions or tasks that are common in any series
to both the foundation and higher tier assessments. To what extent do you agree
or disagree with this statement?
( ) Strongly agree
( ) Agree
( ) Disagree
( ) Strongly disagree
( x) Don’t know/no opinion
Currently the overlap for
listening and reading allows for G - C and E - A* - so the equivalent of three
out of the available 8 grades. This
seems to work adequately. On a 1-9 point
scale therefore, we would expect the overlap to be 2-3 points.
12 Do you have any further comments on the tiering
of modern foreign language GCSEs?
…………………………………………………………………………………………………………………………………………………………………………………………
13 New GCSEs in ancient languages should be
assessed wholly by examination. To what extent do you agree or disagree with
this statement?
(x ) Strongly agree
( ) Agree
( ) Disagree
( ) Strongly disagree
( ) Don’t know/no opinion
Please give reasons for your answer…as for modern languages
(see above)
14 New ancient language GCSEs should not be tiered.
To what extent do you agree or disagree with this statement?
( ) Strongly agree
() Agree
() Disagree
( ) Strongly disagree
( x) Don’t know/no opinion
Please give reasons for your answer
Your points in paras 5.2, 5.3 and 5.4 explain the situation well, and reflect
the position in our school where only the most able and committed take
Latin. However, if a tiered approach
were to be considered on balance more appropriate, we would not complain at
all!
Availability of modern foreign language GCSEs
15 Modern foreign language GCSEs in a wide range of
languages should be available in the future. To what extent do you agree or
disagree with this statement?
( x) Strongly agree
( ) Agree
( ) Disagree
( ) Strongly disagree
( ) Don’t know/no opinion
Please give reasons for your answer…
GCSE is the recognised
'currency ' for employment and opportunity and this should be as widely
available as possible
16 Modern foreign language GCSEs for which there is
low demand should be available in the future. To what extent do you agree or
disagree with this statement?
( x)
Strongly agree
( ) Agree
( ) Disagree
( ) Strongly
disagree
( ) Don’t
know/no opinion
Please give
reasons for your answer…
GCSE is the recognised
'currency ' for employment and opportunity and this should be as widely
available as possible
…………………………………
17 Modern foreign language GCSEs should be
available for students who are existing users of the language. To what extent
do you agree or disagree with this statement?
() Strongly
agree
( x) Agree
( ) Disagree
( ) Strongly
disagree
( ) Don’t
know/no opinion
Please give
reasons for your answer…
This can reward students for their abilities, and encourage them to
maintain and even improve on home language (e.g. requiring all four skills)
18 In your opinion would schools and colleges be
willing to pay a higher fee to enter students for modern foreign language GCSEs
for which there is a lower demand?
Yes
19 What, if any, steps do you think Ofqual should
take to secure the availability of GCSEs in a range of modern foreign
languages?
It seems reasonable to come
to an arrangement with the Awarding Organisations such that they 'share out'
the provision for less widely taught languages
20 A range of ancient language GCSEs should be
available in the future. To what extent do you agree or disagree with this
statement?
( ) Strongly
agree
( ) Agree
( ) Disagree
( ) Strongly
disagree
( x) Don’t
know/no opinion
Please give
reasons for your answer…
This does not affect our school
21 What, if any, steps should Ofqual take to secure
the availability of GCSEs in a range of ancient languages?
…………………………………………………………………………………………….
22 A disabled student should obtain an exemption
for no more than 40 per cent of the available marks for a modern foreign
language GCSE. To what extent do you agree or disagree with this statement?
( ) Strongly agree
( ) Agree
(x ) Disagree
( ) Strongly disagree
( ) Don’t know/no opinion
Please give reasons for your answer…
We are not sure why 40 %
has been selected. Could there be an
allowance for recognition of achievement in any of the four skills attempted
according to the protected characteristic affected? It seems arbitrary that it
is possible to accredit short course listening and speaking OR short course
reading and writing but not to allow other combinations.
23 We have identified a number of ways the proposed
requirements for new GCSEs in modern foreign languages may impact (positively
and negatively) on persons who share a protected characteristic. Are there any
other potential impacts we have not identified?
Yes
If yes, what are they? …
(1) Para 7.15 rightly identifies
the benefits of tiering for those who wish to learn languages for religious
purposes, but if prevented from mixing those tiers, they could be prevented
from demonstrating the full range of their skills
(2) Dyslexic students would
suffer greatly from a requirement to take all foundation or all higher
papers. Typically they do relatively
much better in listening and speaking than in reading and writing. A common pattern for them is to enter higher
listening (for example being able to gain a high 'C' and foundation reading
24 We have not identified any ways by which the
proposed requirements for new GCSEs in ancient languages may impact (positively
or negatively) on persons who share a protected characteristic over and above
those impacts that apply to the changes to GCSEs generally. Are there any
potential impacts we have not identified?
No
If yes, what are they? …………………………………………………………………..
25 Are there any additional steps we could take to
mitigate any negative impact resulting from these proposals on persons who
share a protected characteristic?
No
If yes, what are they? …………………………………………………………………..
26 Do any of the proposals or options being
considered have financial or wider resource consequences, positive or negative,
for:
Schools Yes
Exam boards Yes
Others Yes/no
Please explain your response…
Provision of someone to
read aloud texts for visually impaired would add additional cost to staffing
and rooming.
No comments:
Post a Comment