30 March 2005
Media Statement

Government responds to Scholarship Review

The government has endorsed a series of recommendations made by an expert panel looking at the New Zealand Scholarship exams.

Associate Education Minister David Benson-Pope says changes recommended by the Scholarship Reference Group will deliver the certainty the government, parents, and students require from New Zealand Scholarship.

"The government intervened earlier this year because scholarship exams are an important mechanism for recognising top academic performance, and as a means for distributing significant financial awards," said Mr Benson-Pope. "The unacceptable variability in the 2004 results created an obvious unfairness that had to be addressed."

Mr Benson-Pope says the key recommendation of the review group is that scholarship should be awarded to a set percentage of students in every subject. The review group recommends this percentage be set at a figure between two to three per cent of the total number of students studying a subject at NCEA Level 3.

A National Scholarship Monitoring Panel will be established to advise NZQA on the implementation of these recommendations. Proposed membership and terms of reference for the panel will be presented to Cabinet at the end of May. Final decisions on the target percentage will be made in conjunction with the panel.

Mr Benson-Pope says some departure from this target may be necessary for individual subjects to preserve the integrity of the exam. This might happen in subjects where there are small numbers of students taking a subject or where there are low scholarship entry levels.

"The recommended changes will require students to be ranked, and we have been assured that this is possible within a standards-based system providing the examination is structured appropriately," said Mr Benson-Pope.

Mr Benson-Pope says Cabinet has endorsed all but one of the reference group's 26 recommendations. It has deferred a decision on whether all students gaining a scholarship should get a financial award until the Ministry of Education look at the issue. They will report back by 30 May on this issue.

The minister paid tribute to the reference group: Hohepa Campbell, Terry Crooks, Kate Gainsford, John Langley, Margaret McLeod, Don McLeod, Luanna Meyer, John Morris, Roger Moses, Ray Newport, Tim Oughton, and Graham Young.

Q and As: New Zealand Scholarship

What is the essential difference between scholarship 2004 and what is being proposed for 2005?
The key recommendation of the review group is that scholarship will now be awarded to a set percentage of students in each subject. The review group recommends this percentage be set at a figure between two to three per cent of the total number of students studying a subject at NCEA Level 3. Some departure from this target percentage will be possible to preserve the integrity of the exam to take account of factors like small numbers of students taking a subject; subjects with low scholarship entry levels; or evidence about the overall performance of students in a subject.

Is this a fairer way of allocating scholarship money?
Yes. The solution recommended ensures there is a similar proportion of scholarship awards in each scholarship subject (relative to the number of students studying that subject at NCEA Level 3). This means that students performing at the top of their subject all have an equivalent chance of receiving a scholarship.

Is this still a standards-based assessment?
Yes. The Scholarship Review Group neither recommends for nor against a standards-based examination. What they have requested is an examination that is designed in such a way so that "student performance be assessed against an assessment schedule that ensures a ranking of candidates is produced by marks or grades". The government's policy of having standards-based assessment for NCEA and for the New Zealand Scholarship remains unchanged. Standards-based assessment can be used to meet the goals of the Scholarship Reference Group, as long as it has a more finely graduated assessment schedule to produce a ranking. The Scholarship Review Group further recommends "inter-subject moderation or scaling not be used". This is also compatible with a standards-based assessment system.

Some commentators say you can't have a ranking system with standard-based assessment - that this undermines the new system?
A number of examination systems are compatible with a ranking system. Standards-based assessment is also able to include a ranking system. The existence of 'outstanding performance' and 'scholarship' in the 2004 New Zealand Scholarship was a form of grading. Multiple grades are possible while still requiring a standard to be met.

What will happen beyond 2005?
The government will continue to monitor and respond to feedback about the changes we are implementing this year. The Ministry of Education will report to Cabinet on the awards criteria for the transition year 2005, by June 2006, and on full implementation by June 2007.

How will the integrity of the new system be monitored?
A National Scholarship Monitoring Panel will be established to provide consistency and cohesion to the New Zealand Scholarship, and to give assurance to, and confidence in, NZQA and Ministry of Education processes. This panel will consist of two advisory groups: The Monitoring and Implementation Group (SMIG) will oversee and advise on technical processes; and the Independent Advisory Group (IAG) will have an advisory and assurance role.

What happens if 2% to 3% of the students don't actually meet the Scholarship Standard?
Standards and exams will have to be designed on the basis of 2% to 3% of students studying each subject at NCEA Level 3 will achieve the scholarship requirements. However, if there is clear evidence that less than 2% to 3% have met the requirements of the exam, then the SMIG will only award scholarships where the standard has been met, which may result in fewer passing.

Does this mean that standards are going to have to be rewritten?
Possibly. Some standards may have to be rewritten to meet the objective of 2% to 3% of students meeting the requirements of the exam. Clearly some of the science exams were too hard in 2004 and the standards and/or assessment schedules will need to be adjusted. The exams will also have to meet the requirement of being marked in such a way as to produce a ranking order.

Can you implement these changes in time?
Yes. The Scholarship Reference Group's recommendations were made on the basis that they were deliverable for the 2005 examinations. Where matters need more time they have been explicitly given a longer time frame, for example, policy development with regard to requisites (co-requisites or pre-requisites required for sitting scholarship).

Are the exams already set for 2005 and does this mean that they will have to be rewritten?
NZQA have undertaken to change or adjust any exam where this is required. They are able to do this up until as late as August, when printing of exam papers normally begins.

Will there be any variability in the 2005 scholarship exams?
Yes. Within the 2% to 3% target for scholarship awards there is some defined flexibility built in to the Scholarship Review Group's recommendations. There is a margin of plus or minus 5 scholarships to give some more leeway for subjects with small numbers of students enrolled (e.g. Latin). Also, the scholarship standard must be actually met by the lowest ranked student who meets the standard - this will allow for some further variability.

What if scholarship entry levels for a particular subject are low or uneven, or even close to the 2% to 3% of the students enrolled?
Cabinet has agreed that there should be an additional mechanism to safeguard the integrity of the scholarship exams in the event of low or uneven enrolment patterns. There is a further ability to vary scholarship award numbers by a clearly defined margin (e.g. +/-1%) to account for uneven entry behaviour. This is particularly seen as a mechanism to apply over the 2005 transitional year.

Will there still be a top scholar award?
The Scholarship Reference Group recommends that the top subject scholars continue to be recognized. They also recommend that a new Premier academic award be established that recognizes the achievement of top scholars based on at least three Scholarship subjects, along with their full record of academic performance at school. Such an award would be limited to 5 to 10 recipients. Details of such an award will need to be further developed.

Where did the 2004 New Zealand Scholarship exams come from?
In the early 1990s scholarship was discontinued as a separate examination. The discontinued exam system had used a scaling system when the raw results produced too few or too many students achieving at high levels. This sometimes resulted in movements up or down of up to 10%. In 1998 Cabinet decided to reintroduce a national scholarship examination. Then Minister Wyatt Creech wrote in the Cabinet paper introducing the new qualifications system, which included introducing a scholarship exam at Level 4 of the NCEA: "There will be no inter-subject scaling of the external assessment. This has been a source of dissatisfaction for some time because of the distortions that occur between raw and final marks." The newly formed New Zealand Scholarship was viewed as being at the same level as the first year of study at university. The system was not designed to result in a consistent predetermined number or proportion of scholarships being awarded. For each scholarship subject, two levels of achievement were to be awarded, "scholarship", or "outstanding performance". This is what was offered for the first time in 2004.

What happened in 2004?
In 2004, of the 33,805 eligible year 13 students, some 4,624 (13.7%) entered the new scholarship examinations. There were 7,655 candidate subject entries in 2004, varying from 19 candidates in Latin to 1,007 in Physics. Some schools entered a relatively high proportion of their Year 13 students, others a very small proportion. There was also significant variability in the percentage of the total of level 3 students entered in different subjects. For example, 5% of the total entered the English examination, with 15% and 16% entering Chemistry and Physics respectively. The results of these students showed an unacceptable level of variation with achievement rates in some subjects, particularly the sciences, being very low. In 2003 UEBS the pass rate for scholarship ranged from 0.8 to 8.2%. Pass rates ranged from 0% in media studies, to 17.4% in Latin. In total 1724 scholarships were awarded.

What was the government's response?
The government acted quickly to ensure the variability would not impact on students seeking entry to university programmes. In addition, in those subjects where students had done exceptionally well at NCEA Level 3, and the subject was under-represented at scholarship, the government introduced a 'Distinction Certificate' in recognition of student achievement. At the same time it introduced a new 'Distinction Award', meaning students who had a combination of scholarship passes and distinction certificates in any three scholarship subjects, would be eligible for an award of $1500 per year for three years. In total, an estimated 372 Distinction Certificates were created.

Why did the government instigate a review of the New Zealand Scholarship system?
New Zealand students who compete at the top level are entitled to know they are sitting an exam that is credible. The wide and unacceptable level of variation in the 2004 results led the government to believe that the newly formed New Zealand Scholarship was not delivering as a mechanism for recognising top academic performance, nor as a means of fairly distributing significant financial awards. For this reason the government created the expert Scholarship Reference Group to look at the scholarship system.

Who formed the Scholarship Reference Group?
- Hohepa Campbell (Te Runanga Nui o Nga Kura Kaupapa Maori o Aotearoa)
- Professor Terry Crooks (Co-director Education Assessment Research Unit, Otago University)
- Kate Gainsford (Post Primary Teachers' Association)
- Dr John Langley (Dean of the faculty of Education, Auckland University)
- Margaret McLeod (Principal of Wellington Girls' High School)
- Don McLeod (Post Primary Teachers' Association Principals' Council)
- Professor Luanna Meyer (New Zealand Vice Chancellors' Committee)
- John Morris (Principal of Auckland Grammar School)
- Roger Moses (Principal of Wellington College)
- Ray Newport (New Zealand School Trustees' Association)
- Tim Oughton (Independent Schools New Zealand)
- Graham Young (Secondary Principals' Association of New Zealand).

The Reference Group had to act quickly. How much consultation was there time for with the education sector?
The Reference Group included representatives of the key stakeholder interests from the senior secondary education sector, to ensure that they were not just listened to, but were actively involved in the decision-making process. The Reference Group met as a group for four days over a two-week period. In between sessions, members discussed issues with their colleagues and associations, and considered a draft report from the first session and papers from the Scholarship Review Group.

How do you know this is what schools want?
The majority of members of the Scholarship Reference Group were in fact current school principals representing a spectrum of state and independent schools. There was clear agreement from the Scholarship Reference Group as to the recommendations.

Has the government accepted the Reference Group's findings?
Yes. The Cabinet has accepted 25 of the groups 26 recommendations outright. Cabinet decided to defer a decision on whether all students gaining a scholarship should get a financial award until the Ministry of Education report back (by 30 May, 2005) on the implications of this recommendation.

Reviews into New Zealand Scholarship and NCEA:

The government initiated two reviews. The first was the one carried out by the Scholarship Reference Group, who were looking solely at formulating improvements to the NZ Scholarship exam system.

A second inquiry is being conducted by the State Services Commission. They are also looking at issues around the 2004 NZ Scholarship exams and systems put in place by NZQA. They are expected to report back on 29 April 2005.

However, this review also has scope to look beyond scholarship at other aspects of the assessment system. The Minister recently wrote to the State Services Commissioner to confirm that his office would include issues to do with variability in a small number of achievement standards within NCEA Levels 1-3 in their inquiry. The government seeks assurance that this variability is a result of educational issues and not systems failure.

In addition the Board of the New Zealand Qualifications Authority has advised Associate Education Minister David Benson-Pope of its intention to undertake an internal review of processes relating to 2004 New Zealand Scholarship.

These inquiries are additional to a stocktake into the implementation phase of NCEA Levels 1-3 already underway and being carried out by the Ministry of Education and NZQA in consultation with the education sector, particularly through the Secondary Principals' and Leaders' Forum. Mr Benson-Pope has requested that this be carried out with sharper focus and greater urgency. Improvements identified through this process will need to be made in a timely way so students benefit from them in this academic year.

ENDS

© Scoop Media