NCEA gets students' thumbs up, now they understand

Media release December 2005

NCEA gets thumbs up from students … now they understand the system

Students studying across all three levels of the National Certificate of Educational Achievement (NCEA) regard the assessment as a valuable qualification, and believe their teachers and parents do as well.

These findings are contained in research by the New Zealand Council for Educational Research entitled Shaping Our Futures: Meeting Secondary Students’ Learning Needs in a Time of Evolving Qualifications, the third and final report of the Learning Curves project.

The project explores students’ experiences of the NCEA, and describes the strategies they use to manage assessment workloads, their perceptions of NCEA’s value as a qualification, and the ways they produce individualised qualifications.

“Students are getting more and more savvy about managing their assessments, with their decision making being driven by their strategic understanding of the assessment system as well as the factors that motivate them as learners,” says NZCER Senior Researcher Rosemary Hipkins, who led the project.

“Their perceptions of the NCEA’s value are closely tied to their views of its relevance and to the different ways they engage with learning and assessment. Different students actively produce quite different types of NCEA qualifications, which allows them to keep future ‘learning pathways’ open.”

Ms Hipkins noted that over-assessment of students over the three years of NCEA could have a de-motivating impact, yet was something that could be addressed immediately by schools.

“Students saw skipping assessments as a legitimate strategy for managing over-assessment, or for avoiding the likelihood of failure or potentially embarrassing assessments. While skipping assessments is not new, what has changed is the extent to which students can now make strategic decisions about how they will accumulate the credits they need for their overall qualification, assuming they understand how the system works.”

One outcome of the study which is cause for concern is that while some students see themselves as “successful learners”, many are more likely to see themselves as “successful collectors of NCEA credits”. These students were using risk-management strategies to maximise credit gains with little critical regard to the value of learning gains.

“This is worrying, as such actions are no more conducive to lifelong learning than were previous methods of assessment for qualifications, and it seems unlikely that this situation can change while students are over-assessed,” says Ms Hipkins.

“The ‘positive’ is that the NCEA could offer students more scope to become life long learners, yet at this stage it is apparent that the hopes for the NCEA to become an assessment system that provides better support for lifelong learning are ‘not yet achieved’.”

ENDS

[More info and Executive Summary of Report Follow]

Scope and background – Learning Curves research

The Learning Curves project has documented changes in the subject and assessment choices offered to senior students in six medium-sized New Zealand secondary schools between 2002 and 2004 as the National Qualifications Framework and National Certificate of Educational Achievement reforms were progressively implemented. It has also investigated how students perceive and make their subject choices within the context of each school’s curriculum policies and practices.

This report documents findings from the third and final year of Learning Curves and is subtitled Shaping Our Futures: Meeting Secondary Students’ Learning Needs in a Time of Evolving Qualifications. It builds on findings from the first report, From Cabbages to Kings, which was released in mid-2002, and the second report, Shared Pathways and Multiple Tracks, which was released in April 2004.

The research used six case study schools (three in rural towns, three in cities) that were selected to represent a diversity of student groups and contextual settings. In 2004 the principal and five heads of department (of English, mathematics, science, technology, and the arts) were interviewed, as they had been in 2002 and 2003. Students at Years 11–13 were surveyed in mid-2004 with some also participating in focus groups.

Shaping Our Futures differs in approach from the first two reports as it captures differences between the schools, rather than generalising patterns across them, as was the case with the previous reports.

The research was funded through NZCER’s Purchase Agreement with the Ministry of Education.

About NZCER
The New Zealand Council for Educational Research is an independent, educational research organisation which provides educators, students, parents, policy makers, and the public with innovative and independent research, analysis, and advice. Established in 1934 through grants from the Carnegie Corporation, it became a statutory body in 1945 and now operates under the NZCER Act 1972 (and amendments). It is not formally attached to any government department, university, or other educational organisation.
For more information, please visit our website: www.nzcer.org.nz.

* * * * *

Shaping Our Futures: Meeting Secondary Students’ Learning Needs in a Time of Evolving Qualifications

The Executive Summary of the third and final report

of NZCER’s Learning Curves project

Rosemary Hipkins and Karen Vaughan, with Fiona Beals, Hilary Ferral, and Ben Gardiner

This is the third in a series of reports of findings from NZCER’s Learning Curves project. The project has documented changes in the subject and assessment choices offered to senior students in six medium-sized New Zealand secondary schools as the NQF/NCEA qualifications reforms were progressively implemented from 2002 to 2004. It has also investigated how students perceive and make their subject choices within the context of each school’s curriculum policies and practices.

In 2004 the principal and five heads of department (of English, mathematics, science, technology, and the arts) were interviewed, as they had been in 2002 and 2003. This report captures differences between the schools whereas the first two reports generalised patterns across them.

Students at Years 11–13 were surveyed in 2004 and a number of focus group conversations were also carried out. Thus work with the students constituted a greater proportion of the (expanded) fieldwork time than had been the case in the first two years. Accordingly, this report has a greater focus on the students’ experiences of the NCEA than either of the first two reports. It describes strategies students use to manage their assessment workloads for the NCEA, their perceptions of its value as a qualification, and the ways they are positioned as active producers of an individualised qualification, within an overarching pathways metaphor of schooling.

The report concludes with a brief “stocktake” of the NCEA, and the changes that have accompanied its implementation, against a theoretical listing of the many deeply held ideas and assumptions that are implicated in assessment change. It concludes that many such assumptions have yet to be addressed, and that ongoing change, on a somewhat broader front, will be necessary if the qualifications reforms are to deliver more effective ways of credentialling learners for their adult lives in the 21st century.

School-specific differences

The overall philosophy of the staff, the beliefs and actions of the students, the organisational aspects of each school, and some demographic characteristics such as location and decile, come together to create an unique institution. For example, City Schools A and B shared some differences that set them apart from the other schools in the sample. Both are in urban locations, close to universities and all the cultural amenities of a city. Most of the students who identified as Asian and many of the refugee students, or students of “other” ethnicity, attended one or other of these schools. Many students who said they skipped internal or external assessments in Year 12 or 13 English came from these schools. In both schools students were more likely (compared to the overall sample) to choose combinations of more traditional curriculum subjects. The similarity is interesting because the ethos of each school seemed somewhat different. City School A—the girls’ school—had a focus on helping all students learn via broad and balanced curriculum. It had more compulsory subjects than the other five schools, and was the only one to make English compulsory at Year 13. By contrast City School B—a liberal, non-uniform co-educational school—had a focus on supporting greater student autonomy and freedom in choosing appropriate subjects and pathways. Accordingly, it offered more optional choices at Years 11 and 12 than the other five schools, and almost as many choices at Year 13.

The principal of Town School E had a singular vision of seamless learning pathways through school and beyond that was shared by key members of the senior management team. In this school, some subjects were beginning to diversify across traditional curriculum boundaries and there was a focus on the many national certificates that could be gained within the National Qualifications Framework, not just on the NCEA. The school’s emphasis on unit standards provided a means of assessing course components linked to a much wider range of learning opportunities. In keeping with the gradual broadening of learning pathways, this was the only school of the six to offer a greater number of choices at Year 13 than at Year 12.

Town Schools D and F both worked hard to accommodate the learning needs of a diverse range of students. Both were somewhat vulnerable to the loss of more “able” students because they were in towns within driving distance of regional cities that had a number of secondary schools. In both locations this dilemma created tensions between meeting the needs of some students for alternative pathways and keeping other students engaged and challenged in their home-town school. Town School F had a very active curriculum committee. There was a sense that the allocation of resources to various combinations of courses was constantly under view. Two HODs mentioned the intention to offer half-year courses as one way of maximising the potential for choice. HODs at Town School D similarly described a yearly process of juggling and revising components within courses, and in the overall mix of courses offered. Nothing can be taken for granted in these environments where there can never be an ideal solution to curriculum and timetabling challenges. Students in these two schools also shared some patterns of responses to the NCEA. Perhaps reflecting the yearly debate and juggling of subjects and timetables, the Year 13 students in both schools were more likely than those in the other schools to say that NCEA results in earlier years had influenced their subject choice. The Years 11 and 12 students from all the town schools were more likely to say they had skipped an internal assessment, and many of the Year 11 students who said they had skipped an internal English assessment came from either Town School D or F.

City School C has the lowest decile rating of the six Learning Curves schools and has a greater proportion of students from Mäori and/or Pacific backgrounds on the roll than any of the other five schools. In this school, the need for more time and skills practice for students who lacked the necessary background or personal confidence to make speedy learning gains was a common theme. Restructuring to create more learning time was being tried out at various levels of school organisation, from the timetable structure to the content and organisation of individual courses. HODs in this school were more likely to discuss the importance of helping some students to gain the literacy and numeracy credits that would keep their learning pathways open beyond school. Perhaps reflecting their learning challenges and awareness of the importance of gaining credits where they could, students in City School C were less likely than most other students to say they had skipped an external assessment, especially the Year 12 students. The Pacific students here were more likely than all other students to agree that the NCEA is a valuable qualification and that unit standards are easy to get.

Taking different versions of subjects

Analysis of 2003 course information led us to describe three different types of courses that we called traditional-discipline, locally-redesigned, and contextually-focused. The 2004 fieldwork revealed patterns in the ways students combined these types of courses, with each other and with their optional subjects.

Traditional-discipline courses look similar to the sorts of courses most students would have taken pre-NCEA and are typically mainly assessed by full suites of achievement standards. In 2004, a majority of Year 11 students were taking English courses of this type (77 percent). Slightly fewer took equivalent versions of science (70 percent) and fewer still traditional-discipline mathematics (58 percent). A similar pattern was found at Year 12 where 72 percent of students chose traditional-discipline English, 56 percent took at least one of biology, chemistry, or physics, and 46 percent chose an equivalent version of mathematics. (Note that both the science and mathematics choices are optional at Year 12.) Compared to the overall cohort, Päkehä students were more likely to be taking traditional-discipline versions of these “core” curriculum courses.

There are two types of alternatives to traditional-discipline courses. Locally-redesigned courses take advantage of the NQF/NCEA flexibility to mix and match achievement and unit standards, sometimes at one or more NQF levels, sometimes from more than one discipline area, to provide courses to meet identified learning needs of specific groups of students. Contextually-focused courses locate learning in contexts of relevance to learners and are typically totally internally assessed, mainly with unit standards. Fewer students took these alternative courses. A small number of Year 11 students (10 percent) were taking alternative versions of English, and 32 percent, mainly Pacific or Mäori students, were taking alternative versions of mathematics. At Year 12, 15 percent of students took an alternative version of English, while 23 percent took an alternative version of mathematics. Again Pacific or Mäori students tended to take this type of mathematics. Eleven percent of students took an alternative Year 11 science course and 5 percent took an extension course or science at another year level. In Year 12 students wanting an alternative to the traditional sciences could often access employment-focused options such as electronics, forestry, or aqua-culture.

The cluster analysis revealed a strong tendency for students to combine both core and optional subjects of the same “type”. For example 49 percent of all Year 11 students combined traditional discipline versions of English, mathematics, and science. They were also likely to have chosen other traditional-discipline subjects such as history, geography, languages, economics, and graphics. Nearly a quarter (22 percent) of the Year 11 students took combinations that included alternative versions of the three core subjects (or in some cases that did not identify a version of English despite its compulsory status). These students were more likely to choose combinations of optional subjects with a strongly practical feel, such as practically-oriented versions of technology, computer studies, transition courses, or agriculture/horticulture. The remaining 29 percent of Year 11 students either combined traditional-discipline English with an alternative version of mathematics, or vice versa, with a corresponding “mix” of different types of optional subjects.

Similar patterns, albeit in fewer clusters, were found at Years 12 and 13. Notwithstanding the wide range of subject choices schools offer across the senior years, there is something of a conservative feel to the combinations of subjects students actually take in Year 13. A quarter of the responding Year 13 students had continued with traditional-discipline versions of subjects in all three core curriculum areas (i.e. English combined with statistics and/or calculus, and one of the three sciences). However, nearly a third of the Year 13 students were taking a subject combination with a strongly “alternative” feel, where an alternative version of English was likely to be the only remaining core subject. Mäori and Pacific students were over-represented in this cluster.

The relative popularity of optional subjects

The overall pattern of top-rating Year 11 options remained quite stable across the three years of the study. History (21 percent in 2004) remained the top optional choice, as it was in both 2003 (20 percent) and 2002 (21 percent), albeit with some school-specific differences in uptake. When collated as a group, the various technology options (for example hard materials or food technology) were also very popular. Three of the four arts subjects (visual arts, music, and drama) were taken by 10 percent or more of students across the schools. There were some gender and ethnicity differences in subject uptake, although these were likely to be related to school effects in some cases. For example history was very popular at City School A and was more popular with girls overall, as well as with Päkehä students.

At Year 12, both types of mathematics and the three traditional sciences were all popular optional choices. History was overtaken in the popularity stakes by “vocational pathways” subjects (for example chef training and journalism) and by PE and sports studies. All three of these subjects (or subject types in the case of vocational pathways) were relatively more popular with boys, as was physics. Other popular subjects were mostly the same as those chosen at Year 11, although practical technology slipped below 10 percent. Again, history was more popular with girls.

Versions of English, mathematics, and sciences—all subjects that are compulsory at lower year levels—occupied the six top-rating slots at Year 13, when almost all choices were optional. Visual arts continued to be popular, along with PE and vocational pathways subjects. Although history continued to be taken by more than 10 percent of the cohort, its relative popularity had slipped still further. Physics, calculus, and vocational and sports subjects were more popular with boys, while traditional-discipline versions of English, history, geography, and the visual arts were more popular with girls.

A comparison of the Years 12 and 13 subject choice data with national trends showed that these patterns of relative overall popularity were remarkably close to national patterns of participation in English, mathematics, and science-related subjects. There were minor variations in uptake where other subjects could be directly compared, which is scarcely surprising in light of the school-related variations we found in our small sample.

Students’ and teachers’ perceptions of the NCEA

In 2004 most responding students were happy with their subject choices. Most students across all three year levels saw the NCEA as a valuable qualification and thought that their teachers and parents did too.

In 2003 some teachers reported a perception that credits gained from achievement standards were of more value than those gained from unit standards. Views amongst the teachers on this point were mixed in 2004, with some holding strong perceptions of difference and others saying students did not mind how their credits were gained. The students’ survey responses showed widespread agreement that credits gained from both sources were valuable, with a slightly greater level of support for the statement that “credits gained from achievement standards are valuable”. Where students did perceive a difference, they were likely to feel that achievement standards had more currency in competitive situations, such as gaining entry to university courses or seeking employment. By contrast, unit standards were valued by both teachers and students for the opportunity they provided to assess more practical aspects of learning. Focus group comments suggested that students might be more familiar with one type of standard than the other, depending on the type of subjects they were taking in combination.

Many teachers were reluctant to compare the relative worth of credits gained from internal or external assessment events, saying that they measured different things. Where differences were perceived, teachers generally saw externally gained credits as more valuable, mainly because of concerns about moderation issues and consistency between schools. While opinions can obviously vary between the teachers in any one school, there were some school-related differences in these perceptions overall. Students differed from their teachers in that they placed slightly more value on credits gained from internal assessments. They seemed to see these as the main source of credits from which to strategically build a qualification, with externally assessed credits forming an end-of-year insurance backstop. Again, we found some patterns of differences in responses from students in different schools. For example Town School E’s Year 13 students tended to prefer credits for internally assessed standards, which is in line with the school’s policy of using unit standards (always internally assessed) whenever possible.

As might be anticipated, students taking subject combinations weighted towards traditional-discipline subjects were more likely to agree that they did well in external assessments. These students were also more likely to discuss the motivation to strive for better learning provided by the three-level structure of the achievement standards (where it is possible to achieve, achieve with merit, or achieve with excellence). However the same students were also more likely to perceive they did well in internal assessments. Students taking course combinations with a more “alternative” subject orientation were less likely to agree they did well in either external or internal assessments. For such students, assessment may remain an anxious hurdle, no matter how it is carried out.

Students’ strategies for coping with the NCEA

The second Learning Curves report found that some teachers were becoming concerned about students who chose to skip assessments. All the HODs were asked about this in 2004, with rather mixed results. Some were indeed very concerned and described measures such as sending a letter home to parents. Other teachers actively helped students to decide which assessments to skip as a means of managing their workloads.

About a quarter of Year 11 students, compared with 40 percent of Year 13 students, said they had skipped an assessment at some time (which, for Year 13 students, could have been at any time across two-and-a-half years). Students were more likely to have skipped an internal assessment than an external assessment, and English was the subject where this was most likely to occur.

Many students saw choosing to skip as a legitimate strategy for managing over-assessment, or for avoiding the likelihood of failure or potentially embarrassing assessments such as speeches in English. However the extent to which students said they did these things themselves was somewhat less than teachers’ concerns in 2003 had led us to anticipate. Often they simply reported that they knew others who did this, while they personally felt they could not afford to squander any chance to gain credits. This feeling probably abated as students’ confidence and experience with the NCEA grew.

One of the HODs noted that students have always skipped assessments so this practice is not new to the NCEA. What is new however is the extent to which students can now make strategic decisions about how they will accumulate the credits they need for their overall qualification—provided they understand how the system works. Again, we found that the students taking courses weighted towards traditional-discipline subjects were more likely to know how to make good strategic decisions of this type, and accordingly to only choose to skip when they needed to maximise learning success in those assessments they retained. By contrast, students who took more “alternative” combinations of subjects were more likely to make ad hoc decisions, or to not be in command of their credit totals at all.

Student motivation and the “production” of a qualification

Students’ decision making for managing their assessments was influenced by their strategic understanding of the assessment system and by the factors that motivated them as learners. Their perceptions of the NCEA’s value as a qualification were closely tied to their views of relevance and to the different ways they engaged with learning and assessment. Different groups of students actively participated in the production of quite different types of NCEA qualifications, with associated differences in the “learning pathways” they kept open.

For students taking contextually-focused courses, notions of relevance tended to be tied to usefulness and applicability of learning to everyday life, and to prospects for employment beyond school. By contrast, students taking mainly traditional-discipline courses related relevance to future study plans or to ideas of “learning to learn”. As might be expected, responses of students taking locally-redesigned courses tended to fall between these two poles.

Some students seemed behaviourally motivated by the NCEA. They were interested in the number of credits on offer, and whether these were a “fair trade” for the work involved in passing. Of all the strategies for managing the NCEA available to students, they were most likely to avoid failure (as they saw it) by skipping assessments they did not feel confident of passing. Students taking locally-redesigned and contextually-focused courses were more likely to take such approaches than students taking traditional-discipline courses.

Other students were more emotionally engaged with NCEA. They were aware of the potential importance of their Record of Learning, and actively kept track of their credits. These students were more concerned to produce a point of difference in the qualification they achieved, with an associated likelihood of increased motivation to gain merit or excellence passes, or in some cases, simply to accumulate higher credit totals. For these students, perceived inconsistencies in different subject teachers’ practices when making “holistic” judgements of the level achieved could be a source of bewilderment and indignation.

A small number of students were cognitively engaged. They actively pursued challenging learning experiences, although the satisfaction they gained from these often had to be weighed against strategic management of workload pressures from too many assessments at the same time. These students were also motivated by the opportunity to gain merit and excellence, although they were the students most likely to want to see such achievements acknowledged with the award of additional credits or an acknowledgement of credit type (from unit or achievement standard) on their Record of Learning.

Learner identities and assessment careers

While some students do see themselves as successful learners, it seems that many are more likely to see themselves as successful collectors of credits. Accordingly, they are developing assessment careers that use compliance and risk-management strategies to maximise credit gains with little critical regard to the value of actual learning gains. This is of concern because such learner identities and assessment careers are no more conducive to lifelong learning than were previous methods of assessment for qualifications. It seems unlikely that this situation can change while students are over assessed.

The NCEA has opened up so-called learning pathways. However the intended parity of esteem for qualifications has not been matched by the reality of perceptions and practice. If anything, the widespread acceptance of the pathways metaphor has contributed to a hardening of the academic/vocational divide by producing active compliance with the sorting of students into strongly differentiated courses that provide quite different types of learning experiences. While some students do experience strongly practical learning, the “intellectualisation” of their subjects has been strongly resisted. And “academic” students who want to study subjects with a strong practical element are still likely to have the suitability of their choice questioned.

Notwithstanding its early promise, we have to conclude that the hopes for the NCEA as an assessment system that provides better support for lifelong learning are “not yet achieved”.

Rethinking assessment change

Reforming an assessment system is not just a matter of changing the structure of the assessment instruments and procedures used, although that has understandably been the focus of the NCEA reforms to date. Ideas and assumptions about the nature of knowledge and learning, the relative emphasis that should be given to different purposes for assessment, and issues of power and control of assessment and learning processes, are all implicated. At present the NCEA seems more akin to a complicated system, but it could be rethought as a complex system, where changes to all the interconnected parts become a focus for attention. Tentative ideas for how this ongoing change process might proceed are explored in the final section of the report.

© Scoop Media