PBRF: Everyone's a winner - except students

Whatever the name for the beltway that exists around the tertiary education industry, it was all abuzz on Thursday for the release of the latest Performance-Based Research Fund (PBRF) report by the Tertiary Education Commission (TEC).

This was the third report following an interim one in 2003, and the first full report in 2007.

The PBRF dates from 2002, when the then Labour Government decided that there needed to be greater accountability for research that takes place in tertiary institutions. This was done by gathering the so-called "research top-ups" that had been added to student funding rates to reflect the amount of research that the teachers of different courses were supposed to have done and putting it all into a fund (the PBRF).

This fund (currently $262 million) is allocated to institutions not on the former basis of student enrolments but on "research performance". Sixty percent is allocated on the basis of a quality measure of research outputs by academics, 15% on the basis of the portion of the entire sector's External Research Income, and 25% on the basis of how many research degrees (involving a thesis) that your students complete (Research Degree Completions - or RDCs).

The media focus was almost entirely on the quality measures. Fair enough, it produces a table of rankings, and the media love that sort of thing. There was quite a bit of chatter around VUW's launch up the rankings, from fourth to top, and given that this followed exposure last year of VUW's perceived gaming of the system (by large-scale exclusion of academics whose research performance would drag down the average) that seemed reasonable too. Interestingly, at least four institutions claimed that they "won" the PBRF. SciBlogs captured the trumpeting.

There are actually four quality measures: per academic who submitted a portfolio, per student, per academic, and per post-graduate student. VUW topped the first, but none of the others. It's widely understood that all the institutions game the PBRF, and generally accepted that VUW did this less than some others in the previous two rounds. They've certainly made up for that indiscretion. There is some useful analysis of the numbers on Curia Research’s kiwiblog.

All of this misses some pretty fundamental issues though.

The first is that the PBRF represented a massive transfer of funds from polytechnics to universities. The attraction of the availability of research top-ups on the basis of enrolments had led to many polytechnics expanding rapidly into degree provision, but after its now fully-phased implementation, 97% of the PBRF goes to universities. This may be justified in terms of that's where the research is, and exposes that whilst degrees are meant to be taught by people "actively engaged in research", some clearly aren't, yet it has certainly contributed to the financial troubles of most of the country's polytechnics.

The second is the coupling with how institutions manage themselves, by "allocating" the funding that comes in a particular way (eg with a particular student, or for a particular academic's research) to the school or department that student or academic is "in". There is essentially less money per student, so that has put pressure on institutions with particular mixes of students/staff/research output/etc.

But overwhelmingly the greatest impact on students is the research degree component of the PBRF itself, and the imperilling of support for research degree students that it has led to. What makes it worse is that this is absolutely a predictable but entirely unintended consequence.

The allocation for RDCs is fixed at 25% of the total PBRF pool, but the number of degree completions has grown far more rapidly than the increases in the pool. If we take inflation into account (using the RBNZ inflation adjustment tool), as the PBRF itself does (growing annually by CPI adjustments), then the amount per average RDC has fallen from $20,061 per RDC/year in 2003 to just $11,368 in 2013.

A nascent national post-grad ginger group I was involved in raised this issue in the early 2000s, suggesting that the per-RDC rate be fixed and inflation adjusted. But since this would have solved the problem, naturally it was ignored by the powers-that-be.

Different RDCs get different funding on the basis of the cost category of the programme they are in (to reflect the different cost of research in education vs engineering, for example), ethnicity of the student (to promote Maori and Pacific students) and a weighting exists whereby a PhD = 3 times a Masters (since they take longer). The elements of the calculations aren't published but if we assume that RDCs reflect the categories of academic research, and factor in the ethnicity reporting, it looks like a humanities PhD by a Pakeha student (the base amount) gets an institution a little over $15,500; science double that; with engineering/architecture/medicine 2.5 times the base.

These figures matter because the response to the focus on completions rather than enrolments led to the introduction of more scholarships, as just one example, since a student who can focus on their research completes and completes faster. Such scholarships are generally around $25,000 per year. Forking out $75,000 for a RDC that gets an institution $15,000 at the end makes no sense at all in terms of cost/benefit analysis.

In part this is because the institutions haven't actually done the simple calculations that need to be done in the area. Perhaps they're avoiding that which is unpalatable. When I was on the VUW Research Degree Committee in 2005, the responsible team was working on an estimated $50,000 per base-level PhD completion. My back-of-the-envelope calculations suggested half that, at best. The calculations above confirm (1) my estimate was right, and (2) it's even less now.

The institutions are racing to have more and more completions, since they get more money by having more completions, but in doing so, since everyone is doing it, each one earns them less and less. It is essentially a Prisoner's Dilemma, where the policy settings inspire a competitive, instead of collaborative, environment and that it necessarily leads to a sub-optimal outcome.

This has broader significance. In the first place any academic who deserves the title will acknowledge that for a school to have a thriving research culture relies on post-graduate students. Departments with lots will always do more research, of higher quality, than those without. The international evidence supports this. Further, research students are the future of the academy, their research is often at the cutting-edge of the disciplines and professions. Their enthusiasm engenders further enthusiasm.

It's important contextually that this rapid fall in per student funding comes at the same time as massive cuts in government support directly to post-graduate students, most obviously through the removal of access to student allowances. Institutions might have been expected, or have wished, to step into the breach, but as they face their own cuts that seems unlikely. Indeed surveys of post-graduate students reveal widespread despondency and large numbers wanting to quit either New Zealand or their study altogether.

New Zealand has adopted a strategy of encouraging post-graduate researchers from around the world to study at our institutions. Wherever they are from they pay domestic fees, their kids go to New Zealand schools for free, they get scholarships, they are on an immigration track if they want. Why? Because student research is a public good, it benefits our society and our economy, we get the results of that research, the institution's research culture is enhanced, and New Zealand might get to keep some of these representatives of the world's best and brightest here when they've finished. Yet, since most countries follow this approach, New Zealand's research students are better supported in Australia, and further afield, than they are here.

Minister of Tertiary Education Stephen Joyce and the newly-employed head of the TEC Tim Fowler each wrote a foreword to the PBRF results crowing about the success it reveals in New Zealand's research performance. They, like the institutions, need to stop crowing about vacuous and questionable results and start doing something to address the crisis for post-graduate research. Else, they will oversee a damaging destruction of New Zealand's research base ever witnessed. How's New Zealand's future going to be brighter then?

References:
Sciblogs: http://sciblogs.co.nz/kidney-punch/2013/04/12/spin-doctors-go-to-work-on-pbrf/
Kiwiblog: http://www.kiwiblog.co.nz/tag/pbrf
NZ Inflation Calculator: http://www.rbnz.govt.nz/statistics/0135595.html
Prisoner’s dilemma: http://en.wikipedia.org/wiki/Prisoner's_dilemma
Surveys: http://students.org.nz/2012/11/postgrad-students/

Dr Alistair Shaw is Executive Director of the New Zealand Union of Students' Associations, which has been supporting students since 1929. He has a PhD from VUW, awarded in 2011, during the course of which he served as President of the VUW PGSA and represented postgraduate students on the university's Research Committee and Research Degrees Sub-Committee.

© Scoop Media