REF in Europe

As we approach the REF2013 census date, it’s enlighteneing to look at how other european countries are approaching the problem of ‘measuring’ the excellence of their research base. The UK has for many years relied on the dual-funding system, where a proportion of the funds that we spend on research derive from the so-called ‘QR’ stream. In effect this is a block grant system, where HEFCE allocates public funds according to the number and quality of staff returned in the REF exercise. These funds are awarded to universities annually, typically using the same or a similar funding model each year, until the next research exercise. This is why universities are so concerned about the outcome of the REF – the block funding you receive will be apportioned according to your overall REF score for a number of years following the REF, which amounts to a considerable sum of money. The second public income stream is of course the money that is distributed to the research councils, which is awarded by funding panels (comprised of peer-group academics) in response to research grant applications. Alas, success rates are not very high – typically 20% or so.

I was interested to read how Germany proposes to assess research quality. As noted in the THE this week, the ‘Forschungsrating’ will not be tied to funding at all. Apparently the German Council of Science and Humanities did not believe it sensible to identify institutions that historically have been relatively strong and to put the lion’s share of money in those. On the face of it, this sounds like the ‘funding excellence wherever it is found’ mantra prior to RAE2008. The view of the German Council is that research is very much a question of luck and diversity, so funding on an historic basis does not make sense to them.

Instead, the expectation is that university leaders will use the ratings to better assess the standing of research in each area and to develop strategies accordingly, without the ‘pain’ or ‘gain’ of of REF. Interestingly, the parameters used to assess univesrities and institutes included research quality, promotion of young researchers and transfer of research into society. The latter is presumably akin to the ‘impact’ scorings that we will receive post REF2013, although these are seen by Germany as leading to uncertaintly and increased workloads. The German system will develop detailed criteria for the activities that are relevant in specific subject areas.

Would this system work better for the UK? – maybe. Interestingly, however, the League of European Research Universities apparently views the UK’s success in increasing research quality through RAE/REF with envious eyes. Many countries, it seems, would like to move to a system similar to ours, but they aren’t there yet. However, as one who bears the scars of all seven RAE/REF exercises (including many hundreds of hours of meetings), I can’t help thinking that the bureacratic load and thus cost of REF is becoming out of proportion to its original aims. What would be wrong, for example, with passing all funding to the Research Councils, and distributing it using the conventional peer review process. Sure, it would make planning more difficult because we would not be guaranteed a block grant every year, but the turmoil of the last couple of years has hardly been helpful in that regard. Nonetheless, we have managed to adapt and survive.


About stevehomans

Professor Steve Homans is a structural biologist with an international reputation in the study of biomolecular interactions. He obtained his first degree and DPhil in Biochemistry at Oxford University, and secured his first academic position as Lecturer at the University of Dundee. In 1998 he received the Zeneca award from the Biochemical Society and was elected Fellow of the Royal Society of Edinburgh. Prior to his current appointment he was Dean of the Faculty of Biological Sciences at the University of Leeds. Professor Homans brings extensive expertise of academic leadership and management, with a particular emphasis on organisational change.
This entry was posted in Uncategorized. Bookmark the permalink.

5 Responses to REF in Europe

  1. One comment: I don’t see that the REF places any significant burden on the vast majority of regular members of academic staff (like myself), research associates or PhD students — so the people who do the research. In practice staff just identify their best papers and write short descriptions of why they are good. Head of Schools and Directors of Research have more to do of course; but the REF submission is significantly smaller than your average EU proposal (and it’s for a lot more money). So in my view of things what you describe as the “bureaucratic load” probably an appropriate load and rightly placed on the academic management and research admin teams; I also can’t help thinking that this is at the very at least a useful driver to make them reflect on the state of research within the institution and hopefully think more strategically. So I’m all for the REF as before it existed strategic research leadership (or even really understanding what was going on and who was doing it) within most universities was sadly lacking.

    • steve homans says:

      If only it was an issue just for academic management and research administration teams Patrick. Let’s say we have in excess of 1000 REF eligible staff across the university. Each of the four submitted outputs requires ‘scoring’ by someone – that someone needs to have knowledge and to be research active in the field, ie one of the ‘people who do the research’. So, 4000 publications, at say half an hour each to review, giving 2000 person hours of time. Then there’s the many hours of meetings that the UoA leaders are required to attend – again, these people need to be research active members of the community, as do those that have already spent many hours assessing our impact case studies. Adding this up over UK institutions that will submit to REF, we are talking about a very substantial time commitment on some of our best researchers.

      I agree with you completely regarding the value of REF in developing strategic direction to research in universities. This is also acknowledged in the German system. However, you don’t need explicitly to link this to funding in order to drive strategy.

      • Yeah, I’m still not convinced that the extra effort isn;t something we shouldn’t be doing anyway. I read around 10 papers as part of this process (so far) and it left me with the feeling that this is probably something that I should be doing anyway (i.e. a day spent on something valuable but without the need to do the IQA I would not have done). Some of the comments and scores by my colleagues show that we could probably do with more of this to promote some mutual understanding (although is CS papers weren’t all reviewed by people with knowledge in an specialism; I read all sorts of things). But the fact that our internal review processes makes us actually read each others papers is a good thing and probably happened more in times gone by (when people had more flexibility in how they used their time).

  2. philliplord says:

    My concern with REF is that it pre-supposes silos right from the start. As an external review of one of my papers said: “even thought is interdisciplinary, it’s still relevant to the UoA”. Glad they liked it, but it’s a bit grudging. Combined with the reality that papers are judged by where they are published (ie we judge books by their covers), it’s a very unreactive system.

    Strategy is important but so is diversity. It is hard to be strategic about things you do not yet know, which should surely be what research is about. We can not be proactive about the future, when we assess based on the past.

    • steve homans says:

      I think this is what the Germans are driving at Phillip. However, I feel we need to try to be strategic about things we do not yet know, hard as that may be. After all, our Vision 2021 document is based on what we know now. We need to keep asking whether our strategy is current in light of new information, as our external environment changes, and evolve it accordingly.

      There’s not enough money in the system to invest significantly in everything we do. We must try to back the winners, such as emerging fields. These by the way might be too young to have gained significant impact and thus citations, which rather supports your argument that we need to be careful about assessment based on the past. It’s difficult to see how we can do it any other way though – I guess the argument is that by targetting funding to those who have a good record of research excellence and impact in the past, you have the best chance of innovation and success in the future. I certainly wouldn’t, as a general principle, wish to invest in those with a poor track-record!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s