The Impact of Impact

An interesting article appeared in the latest issue of the THE (19-25 February), which analysed the contribution of impact to REF 2014. From the REF outcome it is immediately possible to determine that the rank order of institutions has not been altered dramatically by the inclusion of impact – generally, institutions that had  a good score for outputs also had a good score for impact. Moreover, impact scored highly across the exercise, with an overall GPA of 3.24 compared with 2.9 for outputs. One interpretation of this latter outcome is that the concept of impact was scored more leniently, although a number of UoA panellists that contributed to the THE article argued otherwise.

What is clear however is that most institutions have recognised the huge significance of the quality of each impact case study submitted. There is indeed  a heavy weight assigned to impact, given that on the order of one case study was required for every ten academics submitted. This explains the significant number of REF submissions that contain staff numbers just below the threshold for submitting an extra case study. While this degree of ‘game-playing’ is perhaps inevitable, it is at the same time very unfortunate, since one must presume that in some instances  ‘REFable’ staff have been excluded on the basis of insufficient case studies. Unlike outputs which, again through a degree of game-playing, can be ‘bought-in’ by attracting research stars – reminiscent of the professional football transfer window – impact case studies need to be ‘home-grown’, and don’t transfer with the individual. Indeed, given the permitted time-window for recording impact, in a number of cases the driving mind behind the study may no longer be on the payroll of the institution.

It’s enlightening to assess how impact has influenced the outcome of the UoA submissions at NU. A key aspect of our mission is to be a ‘Civic University’. This expression is actually often misunderstood in UK circles to mean a University that is principally focussed on the city and region. However, the Vice-Chancellor is clear that the concept of a Civic University lies in the arena of responding to the challenges facing civil society. As such, our reach must be worldwide. With this notion, one would therefore expect that, all things being equal, that NU would score relatively highly in terms of impact. Is this true? Unfortunately, we can’t glean much from the ranking of NU in terms of overall GPA compared with impact – we are =26th and 26th respectively. However, if we drill down to individual SAgE UoAs, a different picture emerges. If we take our top unit (UoA11, Computing Science) for overall GPA ranking from REF 2014 (taking into account the number of submissions) this is ranked 9th by GPA and is in first position by impact. Our worst performer was UoA13 (Electrical and Electronic Engineering), achieving 16th position by GPA but 29th position by impact. This however, was very heavily influenced by the fact that one of the impact case studies submitted was not categorised due to the underlying research, although we understand that the case study itself was highly scored.

This is not a rigorous analysis of course – we’d really need to ask how our competitors  fared in a similar comparison (but by definition in UoA11 the comparison must have been less favourable!). Nonetheless, we would have been shaking our heads if impact was not a significant contributor to our overall score. The challenge will be to ensure that we have sufficient case-studies for the next REF, meaning they need to be in a good state of development now. The Dean for Research & Innovation is on the case (or should I say on the cases) in this regard. Without sufficient impact case studies, we face the prospect of being unable to include certain outputs at 3* and above, so in many respects impact becomes as important as maximising 4* outputs. Of course, it’s not unthinkable that we will be required to return all academic staff on research contracts in the next REF. Personally, I would welcome this development – it would largely (but not entirely) eliminate the ridiculous game-playing that is frankly farcical since everyone can see through it, and would reduce inevitable angst arising from those that are deemed REFable versus those that are not.


About stevehomans

Professor Steve Homans is a structural biologist with an international reputation in the study of biomolecular interactions. He obtained his first degree and DPhil in Biochemistry at Oxford University, and secured his first academic position as Lecturer at the University of Dundee. In 1998 he received the Zeneca award from the Biochemical Society and was elected Fellow of the Royal Society of Edinburgh. Prior to his current appointment he was Dean of the Faculty of Biological Sciences at the University of Leeds. Professor Homans brings extensive expertise of academic leadership and management, with a particular emphasis on organisational change.
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s