It’s league table season once again, and the annual THE rankings arrived on my desk last week. Once again it’s not the best news – we have fallen from 180th position in 2012 to 198th in 2013 to below 200 in 2014. As I mentioned last year, such a fall is unlikely to be statistically significant (especially this time being only four places), but nonetheless our fall to outside the top 200, in common with a couple of other UK institutions, has certainly attracted media attention.
Unlike last year, inspection of our scores does not immediately show where we have lost ground. Our total score is determined by performance in five areas. These are as follows for 2014, with the 2012 and 2013 scores in brackets (in that order): teaching 30.3 (37.9,29.7); international outlook 73.8 (74.4,76.3); industrial income 35.0 (37.4,36.9); research 29.6 (30.5,28.3); citations 70.3 (72.1,68.1). Overall it looks as though we have in essence ‘held our ground’, and this is the key point – others appear to have overtaken us.
I use the word ‘appear’ advisedly. The THE article, that runs to 66 pages, undertakes an in-depth analysis of the ‘ups and downs’ compared to previous years. My gripe is that this analysis is predicated on the fact that these data are an accurate reflection of reality – whereas most likely they are not! Many might argue that I would say that, wouldn’t I (sour grapes and all that), but I suggest you read an interesting article entitled ‘Debating the role of metrics in research assessment’ by Professor Stephen Curry at Imperial College – in a 9th ranked institution one can hardly claim that he has an axe to grind. Moreover, looking at the raw data exposes some dramatic inconsistencies – Tokyo Metropolitan University, in the 226-250 ranked group, apparently scores 100.0 for citations, higher than the first-placed university (Caltech), yet only manages 9.8 for research.
To illustrate the idiosyncrasies of the rankings process, let’s take the ranking of Civil and Structural Engineering in the QS league tables (in which NU fares considerably better than in the THE league table by the way). Imperial college is ranked 9th in the world for this discipline in the QS league table for this year, and NU is ranked in the 151-200 range. Yet, in the last research assessment exercise, NU was placed in the top 3 institutions based on research power, just behind Imperial. I don’t think anyone would argue with the rigour of the research assessment exercise, and given that league tables typically have a strong research bias, on the face of it these data are difficult to reconcile.
So on one level I’m not losing any sleep over our fall in the latest THE league table. But on another it is very important – like it or not many influential people and organisations take these data at face value. To paraphrase Stephen Curry, the apparent objectivity of numbers is very seductive. Nonetheless, few would argue with the notion that the top 20 (or so) institutions in the most prominent league tables really are the best in the world, but to argue that there is any significant difference between say, the 100-150th ranked institutions is rather missing the point. This doesn’t mean of course that we should abandon our vision to improve our international reputation, but our latest fall in the league tables is not an indication that we are focussing on the wrong issues.