The sector needs to address the issue of conflict of interest in university rankings, according to participants at a recent CGHE webinar.
Two studies published over the last few months have discovered “statistically significant” correlations between the use of services offered by rankings companies and an institutions’ subsequent performance in their rankings.
The first, from Igor Chirikov, senior researcher at Center for Studies in Higher Education at Berkeley, found that Russian universities who sought consultancy services from QS saw their QS rankings increase more than expected.
The second study by Julien Jacqmin, associate professor of economics at the Finance Department of NEOMA Business School, found a statistically significant increase in THE rankings positions from institutions who paid for advertising with the company, although it cannot be ruled out that, as Chirikov noted during the webinar, “a university just became better and advertised at the same time”.
While the studies can only prove a correlation rather than causation, a lack of transparency about the use of these services and how procedures work – as well as poor understanding among the public about metrics and methodology – the studies have highlighted long-standing concerns about the use and influence of such rankings.
“University rankers claim that conflict of interest does not affect ranking process and that there are internal procedures in place to prevent consulting and services from influencing the outcomes,” said Chirikov.
“They ask the whole sector to take their word for it and just trust them that everything is working well”
“But basically, they ask the whole sector to take their word for it and just trust them that everything is working well.”
“This is probably not corruption. And this is actually worse, because if it’s corruption, then it could be identified and fixed. This is integral to the business model.”
Jacqmin added that such companies usually emphasise that the consultancy and rankings are done separately of each other, but believes a better alternative would be even greater separation, or the rankings and services being conducted by different institutions entirely.
“The problem here is that there’s nobody governing the higher education world,” he said.
While regulating rankings companies would be difficult to the international aspect of their work, attendees at the webinar noted that rankings remain widely accepted by the public, particularly international students.
“I was on the Russian government’s 5-100 commission. And early on, we urged the commission to forbid the participating universities from paying the rankers for consultancy services,” remembered Philip Altbach, research professor and founding director of the Center for International Higher Education at Boston College.
“That was not listened to. I wish we could think of a way of doing these kinds of evaluations other than rankings. I can’t, nobody else has, the OECD tried a while ago and spent a whole bunch of money on it, but we don’t have anything else.
“Students and their families and governments, want these kind of rankings so that they can benchmark themselves.”
Others called for greater transparency, suggesting that institutions should be required to disclose if they have used consultancy services offered by rankings company.
“I would recommend basically to reconsider the use of global rankings with conflicts of interest,” said Chirikov.
“We as a sector [need] to discuss the impact of commercial rankings on higher education and make sure that rankings with a conflict of interest no longer exist.”
In response, Times Higher Education said its “actively works with the global higher education community to collect data and ensure rankings are fair and reflective”.
“To ensure best practice, for four years between 2016 and 2019 THE’s World University Rankings were independently audited by PwC to validate processes used and ensure that data collection and evaluation is fair, independent and robust,” said a spokesperson.
“While THE offers a number of commercial products, some based on datasets related to its suite of rankings, these are separate to the rankings and no commercial elements are considered when generating the rankings. In this regard, THE believes itself to be unique, clearly and transparently separating its commercial products and rankings to ensure that profit is not a consideration in the rankings process but that they are based solely on merit.
QS added that they “support transparency in procurement”.
“In many countries around the world, there are stringent protocols and official public procurement databases. QS views this as being best practice,” it added.
“QS is a mission driven organisation with integrity at the core of both our values and of our business model. This is underpinned by well-established governance and quality assurance processes, which ensure that our rankings operations and team are functionally independent, with a clear set of conflict-of-interest policies to which all our employees and stakeholders are subject.”
