Industry experts gathered at the House of Commons in London to discuss ‘Measuring the wrong things? How should we judge the success of universities in creating value?’ in what HEPI Director Nick Hillman referred to as a ‘morning of self reflection’.

Alison Johns, CEO of Advance Higher Education, introduced the ‘fresh debate’, for which the need has grown – amid demand across the sector – on how it captures and measures success beyond traditional measures at the breakfast seminar, which was the first hosted by Advance HE and HEPI since February 2020.

Christopher Hales, interim chief executive and director of policy, UUK, spoke about how the organisation undertook extensive work on answering the debate, publishing a framework of programme reviews to support universities in assessing the value of their own courses.

This sets out principles on the role of using metrics in decision making and recognises “whilst metrics are useful, decisions should be informed by metrics rather than led by them”.

It also recommends that “metrics be used to flag anomalies or concerns which should then be investigated further by considering contextual information reflecting the wider sense of factors,” according to Hales.

Key themes of the framework include student graduate views, student outcomes, graduate prospects, supporting economic growth and social responsibility – a popular phrase throughout the seminar.

It’s standing room only at our first in-person event with @AdvanceHE since February 2020. pic.twitter.com/9PYpGOvKCC

— Higher Education Policy Institute (@HEPI_news) May 24, 2022

Issues were raised on how value has been measured historically, focusing on traditional career paths and somewhat ignoring emerging labour trends; Hales highlighted the importance of “value to the individual being” as something to consider.

“We must also focus on how students decide where and what to study and evaluate them as an individual.

“A good outcome for one student might not be the same for another – students will also gain other benefits from studying university that need to be considered but are sometimes harder to quantify, including their knowledge and appreciation of the subject’s competence and friends and connections,” Hales said.

“We must also focus on how students decide where and what to study and evaluate them as an individual”

Susan Lapworth, incoming interim chief executive, Office for Students, said that “continuation of completion are not the only measures of quality“.

“There’s a real risk in basing regulation and funding decisions only on narrow measures,” she added, noting that doing so can fail to recognise the many benefits of education outside of graduate employment.

“We think that students shouldn’t be able to choose courses and draw down student loan funding where outcomes are weak,” said Lapworth.

“These are complex policy questions that we’re going to have to work through collectively to make sure that as the context changes for higher education, the indicators and the measuring that the bridges keeps up to date,” she concluded.