On a panel about trust, talent and technology in assessment, Sarah Hughes, director of assessment, research and validity at Pearson, said AI should only be used where it makes a test stronger.
“The biggest question we have when a test is using AI is to ask why: what are you trying to achieve, what do you think AI is enabling for you? Is it just cost savings, or do you think it’s adding value to your assessment?” she said.
“We don’t blanket use AI for everything in PTE. We use it where it makes sense, where it’s adding value, where it’s increasing the validity of the test.”
She gave fluency and pronunciation as examples. There, AI’s ability to “apply the same standard to everyone is a massive benefit to the fairness of the test”, she said.
“It’s important for testing organisations to demonstrate that value, that it’s not just a gimmick.”
Mary Richardson, professor of educational assessment at UCL, said universities don’t need to understand the technology itself. What they do need is to understand “where and when AI might be used” and whether a test “is actually testing what we need it to do”.
“And then will it get us the students we want for our modules, for our programs, whatever it is we’re recruiting for?” she said. “That’s absolutely key at the start – that you understand you’ve got the right tool for the job.”
Panellists agreed that people must stay at the centre of AI-enabled assessment.
“The people who design and use it, make decisions with it, are absolutely accountable. It’s just a tool at the end of the day,” said Hughes.
She said the field is now looking harder at where humans sit in the process. “How do we avoid things like cognitive surrender, where it’s a human just clicking approve over and over and over again?”
Pearson is researching human oversight in automated scoring and expects findings next year.
Sathena Chan, associate professor of language assessment at the University of Bedfordshire, said AI can make “very reliable” judgments of accuracy and fluency.
“But fluency plus accuracy doesn’t always equal communicative effectiveness. That is where the human can make the judgment.”
An audience member asked whether some groups are underrepresented in AI. Hughes said it helps to separate large generative models, trained on the whole internet, from scoring models built for a specific purpose.
PTE’s automated scoring models, she said, are trained only on PTE test-taker data – “a wide and diverse set of test takers – with the first training covered more than 120 first languages.
“Being upfront and transparent about when you’re using AI, what is it trained on, what does it reflect, and how does it relate to the population who are using it – the only answer I can come up with is being as transparent as possible.”
The genie’s out of the bottle, and we’re not stuffing it back in any time soon.
Mary Richardson, professor of educational assessment at UCL
Another audience member, who called themselves an AI sceptic, asked whether the sector should be expected to push back on AI use. Richardson said: “The genie’s out of the bottle, and we’re not stuffing it back in any time soon.”
She pointed out that AI is “not that new”, and that organisations including PTE have used parts of it “for a long time”.
“We’re the critical thinkers in this. The AI isn’t, and that’s part of the education.”
Beyond testing, the panel also considered how universities should respond to students’ everyday use of AI. Meredith Reeve, lead specialist in research strategy at Pearson, said institutions should “focus on learning integrity over academic integrity”.
“We know that bans for AI don’t really work. They just erode trust between educators and students,” she said.
Reeve cited Pearson research with 1,000 educators in the UK and US. Only half said their institution had an AI policy or strategy in place.
Hughes said that, even as AI spreads, there must be “space for messy human learning along the way”.
“The more that we can emphasise the human aspect of learning and make room for that, the better language proficiency will be.”
Alex Asher, CEO of LearnCube, said the human side of learning will only become more valuable as AI gets more capable.
“I really think that our social skills as humans are the one thing that we cannot be replaced on,” he said.
As an employer, he said, the first things he looks for are initiative, collaboration and communication. He urged universities to help international students build real connections with local students and communities.
“This is the thing humans are great at, and I don’t think we should be relying on AI to take over the friendship category.”
