Effective operational use of AI

ETS has been a leader in AI for more than 20 years and we’re deploying it in a variety of ways throughout our operational work. Consider the TOEFL iBT, for example, which is delivered to learners in test centres and from the comfort of home.

In both instances, robust AI-enabled scoring models support the measurement of the Speaking and Writing sections. For the at-home format, we have security technologies in place that actively monitor test takers’ behaviour, which directly support human proctors that monitor each test from start to finish. Both examples are made possible by AI.

More importantly, they underscore two of the benefits of AI we’re focused on at ETS – enabling a better, more convenient testing experience for learners; and providing support to humans that ensures the validity of assessment.

Eliminating and reducing bias + ethical AI use

There is no silver bullet to remove bias from products and assessments, however, there are ways to help significantly reduce its prevalence. One way is to ensure that the teams working on capabilities and products that leverage AI are diverse, especially those teams that are developing models and applying them for customer use.

In addition, not only does the data being used to train our models need to be representative of the learners that will later use them, but it needs to be continually evaluated to ensure those findings are consistent.

That evaluation comes from our own internal processes, but also from gauging feedback directly from the learners’ who use our offerings to validate that what we’re seeing holds true. All in all, the use of AI needs to be ethically applied. Our Research division has an entire team focused on ethical considerations of AI, which includes setting new standards for its use in measurement to ensure the validity and fairness of things like score automation and content generation.

Tangible, personalised insights for learners

One of the most exciting applications of AI – and one which holds much promise as we look to its future – is the ability to create a personalised experience for learners that generates insights they can leverage to grow and develop.

Take for example, the TOEFL Go! app, which launched with the enhancements we made to the TOEFL iBT test in July. The app provides targeted feedback and coaching, inclusive of AI-powered scoring for all test sections; tip to improve; and exemplar responses.

“The combination of automatic scores and instant feedback provides test takers with tangible insights into their performance”

The combination of automatic scores and instant feedback provides test takers with tangible insights into their performance so they can better prepare for test day, by validating their strengths and helping them navigate their areas of improvement.

Another example of this is a free app we developed called Converse: Workplace, which is designed to build one’s confidence in their English-language skills for workplace communications. The app provides tailored content and recommendations based on learners’ level of English and industry they’re interested in pursuing, as well as feedback in areas such as verbal and nonverbal communication.

For example, the app could be immensely useful to someone who wants to pursue healthcare and learns they could use improvement in maintaining eye contact; or someone who wants to be a teacher and learns they can improve their rate of speech and ability to stay on topic.

Looking ahead

As I look ahead to 2024 and beyond, I can confidently say that AI will impact all parts of our lives, and I’m genuinely excited by the opportunities it presents for education.

AI has the power to fuel the future of personalised and even socioculturally responsive assessment and measurement. The opportunities are endless to meet individuals where they are, based on what they know and can do, and get them to where they want to go.

AI can help humans do more of what they’re good at and provide an extra layer of support and monitoring that didn’t exist previously. It can make processes more efficient – the list goes on.

When it all boils down, it’s going to empower the learners of tomorrow, which is the mission that drives what we do at ETS every day.

About the author: This is a sponsored post from Kara McWilliams, Vice President of Product Innovation and Development at ETS, which includes the ETS AI Labs. Kara and her teams lead the ideation, co design, development and testing of innovative assessment and learning products. Kara is also responsible for the enterprise-wise AI strategy, including the development and deployment of novel AI/NLP capabilities to support both operational efficiencies and new products and services. Kara and her teams work across the K12, Higher Education, Workforce and Global Languages markets.

Previously, Kara was the Vice President of Learning Science and Insights at Macmillan Learning. There, she led the area that worked closely with Product to design, develop and bring to market its next generation learning platform, Achieve. Kara received a Ph.D. in educational research, measurement and evaluation and an M.A.T. in curriculum and instruction — both from Boston College. She received a B.A.in political science & history from the College of the Holy Cross.