Education researchers work to make online assessments smarter and safer

Among the challenges presented by remote learning is the need to accurately assess each

students’  progress and provide timely, constructive feedback that helps them identify their strengths and address their weaknesses—a challenge that’s compounded in higher education settings by large class sizes and complex concepts that call on higher order thinking skills.

Educational Psychology researchers Guher Gorgun and Genan Hamad are both working on ways to harness artificial intelligence to support instructors in providing timely feedback while making the most of classroom time and protecting the integrity of the assessment process. Their respective work has earned them each an Alberta Innovates Graduate Student Scholarship.

Gorgun says sophisticated automated scoring and feedback systems have emerged in recent years as viable options for assessing higher order functions like creativity, critical thinking and problem solving tested by open-ended questions. But, she adds, these systems are often developed without input from the instructors or students they’re meant to help.

“Students have many misconceptions concerning how automated scoring and feedback systems work. Not integrating their perspectives means an essential component is missing in transferring those automated systems into actionable real-life practices,” Gorgun said.

“In my study I’m trying to integrate instructor and student perspectives to develop optimal, actionable systems that can be implemented in large-scale classrooms.”

These systems aren’t just used to generate test scores, but help instructors and students assess progress and identify areas for improvement—feedback that can be provided instantly by an automated system. 

“By integrating these AI-based systems instructors can actually understand students’ performance in a more fine-grained manner and target those gaps to optimize learning in terms of helping everyone succeed,” Gorgun said.

The point of automated systems is not to replace educators, but to make them more effective in allocating their time and attention, she notes, which is why their perspectives are so important in developing automated assessment systems.

“As humans, we want human interaction,” Gorgun said. “We don’t want to replace humans but work in combination with instructors to provide better decision-support in the classroom.”

Guarding the integrity of online assessments

Hamad has been looking at harnessing the power of machine learning to detect cheating and aberrant behavior based on human-computer interactions in online assessments—an aspect she says is especially important when it comes to testing for professional licensing and certification, even if instances of cheating are very rare.

“Test administrators need to be confident their test results represent the true abilities of their examinees, and the stakeholders who are using these results to hire or license, say, a doctor or nurse, need to ensure that the people they hire or license, will be able to perform their jobs safely,” Hamad said.

Her research looks to improve on traditional statistical methods to detect cheating more efficiently with machine learning models, which can aggregate larger sets of data from more sources to detect patterns that indicate unusual user behaviour, such as consistently completing difficult questions faster than average over the course of an assessment.  

Hamad said she’s very grateful to have her research acknowledged and supported.

“I want to thank Alberta Innovates very much for this scholarship, which will have a big positive impact on my graduate studies, on my career, and on my ambition and desire to help the assessment community, especially in online environments,” she said.