Novel discrimination factors in artificial platforms have come to light, says law prof and privacy commissioner
Online proctoring biometrics for remote exams fails to meet Canada’s legal threshold of consent, privacy, and anti-discrimination, according to a new academic report from the University of Ottawa with the support of the Office of the Privacy Commissioner of Canada.
The report examines how artificial intelligence tools like Respondus, Proctorio, Monitor, ProctorU, and Examity have surged in use as online proctoring tools to compensate for the inability to conduct in-person exams during the pandemic.
However, project leader and law professor Céline Castets-Renard has pointed to several of its “legal issues of socioeconomic discrimination and privacy.”
For one, these software platforms use data mining and facial recognition to monitor for cheating. Yet the report found there is no guarantee that these software platforms ensure clear and individual consent from Canadian students whose biometric data is being collected.
The report also considers that data collection and retention control is more difficult because many of these technology companies are US-based, which could require them to transfer data to the US.
Regarding facial recognition, it no longer comes as a surprise that artificial intelligence tools contain a high potential for racial, gender, and age discrimination. However, socioeconomic factors also cause unwarranted software alerts.
For instance, the report found that the technology can flag a single bark or chirp from a noisy pet as a potential cheating incident during an online exam. Other audio-visual variables like living with children, residing in a multigenerational household, being in a rural location, and lack of access to proper equipment can also compound the potential for discrimination.
“Students may be placed in an inequitable position for socioeconomic reasons,” the report read. “Such novel discrimination factors are often completely absent from the AI discussion of discrimination, therefore the use of AI based proctoring software is a particularly important and distinctive scenario that regulations must consider.”
Given these various factors, the report concludes that the “potential for harm is not insignificant.” In particular, intersectional students could be subject to an “extreme risk of discrimination by online proctoring software.”
