TL;DR
We can estimate that 3,588 students of color have experienced racial discrimination by Proctorio 31,397 times in the last year at CU Boulder and CU Denver. If you read on, I’ll tell you how I got those numbers.
---
Over the last few months, I’ve been leading an initiative at the University of Colorado System to ban facial recognition and detection technologies in educational settings. I wrote a letter explaining what they are, why they’re harmful, and how they run counter to our institutional values. Fortunately, most of the system doesn’t use these technologies, with the notable exception of algorithmic test proctoring software. The committees I’ve brought this letter to are mostly in support of banning facial recognition but have reservations when it comes to banning Proctorio or Respondus, the two products our system licenses.
In these meetings, I’ve been asked more times than I can remember to provide direct evidence of harm to the students. Show them the numbers, they say, show them data. This this seems like a reasonable request on its surface. However, the systems that could produce the most relevant data which would demonstrate the harm proctoring creates are designed by the companies that own them. I’m talking about the administrative back end of proctoring tools that collect massive amounts of user and product data but synthesize only company-approved reports about its performance. I’ve tried to explain that Proctorio and Respondus are not going to provide reporting functionalities that give any evidence of their harm. It’s against their financial and existential interests.
The data that I do have is qualitative and mostly from students who have expressed eloquently and viscerally how test proctoring is an invasion of their privacy, how it can’t identify them because they’re Black, how it outed them as trans to their professor, how it flagged them as cheating even though they weren’t. Unfortunately, experiential evidence hasn’t been very compelling for some administrative groups. While this represents a much larger problem about incomplete epistemological justifications in higher ed, it motivated me to come up with a way to find quantitative evidence of harm. I’ve come up with a deductive strategy of taking known data sets and rates of certain phenomenon and applying them to make approximations. While I think I’ve come up with a reasonable method for producing quantitative estimates of harm, there’s obviously room for improvement. If you have suggestions for how to do this, please contact me.
Here goes!
1. Find the total student population. Let’s start with CU Boulder. According to their website, their fall 2020 student enrollment is 34,975. This data is usually public for most institutions and can be more or less easy to find.
2. Find the number of students proctored. This can be tricky to get. Any institution that has a test proctoring software can generate these reports, but not everyone has access to them. I’m referencing reports from a Colorado Open Records Act (CORA) request filed by a journalist. Anyone can file a CORA, but sometimes they cost money. From this report, we know that there are 22,238 active users of Proctorio in 2020.
3. Find the percentage of students proctored. We’re trying to find out what percentage of the total student population has or is using Proctorio. Divide the number of active users (22,238) by the total student population (34,975), and you’ll get the percentage of students proctored (63.5%). We’ll need this percentage for later on.
4. Find the number or percentage of students of color. By student of color, I mean any student that doesn’t identify as white. While the report I found doesn’t give numbers, it does list the percentage of students who don’t identify as ‘white’, ‘international’, or ‘unknown’ as 26.2%. To convert that to a number, multiply the total student population (34,975) by the percentage of Students of Color (26.2%) to find the number of students of color (9,163, which we round up because people aren’t divisible units). That leaves us with 9,164 students of color.
5. Find the number of students of color who are proctored. To do this, we have to assume that proctoring is evenly distributed across the student population. While it’s likely that both proctoring use and student demographics are unevenly distributed across disciplines, without further data we can’t know if or how that distribution would impact what comes next. To find this number, take the percentage of students proctored (63.5%) and multiply that by the number of students of color (9,164), which gives us (5,819.14 which, again, we round up because people aren’t divisible) 5,820 students of color who are proctored. This number could be higher or lower based on which courses CU Boulder proctors and how many students of color are in those courses, both of which we don’t know.
6. Estimate the number of students who experience discrimination. We do this by applying the facial recognition error rate for people of color. According to multiple studies and leading experts in the field, facial recognition can be inaccurate at a rate of 35% for people of color. To estimate this, take the number of students of color who are proctored (5,820) and multiply that by the facial recognition error rate for people of color (35%), which gives us 2,037.
7. Apply the number of tests per student. The Proctorio report shows us that each active user at CU Boulder has an average number of tests of 6.2. We know from above that about 2,037 students of color have been discriminated against. Take that number (2,037) and multiply that by the average number of tests per student (6.2), which gives you 12,629 individual instances of racial discrimination at CU Boulder in 2020.
8. Conclusion. Given what we know about facial recognition technology failure rates and CU Boulder’s data, we can estimate that Proctorio has racially discriminated against 2,037 students 12,629 times.
2,037 students 12,629 times.
One student being discriminated against one time is terrible. This level of racial discrimination is unconscionable. It also doesn’t capture students who are discriminated against because of their disability, medical conditions, parental status, neuroatypical behavior, or trans and non-binary identity. Beyond discrimination, there’s the issue of false accusations of cheating behavior, or false positives. Todd Feathers, the journalist who filed the CORA request, was told by the administration that “we do not maintain any records of this data.” That’s simply not true. Or at least, it’s possible that the software doesn’t save that data for long periods or make it easily exported as a report, but I guarantee you that the data exists. It would be pretty easy to take the number of cases of academic misconduct that determined that students has cheated, divide that by the number of times Proctorio had determined that any student had a suspicion score over 0%, and you’d have the false positive rate of the software. Given that almost every student has a suspicion score over zero regardless if they cheat or not, the false positive rate would likely be over 99%. For any technology, an error rate of 99% is terrible. I’ve written elsewhere why I think universities believe these products regardless of the fact that they don’t work, but it’s still startling to have a university say they won’t even look into it because they don’t maintain the data.
Here’s the breakdown for CU Denver.
1. Find the total student population
Fall 2020: 15,192
2. Find the number of students proctored
2020: 9,322
3. Find the Students Proctored Percentage
Active users 9,322 / total student population 15,192 = 61.3%
4. Find the number or percentage of students of color
2019: 48.3% or 7,224
5. Find the number of students of color who are proctored
Students proctored 61.3% X student of color 7,224 = 4,429 students of color who are proctored.
6. Estimate the number of students who experience discrimination
Facial recognition has a fail rate of 35% for people of color X 4,429 students of color who are proctored = 1,551 students who have experienced racial discrimination by Proctorio.
7. Apply the number of tests per student
Average number of tests per student is 12.1 X 1,551 students who experience racial discrimination = 18,768 instances of racial discrimination.
8. Conclusion
Proctorio has racially discriminated against 1,551students 18,768 times.
I don’t have data for UCC or CU Anschutz, but let’s just add up what we know. Combining data from CU Boulder and CU Denver, we can estimate that 3,588 students of color have experienced racial discrimination by Proctorio 31,397 times.
Fuck.