Why Faculty Should Randomize Grading Order

In elementary school, having a last name that began with the first few letters of the alphabet meant students were in the front of the line for recess. In higher education, a student’s last name could improve their course grades, as well.

Research from the University of Michigan, published last fall, identified a grading bias among students’ assignments that were higher in the alphabet, compared to their peers who held last names toward the end of the alphabet. Students with lower-ranked surnames (initials starting with X, Y or Z, for example) received lower grades, received more negative comments from graders and had lower grading quality.

The cause of the disparity was tied to Canvas, the university’s learning management system (LMS), which automatically organizes assignment submissions for grading by student’s last names. Over time, the assignment-level disparity grows to impact course grades, which researchers theorize could trickle into inequitable job opportunities for students.

The study: Zhihan (Helen) Wang and Jun Li from UM’s school of business partnered with Jiaxin Pei from the school of information to evaluate how the LMS grading system could introduce education inequality.

The four largest LMS—Canvas, Blackboard Learning, D2L Brightspace and Moodle—provide surname-ordered grading as the default option for instructors when evaluating student submissions. However, different factors can impact the fairness of grading in human sequential task performing (or how people complete work in a sequential manner) such as fatigue, contrast effect or generosity-erosion effect.

The three researchers analyzed 30 million assignments graded by humans in Canvas from 2014 to 2022 to understand how the order affects both the grade and comments students receive, as well as students’ post-grade questions and regrade requests.

The results: Overall, assignments graded later in the sequence receive lower grades, around 3.5 points per 100 lower, and students at the end of the grading order had a 0.25 lower GPA. Graders were more likely to leave negative comments on students’ work later in the sequence and be less polite in their language as well.

Students seem to identify this trend, because those with last names lower in the alphabet are more likely to have questions about their grades and are five times more likely to submit regrade requests (compared to the first 10 graded students).

Among the researchers’ sample, 40 percent of assignments were graded in surname order, while 20 percent are graded quasi-randomly, indicating a large share of professors don’t randomize their grading process.

Of the assignments graded in order of last name, there was an even greater impact on sequential grading bias, with students consistently ranking lower if they had a last name at the end of the alphabet. Students with last names from A to E had grades 0.6 points per 100 higher than those with the last name initials U to Z.

Even when graded in sequential order starting with the end of the alphabet, researchers found there was a bias toward those at the front of the line.

The analysis also showed greater gaps in social science and humanities courses compared to science, engineering and medical courses, which researchers attribute to the nature of the assignments, social sciences being more subjective without a single correct answer.

So what? While there is no way to eliminate sequential bias, the world of online learning and digital submissions for coursework makes it so the sequential bias is more prevalent because student assignments are always sorted by last name.

Based on their findings, the researchers recommend LMS providers redesign their grading system to switch from alphabetical order to random order.

On the institutional level, researchers believe this points to the value of balancing work for graders, as fatigue and distraction can impact students’ academic outcomes.

Administrators may need to hire more graders for larger classes to distribute work or randomly distribute work among graders to address the last-name bias. Graders should also be made more aware of grading bias and can receive training to be more aware of the trend.

We bet your colleague would like this article, too. Send them this link to subscribe to our weekday newsletter on Student Success.