It also helps protect against unfairness from the slowly drifting mood of the marker.

At the same time, answers by a mediocre student that happens to be sorted just after answers from a horribly bad student, might seem quite brilliant.

Or seem not so good, if sorted after the best student.

So maybe the answer order should be random.

That's true. Ideally then, as you say, there should be a Fisher-Yates shuffle of the papers after each pass through.