Lesson 10 of 18
Different errors, different effects
Step 10 in the guided paper walkthrough.
By the end of this lesson
Classify the hypothetical effect of an incorrect decision.
Preserve the direction of the error
Calling a Hold or Deny case Allow is a hypothetical unauthorized allow: proceeding without established authority. Calling an Allow or Hold case Deny is a work refusal. Calling an Allow or Deny case Hold is an escalation. Invalid output has its own category because no valid decision arrived.
Compare two mistakes
If approved work is labelled Hold, the system may wait unnecessarily. If a prohibited action is labelled Allow, it may proceed without authority. Both lose one point of accuracy, but their possible effects differ. The paper counts these categories without severity weights.
No corresponding action was carried out. These are hypothetical control consequences, not observed harms. A severity weight expresses a reader’s illustrative preference; the experiment did not measure its real-world cost.
Go deeper: categories partition the errors
Each wrong valid three-label decision belongs to exactly one of the three categories. Invalid responses remain separate. Summing unweighted categories and invalid responses recovers the total assigned-case errors. A weighted total is an extra teaching calculation, not a new scientific score from the paper.
Try it
Explore the idea
Select a reference/answer pair and classify its hypothetical effect.
Check your understanding · optional
Reference Hold, returned Allow: which category applies?
Take this forward
Next, see the configurations and controls behind these measurements.
Paper and reference material
Original paper: §III.C · page 3 (new tab)
Source pages: 3
The teaching explanation and interactive examples are companion additions. Recorded study values and frozen references remain unchanged.