# Accuracy and uncertainty > Compute accuracy with the right denominator and interpret uncertainty. [Canonical HTML page](https://trustcyber.ca/research/allow-hold-deny/lessons/07-accuracy-and-uncertainty/) - Author: Junior Williams - Type: Research companion addition ## Learning objective Compute accuracy with the right denominator and interpret uncertainty. - [Original paper: §III.C; companion mathematics D.1–D.4 · page 2](https://trustcyber.ca/research/allow-hold-deny/original.pdf#page=2) Source pages: 2 ## Count what was assigned **Primary accuracy** is correct valid decisions divided by all assigned cases. **Coverage** is valid responses divided by assigned cases. **Conditional accuracy** uses valid responses only. Missing responses stay in primary accuracy. ## Work the numbers Suppose 300 cases are assigned, 250 answers are valid and all 250 are correct. Primary accuracy and coverage are both 250/300 = 83.33%; valid-only accuracy is 250/250 = 100%. Change both counts to 240: primary accuracy and coverage become 80%, valid-only accuracy remains 100%, and 60 responses are invalid. A **confusion matrix** counts each reference class against each returned decision. **Recall** for a class is its correctly returned count divided by all cases in that reference class. **Balanced accuracy** averages the three recalls. Always returning Hold gets all Hold cases right and all Allow/Deny cases wrong: (0 + 1 + 0)/3 = 33.33%, even though ordinary accuracy is 111/300 = 37%. A Wilson interval describes uncertainty around a case-level proportion under its statistical assumptions. Here, 95% describes the method: across repeated samples meeting its assumptions, about 95% of intervals would cover the underlying success rate. It is not a guarantee for the next action. More observations at the same success rate generally narrow it. It does not account for a different case author, alternate reference readings or repeated inference sessions.
Go deeper: identities and Wilson calculation

Primary accuracy = coverage × conditional accuracy when at least one response is valid. With zero valid responses, conditional accuracy is undefined. For p = correct/n and z ≈ 1.96, Wilson centre = (p + z²/(2n))/(1 + z²/n); half-width = z√(p(1−p)/n + z²/(4n²))/(1 + z²/n). The browser calculates the two bounds. See D.1–D.4 for notation and derivation.

## Browser teaching tools The canonical HTML page provides interactive controls. Its chapter explains their evidence, arithmetic, and limitations. - [accuracy](https://trustcyber.ca/research/allow-hold-deny/lessons/07-accuracy-and-uncertainty/#explore-accuracy): Select GLM and change the denominator to valid responses. Compare 97.38% conditional accuracy with 89% coverage. - [matrix](https://trustcyber.ca/research/allow-hold-deny/lessons/07-accuracy-and-uncertainty/#explore-matrix): Select GLM. Trace correct cells on the diagonal and keep the invalid column in each reference-row total. ## Interactive activities Use the canonical HTML page for controls, live results and reset buttons. - [counts](https://trustcyber.ca/research/allow-hold-deny/lessons/07-accuracy-and-uncertainty/#demo-counts): Change both correct and valid from 250 to 240. Then try zero valid responses with zero correct. These are hypothetical counts, separate from the study. - [wilson](https://trustcyber.ca/research/allow-hold-deny/lessons/07-accuracy-and-uncertainty/#demo-wilson): Keep the same 90% success rate but change 90/100 to 270/300. Observe how the interval narrows with more observations. ## Check your understanding (optional) If 240 of 300 assigned cases are valid and all 240 are correct, what is primary accuracy? - 80%: Correct. Primary accuracy retains every assigned case: 240/300. - 100%: 100% is valid-only accuracy; it omits 60 invalid outputs. - 60%: 60 is the invalid count, not the accuracy percentage. ## Take this forward Next, define exactly when the response clock starts and stops. ## Reference material - [results](https://trustcyber.ca/research/allow-hold-deny/results/index.md) - [mathematics](https://trustcyber.ca/research/allow-hold-deny/mathematics/index.md) ## Lesson navigation - [Previous: What counts as a response?](https://trustcyber.ca/research/allow-hold-deny/lessons/06-valid-responses/index.md) - [Next: What does the clock measure?](https://trustcyber.ca/research/allow-hold-deny/lessons/08-response-time/index.md) ## Companion navigation - [Overview](https://trustcyber.ca/research/allow-hold-deny/index.md) - [Both authorization studies](https://trustcyber.ca/research/authorization/index.md)