# Build a reviewed case bank > Identify model inputs and the reference used to score them. [Canonical HTML page](https://trustcyber.ca/research/allow-hold-deny/lessons/05-cases-and-reference/) - Author: Junior Williams - Type: Research companion addition ## Learning objective Identify model inputs and the reference used to score them. - [Original paper: §III.A · page 2](https://trustcyber.ca/research/allow-hold-deny/original.pdf#page=2) Source pages: 2 ## A case has three pieces A **rule** states the policy. **Facts** describe the situation. A **proposed action** says what the agent intends to do. The proposed action does not prove its prerequisites. Each case also has a reviewed answer used for scoring: its **reference label**. Codex drafted 300 synthetic situations, proposed labels and rationales. Junior Williams was the sole human reviewer. He changed eleven proposed Allow references to Hold before testing and corrected C071's rationale while keeping its Deny label. The resulting **frozen key** contained 89 Allow, 111 Hold and 100 Deny answers. ## Follow C207 through review The draft proposed Allow for a read-only workbook inspection. Review found that the facts did not establish authorization for the target workbook. Its frozen reference became Hold before inference. A later model answer is compared with that fixed key; the key is not adjusted to fit the answer. Models saw policy, identifier, rule, facts, action, choice order and response instructions. They did not see references, rationales, review comments or class totals. Freezing the key prevents outcome-driven rescoring; it does not establish independent agreement about every interpretation.
Go deeper: what single-reviewer means

The key records one author's reviewed policy reading. It is not a consensus of independent human raters or a random sample of deployment events. Later lessons inspect the possible consistency issue in C164/C270 without changing their frozen labels.

## Explore the idea Choose an item and see whether the model received it. - Rule, facts and proposed action: Model-visible: these describe the decision problem. - Frozen reference and rationale: Withheld: these were used for evaluation and interpretation, not supplied as the answer. - Choice order: Model-visible: six saved permutations each occurred 50 times. ## Interactive activities Use the canonical HTML page for controls, live results and reset buttons. ## Check your understanding (optional) When were the eleven Allow-to-Hold changes made? - Before scored model testing: Correct. The review changes were reconciled before the key was frozen and used for inference. - After seeing which model won: The paper preserves the pre-run review history. - Every time an answer looked plausible: Scoring retained the frozen key. ## Take this forward Next, separate a valid answer from an incomplete or malformed response. ## Reference material - [study-design](https://trustcyber.ca/research/allow-hold-deny/study-design/index.md) - [reference-changes](https://trustcyber.ca/research/allow-hold-deny/reference-changes/index.md) ## Lesson navigation - [Previous: Allow, Hold or Deny](https://trustcyber.ca/research/allow-hold-deny/lessons/04-policy-decisions/index.md) - [Next: What counts as a response?](https://trustcyber.ca/research/allow-hold-deny/lessons/06-valid-responses/index.md) ## Companion navigation - [Overview](https://trustcyber.ca/research/allow-hold-deny/index.md) - [Both authorization studies](https://trustcyber.ca/research/authorization/index.md)