TrustCyber / machine
/research/allow-hold-deny/lessons/18-appendices-and-synthesis/index.mdtext/markdown5.2 KB ~1,338 tokens
Open raw

Inspect cases and connect the ideas

Use the complete case details and pinned evidence to explain a result.

Canonical HTML page

Learning objective

Use the complete case details and pinned evidence to explain a result.

Appendix A: every disagreement

Table IV records all seventeen JEV–KEV disagreements, with each selected decision and the full Allow/Hold/Deny percentages. Fifteen have Hold references and two Allow; none has Deny. Case headings help navigation and were not model inputs. C258's display heading is corrected without changing its facts, rationale or reference.

Choose C207 or C228 and read the rule, facts and frozen rationale before viewing the model outputs. Explain which required condition remains unestablished. Then compare probability with correctness. Rounded KEV display percentages can sum to 100.01%; these are not the original decimal response bytes.

Appendix B: identify what was tested

CLM source commit is 4858da6fef7e516847ce56e5274077f32cb5b350. KEV source is 5920c5fe4ca8e0970ed4209ac2c9b8e18bea5109 and its adapter is 139fdd94f1b6a6ad80cc15e08fcb99cac885a101. JEV requested and validated version 1.13.0. GLM's observed identifier was z-ai/glm-5.3-flashx-20260918; it is not an exposed immutable weights revision. CLM CUDA parity remains unverified.

Appendix C: all eleven pre-run changes

Inspect C046, C090, C093, C094, C201, C207, C210, C228, C249, C255 and C298 in the changed-reference activity. Every change was proposed Allow to frozen Hold, before testing. The comments explain the author's reference reading rather than the models' internal reasoning. C071's separate rationale correction retained Deny.

Put the explanation together

Pick one case. State its rule and missing or established condition, identify the frozen decision, explain one model's returned answer and probability, then name a limit on interpreting it. For the study as a whole, combine accuracy, coverage, timing and probability provenance before making a comparison.

Go deeper: complete references and original evidence

The reference chapters retain the full bibliography, all eleven changed cases, figures, mathematics and reproduction information. The original PDF and evidence deposit are the sources for the study. This guided explanation and its toy activities are companion additions.

Read the original visual evidence

Every JEV–KEV disagreement is available in the case selector below, including full rounded display percentages.
Table IV · All seventeen disagreements. Every JEV–KEV disagreement is available in the case selector below, including full rounded display percentages. Select the image for full resolution.

Browser teaching tools

The canonical HTML page provides interactive controls. Its chapter explains their evidence, arithmetic, and limitations.

Interactive activities

Use the canonical HTML page for controls, live results and reset buttons.

Check your understanding (optional)

Which synthesis best explains GLM’s results?

Take this forward

You have completed the walkthrough. Revisit any lesson, inspect the reference chapters, or read the original paper with the measurements and limits in mind.

Reference material

Lesson navigation

Companion navigation