TrustCyber / machine
/research/allow-hold-deny/lessons/09-probability-quality/index.mdtext/markdown4.4 KB ~1,121 tokens
Open raw

Understand reported certainty

Distinguish a selected-answer probability from demonstrated correctness.

Canonical HTML page

Learning objective

Distinguish a selected-answer probability from demonstrated correctness.

A distribution describes three choices

The vector [0.47, 0.53, 0] assigns 47% to Allow, 53% to Hold and 0% to Deny. Its largest value selects Hold. A larger selected probability does not guarantee that the answer agrees with the reference. The native classifier probabilities and GLM's generated estimates have different origins.

Score the entire vector

The Brier loss adds the squared differences between probabilities and the reference's one-hot vector, which has one at the reference class and zero elsewhere. With reference Hold, the target is [0,1,0]. Loss for [0.47,0.53,0] is 0.47² + (0.53−1)² + 0² = 0.4418. The exact target scores 0; complete certainty in wrong Allow scores 2. Lower is better, with no division by three.

Brier skill compares mean loss with a fixed baseline forecast. Correctness AUROC asks whether chosen-answer probability ranks correct decisions above incorrect ones; a tied pair receives half credit. Reliability bins group similar reported probabilities and compare them with observed correctness. These measurements answer different questions.

Imagine ten forecasts all at 90% certainty but only six correct. Their observed fraction is 60%, a 30-point gap. That toy group illustrates calibration; ten observations cannot establish general calibration.

Go deeper: loss, skill, rankings and bins

Brier = mean Σ(pₖ−yₖ)². On an ideal three-choice simplex its range is 0–2, and its expected loss is minimized by reporting the true distribution. For true class probabilities r and reported q, expected loss = Σ(qₖ−rₖ)² + 1 − Σrₖ². Only the first term depends on q; its minimum is zero at q = r. Skill = 1 − model loss/baseline loss; the paper's baseline uses full-bank frequencies 89/300, 111/300 and 100/300 on the same eligible subset. For toy correct scores 0.9,0.6 and incorrect scores 0.8,0.6, pair credit is 1 + 1 + 0 + 0.5 = 2.5 of 4, so AUROC = 0.625. Reliability uses ten fixed bins; the last includes 1.0. Bin counts alone cannot reconstruct average confidence or correctness in each bin.

Browser teaching tools

The canonical HTML page provides interactive controls. Its chapter explains their evidence, arithmetic, and limitations.

Interactive activities

Use the canonical HTML page for controls, live results and reset buttons.

Check your understanding (optional)

Which vector has zero Brier loss when Hold is the reference?

Take this forward

Next, classify the possible consequences of an incorrect decision.

Reference material

Lesson navigation

Companion navigation