TrustCyber / machine
/research/allow-hold-deny/lessons/15-speed-and-tradeoffs/index.mdtext/markdown4.2 KB ~1,086 tokens
Open raw

Compare speed and consequences

Interpret measured response times and explore illustrative error weights.

Canonical HTML page

Learning objective

Interpret measured response times and explore illustrative error weights.

Read the observed speed

Compliant-response median HTTP times were JEV 139.1 ms, KEV 659.7 ms, CLM 1113.2 ms and GLM 2966.5 ms. Their p95 values were 211.1, 678.3, 1183.1 and 5146.6 ms. KEV's median was 4.74 times JEV's in this run, comparing a local MLX system with a hosted service.

JEV's summed scored HTTP intervals were 43.80 seconds, scored loop 118.53 seconds and full session 127.21 seconds. KEV's were 199.07, 270.89 and 412.01 seconds. The session includes work beyond response transfers; KEV's also includes cooldown and startup.

Give each error its own count

JEV had nine unauthorized allows, sixteen refusals and three escalations. KEV had eleven, thirteen and three. CLM had 189 escalations. GLM had two unauthorized allows, four refusals, one escalation and 33 invalid outputs. GLM's missing answers prevent interpreting its lower unauthorized-allow count alone as greater conservatism.

Equal teaching weights give totals CLM 189, JEV 28, KEV 27, GLM 40. Increase the unauthorized-allow weight to 5: JEV becomes 64 and KEV 71. These weights express illustrative preferences; no severity or real cost was measured.

Go deeper: the weighting identity

JEV minus KEV = −2 × unauthorized-allow weight + 3 × refusal weight. Escalation and invalid counts cancel for those two models. With weights 3 and 2 they tie. This sensitivity calculation leaves the original results and labels unchanged.

Go deeper: slower failures and early-response diagnostics

GLM's median over all responses was 3172.1 ms; over its invalid responses, median was 6004.5 ms and p95 6812.6 ms. The prespecified first-ten-versus-rest compliant median ratios were CLM 1.001, JEV 0.967, KEV 1.003 and GLM 1.050. This compares early responses with the remaining responses; no early requests were removed because of it. These diagnostics do not replace the compliant-response headline metric.

Browser teaching tools

The canonical HTML page provides interactive controls. Its chapter explains their evidence, arithmetic, and limitations.

Interactive activities

Use the canonical HTML page for controls, live results and reset buttons.

Check your understanding (optional)

Does the lower weighted total prove a system is safer to deploy?

Take this forward

Next, identify where interpretation must stop.

Reference material

Lesson navigation

Companion navigation