Research · Junior Williams

AI agent authorization studies

When should an agent act, wait for approval, or refuse? Read two studies of decisions at that boundary.

The studies use separate case banks and protocols. Their results are not pooled. Each evaluates decisions under supplied written policies; neither executes the proposed actions.

First page of Allow, Hold, or Deny: Comparing the Speed, Accuracy, and Reported Certainty of Four AI Models.

Four-model comparison · September 2026

Allow, Hold, or Deny: Comparing the Speed, Accuracy, and Reported Certainty of Four AI Models

A 300-case comparison of CLM, JEV, KEV, and GLM 5.3 FlashX examines accuracy, response time, and reported certainty when authorizing agent actions. The results show tradeoffs across the tested configurations, with no general winner.

Read the four-model companion Manuscript DOI
First page of Jev at the Agent Authorization Boundary: Evaluating TypeSafe’s Decision Model on Allow, Hold, and Deny.

Earlier JEV study · September 2026

Jev at the Agent Authorization Boundary: Evaluating TypeSafe’s Decision Model on Allow, Hold, and Deny

Two paired evaluations compare Jev with GPT-4.1 mini and DeepSeek V4.1 Flash on 270 constructed cases. The study examines false allows, withheld authorized actions, and refusals of actions that could proceed with human approval.

Read the earlier JEV study