AI agent authorization studies
When should an agent act, wait for approval, or refuse? Read two studies of decisions at that boundary.
The studies use separate case banks and protocols. Their results are not pooled. Each evaluates decisions under supplied written policies; neither executes the proposed actions.
Allow, Hold, or Deny: Comparing the Speed, Accuracy, and Reported Certainty of Four AI Models
A 300-case comparison of CLM, JEV, KEV, and GLM 5.3 FlashX examines accuracy, response time, and reported certainty when authorizing agent actions. The results show tradeoffs across the tested configurations, with no general winner.
Jev at the Agent Authorization Boundary: Evaluating TypeSafe’s Decision Model on Allow, Hold, and Deny
Two paired evaluations compare Jev with GPT-4.1 mini and DeepSeek V4.1 Flash on 270 constructed cases. The study examines false allows, withheld authorized actions, and refusals of actions that could proceed with human approval.