# What is the paper asking?

> Explain the research question without assuming a universal winner.

[Canonical HTML page](https://trustcyber.ca/research/allow-hold-deny/lessons/02-the-research-question/)

- Author: Junior Williams
- Type: Research companion addition

## Learning objective

Explain the research question without assuming a universal winner.

- [Original paper: Abstract and §I · page 1](https://trustcyber.ca/research/allow-hold-deny/original.pdf#page=1)
Source pages: 1
## Permission has conditions

An agent can know how to send a message while lacking permission to send it. A rule might require approved final text, a verified recipient list and an unused single-send approval. Knowing how to operate the email tool establishes none of those facts.

The paper asks how four tested configurations compare on **accuracy**, **response time** and **reported certainty** when given the same 300 authorization situations. Accuracy means agreement with the reviewed answers. Response time measures a defined HTTP interval. Reported certainty comes from the probabilities returned with a decision.

## Start with a small contrast

If an approval covers a draft, and the final text changes, the system needs to check whether approval covers the final text. An efficient tool call could still carry out an unauthorized send. A correct decision and a fast response answer different questions.

The four labels CLM, JEV, KEV and GLM identify the study's configurations. CLM and KEV ran locally; JEV and GLM used hosted services. Their exact settings appear later, before interpreting results.

<details><summary>Go deeper: the planned expectation</summary><p>The planned expectation was that the tested native Choice configurations might follow the policy more reliably than the generative comparator. CLM's always-Hold outcome prevented uniform support for that expectation. The paper describes observed tradeoffs rather than declaring a general winner.</p></details>

## Explore the idea

Compare two possible conclusions.
- A model was faster in this run: This is a statement about an observed timing metric under the tested conditions.
- The model will always be faster: The study did not repeat dates, provider loads or machines to establish that prediction.
- The model can authorize all real actions safely: The study measures written-policy classification and supplies no deployment safety certification.

## Interactive activities

Use the canonical HTML page for controls, live results and reset buttons.

## Check your understanding (optional)

What comparison does the paper support?
- These configurations under this recorded protocol: Correct. The case bank, settings and conditions define the experiment.
- The best model for every deployment: One authored bank and one session cannot establish that.
- Which model safely executes actions: The proposed actions were never executed.

## Take this forward

Next, learn the existing ideas that help distinguish authorization, completion and probability quality.

## Reference material


## Lesson navigation

- [Previous: Meet models and agents](https://trustcyber.ca/research/allow-hold-deny/lessons/01-ai-and-agents/index.md)
- [Next: Ideas behind the experiment](https://trustcyber.ca/research/allow-hold-deny/lessons/03-related-ideas/index.md)

## Companion navigation

- [Overview](https://trustcyber.ca/research/allow-hold-deny/index.md)
- [Both authorization studies](https://trustcyber.ca/research/authorization/index.md)