The demonstration ends with a convincing answer on screen. The buying decision begins with everything the demonstration did not show: where the input travelled, whose credentials were used, which model produced the answer, and what the customer could recover if the service changed tomorrow.

Where does our information go?

Ask the vendor to trace one representative transaction. Follow the prompt, retrieved records, response, diagnostic logs, backups, and support access. Identify the model providers and subprocessors along the route. Data residency, retention, and training use should be explicit for each relevant service and configuration.

A setting in an administration console establishes a configuration option. A contract establishes a commitment. Ask which protections are enabled by default, which require a paid tier, and which the supplier is prepared to warrant. Keep those answers attached to the version and service being purchased.

What can the product actually change?

For an agentic product, inspect a tool call. A friendly interface may sit over a service account with broad permissions. Ask how the product identifies the acting user, limits delegated authority, handles denied actions, and prevents a retry from widening the operation. Request an example log from the configuration you expect to deploy.

Turn a claim into an acceptance request
Vendor claimAsk to see
“Your data stays private”The data flow, retention settings and applicable contract terms
“Every action is auditable”An exportable trace from task through downstream result
“Humans remain in control”A consequential action held for specific approval
“Updates are seamless”Notice, evaluation and rollback arrangements
“You can leave at any time”A usable export and a documented deletion process

A completed vendor decision brief

Hypothetical procurement record V-01: an operations team wants an agent to draft an internal weekly report and send approved copies. The evidence and outcomes below are invented to show how a buyer can record a decision; they are not results from a client engagement or product test.

V-01 — decision after reviewing the supplier pack
Record fieldCompleted example
Claim and received evidenceSupplier claims recipient-specific human approval and exportable logs. Pack E-01 contains an approval-setting screenshot, a sample event export and a proposed retention schedule.
Unresolved dependencyThe export omits the approved recipient and payload reference. It cannot establish that approval covered the message actually sent. Incident-log access terms remain unsigned.
Accountable ownersProcurement lead owns the evidence-access terms. Integration lead owns the representative send/retry check. Operations director owns acceptance of the use.
Decision and conditionsHold purchase acceptance for automated sending. Accept a draft-only evaluation with synthetic reports, no mail credential and review by the operations lead. Manual dispatch remains the comparison option.
Evidence required to reconsiderSupplier must provide a trace linking approval, recipient, report version, expiry and downstream request ID, including a changed-recipient refusal. Procurement must agree log access and retention in writing before the acceptance review.

What will we receive when something goes wrong?

Agree on the records available during an investigation: timestamps, request identifiers, model and software versions, tool events, and relevant access history. Establish how long those records remain available and how the customer requests them. A general promise to cooperate can be difficult to use during a time-sensitive incident.

Who decides when the product changes?

Suppliers need to improve their services. Buyers need to know when a change affects the approved use. Discuss model substitutions, new subprocessors, changed data use, expanded tool permissions, and altered service locations. Decide which changes require notice, testing, or agreement before they reach the customer’s workflow.

NIST’s AI Risk Management Framework is voluntary. Its Playbook offers suggested actions, rather than a checklist to apply in full. The questions here are procurement recommendations informed by that approach; contractual rights must fit the transaction and the buyer’s obligations.

Can we test our own difficult cases?

Bring a small acceptance set that includes ambiguous requests, restricted information, unavailable dependencies, and actions the system should refuse. Write down the expected behaviour before running it. Evaluate the path the product takes as well as its final answer, and preserve unresolved failures in the purchase decision.

Can we leave with something usable?

Before signing, examine the export format, migration assistance, access to relevant logs, and deletion arrangements. Run a sample export if the product allows it. The exit terms are part of the architecture decision: they determine how much of the operating record, configuration, and business continuity remains under the buyer’s control.

Turn supplier evidence into a buying decision

Enterprise architecture advisory connects supplier commitments, technical dependencies and acceptance conditions.

Selected primary sources

Open the primary-source pages used to verify the claims summarized here.