An assessment question can become a procurement problem surprisingly quickly. If the buyer needs to explain how a system was tested for unfair impacts, but the supplier will provide only a marketing summary, the missing evidence cannot be repaired by writing a more confident answer in the form.
Canada’s Directive on Automated Decision-Making makes the Algorithmic Impact Assessment part of the governance of covered federal systems. The procurement implication is practical: identify the information and review rights the buyer will need while there is still an opportunity to secure them.
First establish whether the directive applies
The directive concerns production systems used to make an administrative decision or a related assessment about a client; systems used solely for research, experimentation or testing are excluded (sections 5.1–5.2). Applicability also depends on institutional provisions in section 8 and timing provisions in section 1. Provincial and municipal buyers must examine their own instruments; federal requirements are not a universal Canadian AI procurement rule.
For a covered system, section 6.1.1 requires completion, approval and publication of the final AIA before production, subject to the institutional provisions. Section 6.1.3 requires scheduled review, approval and updates, including when functionality or scope changes. The responsible institution needs a way to obtain the supplier information those duties require.
| Need to establish | Supplier contribution | Buyer responsibility |
|---|---|---|
| What role automation plays | Workflow and component documentation | Confirm how the actual service uses the output |
| How performance was tested | Methods, results and known limitations | Assess relevance to the intended population and use |
| How decisions can be examined | Version information and accessible records | Verify a sample decision can be reconstructed |
| What changed after acceptance | Change notices and updated documentation | Review the assessment and affected controls |
A completed supplier-dependency record
Hypothetical record DEP-01 concerns a federal department procuring software that recommends eligibility decisions about clients, with applicability already confirmed. This is a separate use from the internal-report example. The delivery and acceptance terms below are proposed procurement mechanisms, not prescribed contract wording.
| Record field | Completed example |
|---|---|
| Assessment issue | Can the institution explain the relevance and limitations of testing for accuracy and unfair impacts on its intended client population? |
| Supplier-controlled evidence | Versioned model/build manifest V-3 and test methods/results TEST-3, including population coverage and known limitations. Received: aggregate results and a build name; missing: coverage breakdown and reproducible version reference. |
| Buyer owner and delivery point | Program director owns assessment completeness; evaluation lead reviews evidence. Procurement lead makes the missing materials due before acceptance review and before the final AIA approval/publication required for production. |
| Decision | Hold production acceptance. Supplier must provide TEST-3 detail and V-3 identification under agreed review/access safeguards; evaluation lead must determine whether further institution-specific testing is needed. |
| Refresh dependency | Record supplier notice and refreshed evidence for functionality/scope changes, plus the institution’s scheduled review. The responsible federal official retains the AIA review, approval and update duty. |
Acceptance should include a traceable decision
Select a representative case and follow it from input to outcome. Identify the software and model versions, the information used, the rules applied, and any human intervention. Examine whether the records support the explanation the institution needs to give. A dashboard showing a success rate cannot answer all of those questions.
The directive includes testing for accuracy, unintended bias, and unfair impacts, together with ongoing outcome monitoring. Procurement teams should work with service, policy, privacy, security, and legal owners to determine the test access and documentation needed for their system. The supplier’s own benchmark may inform that work without being sufficient for the institution’s decision. See sections 6.3.1–6.3.3.
Write for the second version
The first accepted release will change. A supplier may replace a model, revise a decision rule, or introduce a new data source. Access to released component versions and appropriate review, testing, monitoring, and audit rights help the institution understand those changes. Confidentiality safeguards can be part of the arrangement without making the evidence inaccessible. The component-access requirements appear in sections 6.2.5–6.2.6.
Keep unresolved supplier dependencies visible before contract award. Where evidence cannot be obtained, the institution needs to decide how that affects the proposed use, procurement terms, or acceptance. The AIA is easier to maintain when the contract has already made its questions answerable.
Make supplier dependencies visible
AI governance advisory can help connect assessment evidence needs to ownership and procurement decisions.
Selected primary sources
Open the primary-source pages used to verify the claims summarized here.