Record and review a material AI workflow change

Purpose and fit

Use this before a material change to an existing AI-assisted workflow, including a model switch, revised system instructions, new connector or expanded audience. A small technical edit can alter outputs or permissions. Assess materiality by the expected consequences in this workflow, using the organisation’s change rules. Keep a lightweight record when appropriate, while preserving the evidence needed to distinguish the proposed change from what actually reached users.

Before you start

Use the baseline description, proposed change notes and review criteria already available. An existing version record or a short authorised before-and-after description is sufficient for a first draft; you need not manufacture a test dataset.

If your available data uses different fields or terminology, adjust the prompt to match it. State the intended decision, audience and acceptance criteria; ask for essential missing information, and label any non-essential working assumptions. Do not fill factual gaps with invented information.

Required inputs

• The current approved workflow and configuration baseline, including version references where available.
• A plain-language description of the proposed difference, reason and affected tasks or recipients.
• Relevant acceptance criteria, representative safe test cases and known failures.
• Existing approval requirements, a release owner, recovery options and a post-change observation plan.

Practical workflow

1. Identify the exact before-and-after state. List changed prompts, models, parameters, sources, connections or operating scope, plus configuration the team cannot inspect. Avoid bundling unrelated changes when separating them would improve diagnosis.
2. Assess the possible effect on output quality, data handling, permissions, human review, cost and downstream work. Ask affected owners which checks must be repeated and which requirements block release until resolved.
3. Define a proportionate test plan before running it. Include representative normal cases, known failure cases and relevant permission boundaries. Specify acceptable outcomes, evidence to retain and who will interpret ambiguous results.
4. Run the approved checks in a safe environment. Record failures and unavailable evidence alongside successful cases. Compare with the baseline using the same task definitions, and describe any important configuration differences.
5. Prepare the approval packet. The authorised person records proceed, proceed with explicit conditions, or defer, with the supporting rationale. The release owner checks those conditions before making the authorised change; a drafted approval field is not approval.
6. Verify the deployed state and monitor the agreed early-use window. Check for specified failure signals and use the authorised containment or recovery route if triggered. Close the record only when required evidence and outstanding actions have owners and an explicit disposition.

Expected output

Expect a draft change packet with evidence gaps, proportionate test proposals and release/recovery decision fields. It is not an approved release or a completed validation record.

A change record linked to test evidence and the release record. Copy these fields:

Change ID: [ ]. Workflow/owner: [ ]. Reason: [ ]. Current baseline: [ ]. Proposed change/version: [ ]. Affected users/systems/data: [ ]. Expected consequences: [ ]. Unchanged assumptions requiring confirmation: [ ]. Required reviewers: [ ]. Test cases/criteria: [ ]. Results and gaps: [ ]. Dependencies: [ ]. Recovery/containment limits: [ ].

Approval decision, person, time and conditions: [ ]. Actual release owner/time/version: [ ]. Deployed-state verification: [ ]. Observation window/measures: [ ]. Stop triggers/contact: [ ]. Observed outcome: [ ]. Remaining actions and owners: [ ]. Closure decision: [ ].

For a hosted service without version pinning, record the observable product configuration and dates, plus what cannot be reproduced or controlled.

Human checks

The workflow owner checks whether scope expanded beyond earlier approval. Technical owners verify the deployed configuration and actual recovery behaviour. Relevant security, privacy, legal or domain specialists address questions within their expertise. Keep test evidence accessible to approvers, with sensitive details restricted to those who need them.

Limitations

Tests cannot guarantee behaviour on unseen inputs or identify every provider-side change. Some external actions cannot be reversed; containment may be the only available response. This proposed record supports change discipline and does not replace mandatory organisational or specialist review.

Responsible use

Use relevant data that exists or that you provide, with accuracy and completeness sufficient for the task. Different AI models and versions may produce different results, and responses may vary between runs. You are responsible for checking and validating the output against your own requirements, source information and acceptance criteria before relying on it.