Purpose and fit

Use this before a pilot starts or before a team makes a claim about its value. The measurement should cover a complete work item through acceptance, including the human review and rework that occur after a first draft.

Before you start

Use existing task logs, time records, quality checks or a short approved description of the workflow. Tell the AI which fields, units and observation windows actually exist. If there is no baseline yet, ask for a measurement plan and collection fields, not a retrospective result. No fabricated observations are needed.

Use only information you are authorised to share with the chosen AI tool. Adjust the prompt to the fields and context you actually have; a specially created dataset is not required.

Required inputs

  • A defined unit of work and the people who receive it
  • A baseline observation window and a proposed pilot window
  • Acceptance criteria, known failure cases and available quality records
  • Human effort, tool costs and other implementation or support costs that can actually be observed

Practical workflow

  1. Define a comparable work item and the task types included. Note important differences in complexity or users across the baseline and pilot.
  2. Choose a small set of measures before observing results: useful accepted work, quality, human effort, elapsed time, cost and adverse events where relevant.
  3. Record the baseline using the same definitions. If the baseline is weak, identify it as a limitation rather than filling gaps with an AI estimate.
  4. During the pilot, record every included attempt and its final outcome. Count review, corrections, abandoned attempts and fallback work. Keep exclusions and their reasons visible.
  5. Compare results by task type and decide whether the observed evidence meets the agreed continuation criteria. Record alternative explanations and what additional observation is needed.

Expected output

Expect a usable measurement-plan draft. If actual observations are supplied, any resulting calculation is provisional and must be recomputed from the original records before use; this resource does not provide a tested calculator.

A measurement sheet with two parts:

  • Definitions: Measure; unit; data source; inclusion/exclusion rule; baseline window; pilot window; owner; decision threshold.
  • Observed work: Task ID; task type; approach/version; accepted/rejected; defect type; human drafting time; review/rework time; elapsed time; attributable tool cost; fallback/incident; reviewer.

Useful calculations, when the inputs are available:

  • Acceptance rate = accepted items ÷ all evaluated items, with both counts reported.
  • Human effort per accepted item = all recorded human effort on included attempts ÷ accepted items. If none are accepted, report the counts and mark the ratio undefined.
  • Record setup effort and ongoing effort separately, so neither disappears from the decision.

Human checks

Reviewers apply the same acceptance rules across comparisons. The initiative owner checks missing records and comparable task mix. Finance or the appropriate budget owner validates any monetary interpretation. Time released is not automatically a reduction in cash spending; report any conversion as a separate, evidenced decision.

Limitations

A short pilot may reflect learning effects, unusual workload or participant selection. A before-and-after comparison alone does not establish causation. Low incident counts over a small window cannot establish low risk.

Starter prompt

Create a measurement plan for [bounded AI-assisted workflow]. Our baseline information is [insert approved summary]. Our acceptance rules and decision criteria are [insert rules].

Propose observable measures for accepted work, quality, total human effort, elapsed time, costs and failure consequences. For every measure, give the definition, unit, denominator if relevant, source, missing-data treatment and responsible owner. Include failed attempts, review, rework and fallback work.

Separate measured facts from estimates and unknowns. Flag task-mix or timing differences that weaken a comparison. Do not calculate ROI, realised savings or causal improvement unless the supplied evidence supports those claims.

Return definitions, units, sources, inclusion rules, denominators, missing-data treatment and owners for each proposed measure. Keep setup and ongoing effort separate. For any requested calculation show the included counts, source rows, formula and units; do not treat missing values as zero, combine incompatible units, or calculate a ratio with a zero denominator.

Before drafting, ask a focused question if a missing detail materially changes the task or acceptance criteria. Otherwise produce a clearly limited draft and list the gaps. Label any non-critical assumptions; never invent facts, figures, sources, approvals or completed actions.

Data and AI output disclaimer

Use relevant data that exists or that you provide, with accuracy and completeness sufficient for the task. Different AI models and versions may produce different results, and responses may vary between runs. You are responsible for checking and validating the output against your own requirements, source information and acceptance criteria before relying on it.