Purpose and fit

Use this when people disagree about whether an AI-assisted output is usable. Define the intended use first; an internal draft and a customer-facing response can require different checks.

Before you start

Use the actual task brief, intended recipient and approved references you already have. Add examples of acceptable work if available, but they are not required to draft a rubric. If no reference truth or examples exist, request provisional criteria and questions for a reviewer, not a validated evaluation.

Use only information you are authorised to share with the chosen AI tool. Adjust the prompt to the fields and context you actually have; a specially created dataset is not required.

Required inputs

  • The task, intended recipient and how the output will be used
  • Authorised reference material or expected facts
  • Examples of acceptable work and common defects
  • Consequences of errors and any non-negotiable requirements

Practical workflow

  1. Define the output’s job and the conditions under which a person may use it. List the decision or action that follows acceptance.
  2. Write observable criteria for correctness, evidence, completeness and usability. Add task-specific requirements such as preserving uncertainty or excluding restricted data.
  3. Mark critical failures that block use, regardless of strengths elsewhere. Decide what must be fixed and what can be accepted with a recorded limitation.
  4. Have two reviewers independently assess a few representative examples where practical. Compare disagreements and clarify the rubric before using it for decisions.
  5. Apply the rubric, record the reason for each judgment and revise unclear criteria. Keep rubric versions so comparisons remain interpretable.

Expected output

Expect a task-specific draft acceptance rubric. It does not certify accuracy or replace a domain reviewer; comparisons between reviewers are optional calibration work using existing authorised examples.

A task-specific rubric:

Criterion | observable check | evidence needed | pass condition | fix condition | block condition | reviewer note

Illustrative criteria to adapt:

  • Correctness: Material statements agree with verified reference evidence.
  • Evidence: Sources exist and support the claims attributed to them.
  • Completeness: Required elements and important qualifications are present.
  • Usability: The intended recipient can understand and use the output for the specified task.
  • Critical boundaries: No fabricated material evidence, unauthorised disclosure or unsupported consequential action.

Suggested outcome labels: Accept, Revise and recheck, Reject or escalate. These are workflow decisions, not a universal numeric quality score.

Human checks

Domain reviewers choose the critical failures and inspect important sources. Record judgment disagreements rather than hiding them in an average. If AI helps identify possible defects, a qualified person still checks the relevant evidence and acceptance decision.

Limitations

A clear rubric cannot fix missing source truth or a poorly defined task. Reviewer agreement alone does not prove accuracy. A rubric approved for one output type may be unsuitable for another.

Starter prompt

Draft an acceptance rubric for [specific output] used by [recipient] to [purpose]. Required source material is [approved sources/summary]. Known risks and required elements are [details].

For correctness, evidence, completeness, usability and task-specific boundaries, propose observable checks and distinguish “accept”, “revise and recheck” and “reject or escalate”. Identify critical failures that cannot be averaged away. Do not use a universal quality score. Leave unresolved standards as questions for the domain reviewer. Include a short exercise for two reviewers to compare judgments on supplied examples. Do not claim the rubric is validated.

For each criterion supply an observable check, source evidence needed and explicit accept/revise/block conditions. Separate non-negotiable failures from preferences. Mark standards that require domain input provisional and do not fabricate examples or claim reviewer agreement.

Before drafting, ask a focused question if a missing detail materially changes the task or acceptance criteria. Otherwise produce a clearly limited draft and list the gaps. Label any non-critical assumptions; never invent facts, figures, sources, approvals or completed actions.

Data and AI output disclaimer

Use relevant data that exists or that you provide, with accuracy and completeness sufficient for the task. Different AI models and versions may produce different results, and responses may vary between runs. You are responsible for checking and validating the output against your own requirements, source information and acceptance criteria before relying on it.