Write an AI incident postmortem from trace evidence

An AI incident review should reconstruct what the system did and why its controls failed. A transcript alone rarely shows retrieval, versions, tool decisions, validators or missing alerts.

Connect the timeline to corrective work

The LLMOps and security lab creates a three-span fixture: retrieval succeeds, prompt-5 produces an unsupported claim and gate-2 misses it.

from llmops_cases import incident_case

result = incident_case()
assert [x["span"] for x in result["trace"]] == ["retrieve", "generate", "output_gate"]
assert result["postmortem"]["trigger"] == "prompt-5"
assert result["postmortem"]["corrective_action"] == "add regression I-17"
assert result["blame_field_present"] is False
assert result["real_incident"] is False

This is an authored scenario, not a NeuraPath or customer incident.

Establish facts before causes

Record discovery, start and containment times; affected users/tasks; data and side effects; current and previous release manifests; and the evidence sources. State what is known, inferred and unknown. Preserve privacy-minimized traces, evaluation rows, approvals and configuration under the incident retention process.

Build a timeline across ingress, policy, retrieval, generation, validation and tools. Ask why prevention failed, why detection failed and why impact was not smaller. “The model hallucinated” is a symptom. The system-level causes may include an unsupported task, stale source, weak evidence contract, missing validator, changed prompt or absent regression.

Describe immediate containment separately from permanent correction. Disable a route or revert a release first; then add a test, improve a gate, repair observability and update the threat model. Give every action an owner, due date and verification method. Link the new regression ID to the incident.

Learn without inventing certainty

Avoid blame and unsupported root-cause claims. Record competing hypotheses and experiments. Include what worked, such as a fast rollback, so effective controls remain visible. Review completion only after actions are verified.

The Generative & Agentic AI course connects traces, versioning, rollback and regression evidence into one incident workflow.

Reproducible exercise

Inject a known unsupported output into a local trace, write the timeline, add its case to the gate and prove the same release bundle can no longer pass.

Continue learning

This article is part of the LLMOps, security and operational evaluation sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Measure disagreement between automated graders.
• Continue with Build a release gate that blocks a known regression.

Reference: NIST AI RMF Generative AI Profile.