Build a redaction check for generated output

Input filtering cannot guarantee safe output. Sensitive content may come from retrieval, tool results, memory, a model’s learned behaviour or a prompt-injection path. Apply an output control before content crosses into the browser, message system or downstream API.

Redact synthetic patterns

The LLMOps and security lab replaces a reserved-domain email and a fake token pattern.

from llmops_cases import output_redaction_case

result = output_redaction_case()
assert result["redacted"] == [
    "Contact [REDACTED_EMAIL] about order ORD-42",
    "Token [REDACTED_TOKEN] should never appear",
]
assert result["email_remaining"] is False
assert result["token_remaining"] is False
assert result["raw_persisted"] is False

Regex catches only known formats. It does not identify every name, address, secret, encoded value or context-dependent disclosure.

Classify before choosing an action

Define protected data classes and destinations. A support agent may be allowed to show a verified customer their own masked order number but must not expose another customer’s record. Pair pattern detectors with entity recognition, exact secret scanning, source authorization and structured-field policies. Normalise Unicode and inspect rendered links or markup where relevant.

Choose reject, redact, mask or require review per class. Preserve meaning when safe: a masked last-four value may help the user, while a credential should be removed completely and rotated if exposure is suspected. Avoid echoing the detected value in error text.

Run the check on the complete assembled output, including citations and tool-rendered fields. Streamed output needs buffering or a streaming-safe detector because already-sent bytes cannot be recalled. Validate encoded and obfuscated variants and improper HTML/command handling.

Verify every sink

Use seeded synthetic canaries and assert they are absent from the user response, traces, exception logs, analytics and review queues. Record detection type and action without retaining raw content. Keep human review access narrow and time-limited.

The Generative & Agentic AI course places redaction after model generation and before external side effects.

Reproducible exercise

Create a corpus of emails, phone formats, account IDs, fake keys and encoded variants. Measure false negatives and false positives by class, then add destination-specific policy tests.

Continue learning

This article is part of the LLMOps, security and operational evaluation sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Set per-user quotas for expensive AI operations.
• Continue with Threat-model an AI application from data flows.

Reference: OWASP LLM02:2025 Sensitive Information Disclosure.