Build an AI report review workflow with human sign-off

Human sign-off should apply to an exact report version and its evidence. If an assistant changes the narrative, period or source after review, the earlier decision must not silently carry forward. Store the reviewed artifact identity and check it before any later use.

This tutorial demonstrates local review-record consistency. It does not authenticate a reviewer, send a report or represent a real person's approval. Those distinctions matter when a prototype is used to explain a production design.

Define the review packet

The original review component combines a structured candidate answer with hashes of its source, metric contract and deterministic evidence. A canonical serialization produces a packet hash.

A review record contains that packet hash, a reviewer field, a decision and a flag indicating that semantic review was recorded. The checker requires valid structured facts, current evidence and an exact version match.

The reviewer field is plain local data in this teaching implementation. Anyone who can write the record can type a name, so the record is not an authentication boundary. A real application must obtain identity and authorization from its trusted account system.

Demonstrate an edit invalidating the record

from copy import deepcopy
from calculator import seeded_cases
from review import make_packet,digest,check_review

answer = seeded_cases()[0]['answer']
packet = make_packet(answer)
decision = {'packet_sha256':digest(packet),'reviewer':'synthetic-reviewer',
            'decision':'approve','semantic_review_passed':True}
checked = check_review(packet,decision)
assert checked['record_checks_passed'] is True
assert checked['distribution']=='not_sent'
edited = deepcopy(packet)
edited['answer']['claims'][0]['text'] += ' Additional wording after review.'
changed = check_review(edited,decision)
assert changed['record_checks_passed'] is False
assert 'reviewed_version_mismatch' in changed['issues']
rejected = deepcopy(decision)
rejected['decision']='reject'
assert check_review(packet,rejected)['record_checks_passed'] is False
print({'synthetic_record_matches':True,'post_review_edit_rejected':True,
       'real_human_approvals':0,'distribution':'not_sent'})

The added sentence does not change the total, yet it changes the packet. That is intentional: prose can introduce a new unsupported claim even when every numeric field stays identical.

Separate checks from the review judgment

The deterministic checker verifies amounts, counts, units, periods and evidence IDs. A human review should assess whether the narrative is supported, whether limitations are clear and whether the proposed decision is appropriate.

A stored semantic_review_passed: true flag records a claimed review outcome. The code cannot establish that a person actually read the report or reached a sound judgment. Do not describe the flag itself as automated semantic verification.

For a real workflow, present the exact packet to an authorized reviewer, capture the decision through a controlled interface and retain an audit record. Regenerating the report after approval should produce a new review requirement.

Use explicit states

A useful design distinguishes prepared, failed checks, awaiting review, changes requested, approved for a named use and distributed. Approval for an internal analysis should not imply permission for public publication or a different audience.

This local lab stops at record validation and has no distribution function. A production sender would need to check the authorized action, current artifact version and idempotency before performing an external side effect.

Source corrections deserve the same treatment as narrative edits. If the evidence hash changes, rerun the calculation and request review of the new packet. Do not patch the stored hash merely to keep an old approval green.

Test the failure path

The extension suite checks a matching synthetic record, post-review edits, an explicit rejection and a stale source hash. Add missing-reviewer and incomplete-semantic-review cases when extending the interface.

The value of the workflow is that an ambiguous “looks good” comment cannot accidentally apply to an evolving file. The report, evidence, decision and intended use should remain connected.

Exercise: change only the period label in a copy of the packet. Explain why both metric checking and review-version binding should fail, and identify which failure requires recalculation versus a new review.

NeuraPath's Data Analytics with Generative AI course connects AI-assisted reporting with accountable handover. A review workflow is useful when a later reader can determine exactly what was checked and approved.

Continue learning

This article is part of the Generative AI for verified analyst work sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Know when a spreadsheet formula is better than an AI agent.
• Continue with Detect prompt injection inside an uploaded business document.