Write an analyst prompt with a metric definition and evidence contract

An analyst prompt should specify the question's population, period, grain, units and evidence requirements. “Analyze revenue and give insights” leaves the assistant to invent definitions that may not match the business.

A precise prompt reduces ambiguity, but it does not enforce correctness or tool permissions. Pair it with deterministic validation and an explicit review boundary.

Write the contract before polishing the wording

The original prompt contract asks for January completed-order amount from the supplied synthetic orders. It defines one order ID as the grain, completed status as eligibility, local Asia/Kolkata time, an inclusive start and exclusive end, and integer paise as the unit.

It also states two consequential rules: use the supplied order total without subtracting discounts again, and retain eligible orders whose customer dimension record is missing.

Those rules are more valuable than a request to “act as a world-class analyst.” They describe what the correct calculation actually means.

Make the output reviewable

The success object must include metric identity, period, timezone, currency, unit, value, eligible-order count, evidence order IDs and structured descriptive claims.

A human-readable answer can be generated from that object, but the structured facts remain available for checking. Require every included order once in the evidence list so a reviewer can identify omissions or duplication.

The contract forbids inventing prior-period changes, causal explanations or business outcomes. If required source evidence is missing or inconsistent, the assistant should return a separate needs-data response rather than fabricate a numeric success object.

The current numeric checker treats that abstention as outside its success schema. A complete application should route it to an appropriate review or data-request path, not count it as a successful calculation.

Test the checker against the requested structure

from copy import deepcopy
from calculator import seeded_cases,check_candidate

base = seeded_cases()[0]['answer']
assert check_candidate(base)['structured_checks_passed'] is True
missing = deepcopy(base)
del missing['unit']
assert check_candidate(missing)['issues']==['answer_schema_mismatch']
wrong_period = deepcopy(base)
wrong_period['period_end']='2026-02-02T00:00:00'
assert 'period_end_mismatch' in check_candidate(wrong_period)['issues']
duplicate_evidence = deepcopy(base)
duplicate_evidence['evidence_order_ids'].append(duplicate_evidence['evidence_order_ids'][0])
assert 'evidence_ids_mismatch' in check_candidate(duplicate_evidence)['issues']
print({'required_field_checked':True,'period_checked':True,'duplicate_evidence_checked':True})

These are seeded local examples, not a claim that the prompt has achieved a particular success rate with any model. The tests establish selected behavior of the application checker.

Keep source content below application instructions

An uploaded document may contain text that looks like instructions. Treat it as data to inspect, not as authority to change the metric definition or authorize a tool action.

The prompt can state this boundary, but the application must enforce the permitted tools and data access independently. A request to use read-only tools is not equivalent to a database permission that prevents writes.

Similarly, a retrieved glossary definition may be stale or conflict with the selected metric contract. Record its version and resolve the conflict rather than allowing whichever passage appears first to redefine the calculation.

Avoid leaking the answer into an evaluation prompt

For teaching, a reference answer helps explain the method. For evaluating a model's ability to perform the task, keep the expected answer and held-out cases outside the prompt unless supplying that answer is intentionally part of the task.

Otherwise a high score may measure copying rather than analysis. Record the exact prompt version, supplied tools and source context when comparing runs so the task remains comparable.

Review meaning after structure

A claim labeled descriptive can still contain an unsupported causal statement. The lab includes that deliberate blind spot. The checker does not parse arbitrary prose into a complete evidence model, so a structural pass still requires semantic review.

Exercise: rewrite the prompt for average completed-order value. Define numerator and denominator explicitly, update the output contract and create a case where the total is correct but the order count is wrong.

NeuraPath's Data Analytics with Generative AI course connects prompt design with measurable verification. A strong analyst prompt makes the task precise and leaves enforcement to controls that can actually check it.

Continue learning

This article is part of the Generative AI for verified analyst work sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Verify an AI-generated revenue answer against source rows.
• Continue with AI-generated SQL: test the query before trusting the explanation.