Three LLM portfolio pieces, specified with acceptance criteria

What makes a spec, not an idea

"Build a chatbot for PDFs" is an idea. A spec answers: for whom, from what, producing what, judged how. The acceptance criteria are the important part. Each should be something a stranger could re-run and get the same verdict.

Each piece below has five parts: user and job, inputs, outputs, acceptance criteria, and evidence to publish. All data should be synthetic or openly licensed; state which.

Piece 1: Invoice field extractor with a review queue

User and job. An accounts assistant who keys invoice details into a ledger and wants the fields pre-filled, with doubtful ones flagged.

Inputs. Forty synthetic invoices as text or PDF, in at least three different layouts, including five with deliberate problems: a missing GSTIN, a total that doesn't match line items, a date in an ambiguous format.

Outputs. One JSON record per invoice with fixed fields (vendor name, invoice number, date, taxable value, tax, total) plus a needs_review flag and a reason.

Acceptance criteria.

1. 1Every output parses against a JSON schema you publish. Zero parse failures across all forty.
2. 2Field-level accuracy is reported per field against a hand-labelled answer key, not as a single average.
3. 3All five problem invoices are flagged needs_review with a reason that names the problem.
4. 4Totals are checked in code (sum of line items plus tax), not by the model.
5. 5Running the full set twice produces the same flags on the problem invoices.

Evidence to publish. The answer key, a results table per field, the list of failures with one-line diagnoses, and the command to reproduce.

Piece 2: Support-reply drafter that knows when to stop

User and job. A customer support agent for a fictional appliance company who wants a draft reply grounded in the company's policy documents.

Inputs. A synthetic policy set of ten short documents (warranty, returns, installation, service areas) and sixty synthetic customer messages, twenty of which ask things the policies do not cover.

Outputs. A draft reply with citations to document and section, or an explicit hand-off to a human when the policy does not answer the question.

Acceptance criteria.

1. 1Every factual statement in a draft cites a section that actually contains it; check a random sample of twenty by hand and report the result.
2. 2For the twenty uncovered questions, the system hands off rather than inventing an answer. Report how many it handled correctly.
3. 3No draft promises a refund, replacement or timeline not stated in the policy documents.
4. 4Messages written in Hinglish are included and reported as a separate slice.
5. 5Median response time and approximate cost per draft are recorded.

Evidence to publish. The policy documents, the message set, a table of results by slice (covered, uncovered, Hinglish), five failure examples with explanations, and the prompt or configuration used.

Piece 3: Read-only analytics agent with a tool log

User and job. A store operations manager who asks plain-language questions about a small synthetic sales database and needs answers they can trust.

Inputs. A SQLite database with three tables (stores, products, daily sales) of synthetic data, plus thirty questions with known answers computed by you in SQL.

Outputs. An answer, the SQL the agent ran, and the rows it used.

Acceptance criteria.

1. 1The database connection is read-only; demonstrate that an attempted DELETE fails.
2. 2Every answer is accompanied by the executed SQL and can be reproduced by running that SQL.
3. 3At least twenty-five of thirty answers match your precomputed values; publish the ones that don't with reasons.
4. 4Questions about data not in the database ("what were last year's sales?" when only this year exists) get a clear "not available" answer.
5. 5The agent stops after a fixed number of tool calls and says so, instead of looping.

Evidence to publish. The database generation script, the question set with expected answers, a full tool-call log for five questions, and the failure table.

Choosing which to build first

A note on acceptance criteria you don't meet

Publish them anyway. "Criterion 3: 17 of 20 uncovered questions handed off correctly; the three failures all involved installation charges, which the policy mentions only indirectly" is a stronger portfolio line than an unexplained 100%. Reviewers look for evidence that you know where your system breaks.

For why a basic retrieval demo falls short of these specs, read why a RAG demo isn't a portfolio. To present any of these as a narrative, use the AI project case study structure. The Generative & Agentic AI add-on builds evaluated projects of this kind across its three months.