LLM application developer: contracts, tools and failure handling

Treat the model like an unreliable external service

Experienced developers already know how to work with a flaky payment gateway: timeouts, retries, validation of every response, and a fallback when it is down. An LLM needs the same treatment plus one more rule: a response that parses correctly can still be wrong. The developer's job is to put checks between the model and anything that matters.

Consider a hypothetical feature for an Indian accounts-payable team: extract invoice fields from a supplier's PDF so a clerk can approve it. The model returns JSON. Before anything is saved, the application checks it.

The artefact: an output contract and its validator

This self-contained example uses only the standard library. The model_output string stands in for what the model returned; no API is called.

import json
import re
from datetime import date

GSTIN_PATTERN = re.compile(r"^[0-9]{2}[A-Z]{5}[0-9]{4}[A-Z][1-9A-Z]Z[0-9A-Z]$")

def validate_invoice(raw: str):
    errors = []
    try:
        data = json.loads(raw)
    except json.JSONDecodeError as exc:
        return None, [f"not valid JSON: {exc.msg}"]

    required = {"invoice_number": str, "invoice_date": str, "supplier_gstin": str,
                "taxable_value": (int, float), "tax_amount": (int, float), "total": (int, float)}
    for field, typ in required.items():
        if field not in data:
            errors.append(f"missing {field}")
        elif not isinstance(data[field], typ) or isinstance(data[field], bool):
            errors.append(f"{field} has wrong type")
    if errors:
        return None, errors

    if not GSTIN_PATTERN.match(data["supplier_gstin"]):
        errors.append("supplier_gstin format invalid")
    try:
        d = date.fromisoformat(data["invoice_date"])
        if d > date.today():
            errors.append("invoice_date is in the future")
    except ValueError:
        errors.append("invoice_date not YYYY-MM-DD")
    if abs(data["taxable_value"] + data["tax_amount"] - data["total"]) > 1:
        errors.append("taxable_value + tax_amount does not equal total")
    return (data if not errors else None), errors

model_output = '''{"invoice_number": "INV-2291", "invoice_date": "2026-09-18",
 "supplier_gstin": "09ABCDE1234F1Z5", "taxable_value": 10000, "tax_amount": 1800, "total": 11700}'''

result, problems = validate_invoice(model_output)
print("accepted" if result else "sent to manual review")
for p in problems:
    print("-", p)

Running it prints "sent to manual review" with one problem: the amounts do not add up (₹10,000 + ₹1,800 is ₹11,800, not ₹11,700). The JSON is syntactically perfect, every field is present, and the GSTIN matches the published structural pattern, yet the record is wrong. Only a business-rule check catches it. The GSTIN regex checks structure, not whether the number is registered; a real system would verify against an authoritative source or the purchase register.

A failure table you build for every feature

The last row matters for anything that reads outside content. A supplier PDF could contain text written to manipulate the model; the application must never let model output alone approve a payment.

What the week involves

Adding a new document type to the extractor, investigating why accuracy fell after the provider updated a model version, writing tests for a tool that lets the assistant search the vendor master, reviewing logs to confirm no personal data is being sent where it should not be, and pairing with a product manager on what the clerk sees when validation fails.

Parse failures, fallbacks and manual-review rates

Share of outputs accepted without human correction, share of wrong outputs caught by validation rather than by users, latency, cost per processed document, and incident count. A developer who raises automatic acceptance while wrong-but-accepted outputs also rise has made things worse.

Backend leads, operations reviewers and product

Typically an engineering manager or AI product lead. You work with product managers, the business users who review outputs, backend teams who own the systems you call, and security for data handling.

How it differs from AI engineer and prompt engineer

Many companies use AI engineer for the same work; others reserve it for broader system design or model work. The difference from a prompt engineer is that the developer owns the surrounding code, validation and production operation, not just the instructions sent to the model.

Function calling, structured output and LangChain in postings

• "Integrate LLM APIs into applications, structured outputs, function calling" — this role.
• "LangChain, LlamaIndex, orchestration frameworks" — framework experience; ask whether they are required or just current tools.
• "Full-stack, React, Node" plus LLM integration — product engineering with AI features.
• "Evaluate and fine-tune models" — more ML-focused.

Contracts first, features second

Month one: add validation and logging to the feature you inherit. Month two: build a test set from real failures. Month three: ship an improvement with measured before-and-after results. Next roles include senior AI engineer, AI platform engineer or forward-deployed engineer. The Generative & Agentic AI add-on covers structured outputs, tool contracts and evaluation in its three months.