Prompt engineer as a job title: what the work actually involves

What the title tends to mean in postings

Read ten postings with "prompt" in the title and they fall into a few groups:

• Prompt work inside an engineering role. Designing system prompts, tool descriptions and output formats, plus the code around them. This is really AI engineering.
• Domain specialist prompt writer. Someone with legal, medical, finance or teaching expertise who writes and evaluates prompts for that domain, often with an engineer.
• Content or marketing prompt operator. Using AI tools to produce copy or images at volume. Useful work, but not engineering, and often short-contract.
• Evaluation and rating roles described with the word "prompt". Closer to AI trainer work, explained in the AI trainer and annotation guide.

Knowing which group a posting belongs to tells you what the interview will test.

Why writing a good prompt is the smaller part

A prompt that works on five examples you tried may fail on the sixth. The skill that makes prompt work reliable is measuring it: a set of test inputs with expected behaviour, run every time the prompt, model or retrieved context changes. Without that, a "better" prompt is an opinion.

Prompts also interact with everything around them: the retrieved documents, tool definitions, output parsers and the model version. A change that fixes tone can break JSON output. That is why prompt work that lasts is done as engineering, with versions and regression checks.

The artefact: a prompt change log with regression results

This is how one change to a hypothetical customer-support assistant for an electronics retailer might be recorded. The evaluation set has 120 synthetic cases grouped by what they test.

PROMPT CHANGE LOG — support-assistant/system

v14 -> v15   Author: [name]   Reviewed by: [name]   Date: 22 Sep 2026
Problem:     Assistant promised replacements for damaged items outside the
             7-day policy window (4 incidents in last week's sample review).
Change:      Added explicit rule: "Replacement eligibility depends on the
             delivery date in the order record. If it is not available, say
             you will check and hand over to an agent. Never promise."
             Moved policy text from the middle of the prompt to a tagged block.
Model:       unchanged
Retrieval:   unchanged

Evaluation (120 cases, same set as v14)
  Group                         Cases   v14 pass   v15 pass
  Policy eligibility              30       21         29
  Order-status lookups            25       24         24
  Tone and apology                20       19         18
  Refusals (out-of-scope asks)    15       15         15
  Hindi and Hinglish queries      20       16         17
  Structured handover JSON        10       10         10
  Total                          120      105        113

Regressions:  1 tone case now sounds abrupt ("I will check.") — accepted,
              ticket opened to add a softer handover phrase in v16.
Decision:     Ship v15 to 10% of traffic for 3 days, then 100% if the
              weekly sample review shows no new promise errors.

The log answers the questions anyone will ask later: what problem prompted the change, what exactly changed, whether anything else changed at the same time, what improved, what got worse, and how it was rolled out. The one regression is recorded and accepted deliberately, not discovered by a customer.

A realistic week

• Reviewing a sample of live conversations with the support lead and turning failures into new test cases.
• Drafting two candidate fixes for a recurring issue and running both against the evaluation set.
• Working with an engineer to change a tool's description because the model keeps calling it with the wrong parameter.
• Checking whether a new model version changes pass rates before the team switches.
• Writing short guidance for colleagues on how to add test cases.

Pass rates, incidents and cost per conversation

Pass rates on agreed evaluation groups, incidents in production traced to prompt behaviour, speed of fixing reported failures, and cost per conversation when prompt length or model choice changes it.

AI engineering leads, domain experts and reviewers

Typically an AI engineering lead or an AI product manager. Colleagues include engineers who own the application, domain experts, quality or support teams who review conversations, and the AI product manager who decides what ships.

Confused with AI engineer

The AI engineer owns the whole application: retrieval, tools, orchestration, deployment and monitoring. Prompt design is one of its tasks. If a posting titled "prompt engineer" lists Python, APIs, vector databases and evaluation frameworks, it is an AI engineering role and you should prepare for one.

Prompt, evaluation or content: which job is advertised

• "Design, test and optimise prompts for LLM applications" — core prompt work; ask how testing is done.
• "Build evaluation datasets, measure quality" — the reliable part of the job.
• "Python, LangChain/LlamaIndex, APIs" — an engineering role.
• "Create content using generative AI tools" — content production, not engineering.
• "Subject-matter expertise in [field]" — domain writing and evaluation.

Version the prompts, then improve them

Month one: build or extend the evaluation set from real failures. Month two: put prompts under version control with a change log like the one above. Month three: ship a measured improvement. Most people grow from here into AI engineer, LLM application developer, evaluation engineer or AI product roles. The Generative & Agentic AI add-on teaches prompt design together with evaluation, retrieval and tools, which is how the work is done in practice.