Prompt engineering interview questions that test judgement

Comparing prompts

Pair 1: vague versus specified

• A: "Summarise this customer complaint."
• B: "Summarise this customer complaint for a support team lead in two sentences: first the problem, then what the customer is asking for. If the request is unclear, say 'Request unclear'."

Better: B. It names the reader, the length, the structure and what to do when information is missing. A leaves every one of those to the model, so outputs vary run to run.

Judgement being tested: seeing that most prompt failures are underspecified tasks.

Pair 2: politeness versus context

• A: "Please, it's very important that you answer accurately. Which plan should this customer move to?"
• B: "Here are our four plans with prices and limits: [table]. Here is the customer's usage for three months: [table]. Recommend the cheapest plan that covers their highest month, and show the comparison."

Better: B. Emphasis does not supply information; context and a decision rule do. Without the plan table, the model can only guess.

Pair 3: free text versus structured output

• A: "Extract the order ID, amount and issue from this email."
• B: "Return only JSON with keys order_id (string), amount_inr (number or null) and issue (one of: delivery, refund, quality, other)."

Better: B, because code consumes the result. But a strong candidate adds that you still validate the output, because the model may return extra text or an invalid category:

import json

ALLOWED = {"delivery", "refund", "quality", "other"}

def parse(output):
    try:
        data = json.loads(output)
    except json.JSONDecodeError:
        return None, "not valid JSON"
    if set(data) != {"order_id", "amount_inr", "issue"}:
        return None, f"unexpected keys {sorted(data)}"
    if data["issue"] not in ALLOWED:
        return None, f"issue '{data['issue']}' not allowed"
    if data["amount_inr"] is not None and not isinstance(data["amount_inr"], (int, float)):
        return None, "amount_inr must be a number or null"
    return data, "ok"

samples = [
    '{"order_id": "B77", "amount_inr": 899, "issue": "refund"}',
    'Here is the JSON: {"order_id": "B77"}',
    '{"order_id": "B77", "amount_inr": "899", "issue": "refund"}',
    '{"order_id": "B77", "amount_inr": null, "issue": "late delivery"}',
]
for s in samples:
    print(parse(s)[1])

Only the first sample passes. The failures — surrounding text, a string amount, an out-of-list category — are exactly what appears in real outputs. Where the API offers a structured-output or JSON-schema mode, use it, and keep the validation anyway.

Pair 4: one example versus several chosen examples

• A: One example of a complaint classified as "delivery".
• B: Four examples: one per category, including a borderline case with the reasoning for its label.

Better: B. A single example biases outputs toward its label and format. Examples should cover the categories and the hard boundary cases, and should not all look alike.

Pair 5: asking for reasoning

• A: "Is this transaction suspicious? Answer yes or no."
• B: "List the factors from the rules below that apply to this transaction, then answer yes or no."

Better: usually B for multi-step judgements, because working through the rules first tends to improve consistency and gives a reviewer something to check. The trade-off is more output tokens and latency; for simple lookups A is fine. Say you would compare both on a test set rather than assume.

Pair 6: instructions in the user text

• A: System prompt: "You are a helpful assistant." The user's uploaded document is pasted directly after it.
• B: System prompt states the task and that the document is data to analyse, not instructions; the document is wrapped in clear delimiters.

Better: B, because documents can contain text that reads like instructions. Delimiters and a clear role reduce, but do not eliminate, prompt injection — anything high-impact still needs checks in code.

Questions without a pair

"How do you know a prompt change is an improvement?"

Keep a fixed set of inputs with expected outputs or a rubric — including edge cases and inputs that should be refused. Run old and new prompts on the full set, compare pass rates and look at the cases that changed. Changing a prompt because one example looked better is how regressions ship.

"The prompt works on one model and fails after switching to another. Why?"

Models differ in how they follow formats, how they weigh system versus user instructions and how they treat examples. Treat a model switch like a code change: rerun the evaluation set and adjust.

"When should you stop prompting and do something else?"

When the model lacks the information (add retrieval), when the task needs exact computation (use code or SQL and let the model explain the result), or when you need a consistent style across thousands of outputs that prompting cannot hold (consider fine-tuning). Recognising that limit is itself a judgement interviewers look for.

"What would you put in a production system prompt?"

The role and task, the audience, the rules and their priority, the output format, what to do when information is missing or a request is out of scope, and nothing secret — assume a determined user can extract it.

How to prepare

Take one task you care about, write two competing prompts, create twenty test inputs including five awkward ones, and score both. Being able to describe that exercise in an interview is more persuasive than a list of techniques.

For how prompts feed tools, see the AI agent tool-calling bank; for retrieval context, see the RAG bank. Prompt design and evaluation sets are practised throughout the Generative and Agentic AI programme.