ML system design interview, a 45-minute round with a sample answer
In this article (10 sections)
Related programme: the Data Science course (6 months; ₹1,30,000 (inclusive of GST) · EMI from ₹5,000/month).
An ML system design interview gives you an open problem — "predict delivery time", "rank the home feed", "flag fraudulent sellers" — and about 45 minutes to turn it into a working design: what exactly is predicted, from which labels, with what features, measured how, served where, and watched for what. The round is scored less on the model you pick than on the order you think in. This article gives a minute-by-minute plan and then runs it once on a hypothetical food-delivery ETA prompt, with the questions an interviewer typically asks at each stage.
The 45-minute plan
| Minutes | Stage | What you produce |
|---|---|---|
| 0–5 | Clarify | The user, the decision the prediction drives, scale, latency budget |
| 5–12 | Frame | Prediction target, label source, offline and online metrics |
| 12–20 | Data and features | Feature groups, where each comes from, what is known at prediction time |
| 20–27 | Model | A baseline first, then one stronger model and why |
| 27–35 | Serving | Batch or online, the request path, latency and fallback |
| 35–42 | Evaluation and monitoring | Offline test design, A/B test, drift and alerting |
| 42–45 | Trade-offs | What you would do with more time, what could go wrong |
Say the plan aloud at the start. It lets the interviewer redirect you early instead of late.
The prompt
"Design the system that shows an estimated delivery time when a customer opens a restaurant page in a food-delivery app."
Minutes 0–5: clarify
Questions worth asking, with the hypothetical answers used below:
- What decision does the estimate drive? The customer's choice to order; also shown to the restaurant and rider.
- Scale? Assume 20 million restaurant-page views per day, peak about three times the average.
- Latency budget? The estimate must appear with the page, so under 100 ms for the model call.
- Is over-promising worse than under-promising? Assume a late order costs more than a slightly pessimistic estimate.
The last question matters most. It decides the loss function later.
Minutes 5–12: frame
Target: minutes from order placement to delivery at the door.
Label: the actual delivery timestamp minus the order timestamp, from completed orders. Point out that cancelled orders have no label and are likely the slowest ones — dropping them biases estimates downward.
Offline metric: mean absolute error in minutes, plus the share of orders delivered later than the estimate plus five minutes. Because lateness is costlier, consider predicting a high quantile (say the 70th percentile) rather than the mean, trained with quantile loss.
Online metric: conversion from page view to order, and complaint or cancellation rate for late orders, measured in an A/B test.
Interviewer probe: "Why not just use the average of past deliveries from that restaurant?" Answer: that is the baseline, and you should build it first. It ignores current kitchen load, rider availability, weather and distance to this customer.
Minutes 12–20: data and features
Group features by how fresh they must be:
- Static or daily: restaurant's historical preparation time by hour and weekday, cuisine, typical order size.
- Near real-time (minutes): open orders at the restaurant, riders available nearby, recent average pickup wait.
- Request-time: distance and route from restaurant to customer, time of day, weather.
State the leakage rule: every feature must be computable from data available at the moment the page is opened. "Rider assigned" or "actual pickup time" leak the future.
A quick sizing check shows whether the near real-time features need a streaming store:
views_per_day = 20_000_000
peak_factor = 3
avg_qps = views_per_day / 86_400
peak_qps = avg_qps * peak_factor
print(f"average {avg_qps:,.0f} requests/s, peak {peak_qps:,.0f} requests/s")
restaurants = 200_000 # hypothetical active restaurants
bytes_per_feature_row = 200
print(f"online feature store size about {restaurants * bytes_per_feature_row / 1e6:.0f} MB")About 231 requests per second on average and about 694 at peak, against a feature table of about 40 MB. That fits comfortably in an in-memory key-value store; the harder problem is keeping it fresh, not storing it.
Minutes 20–27: model
- 1Baseline: restaurant median delivery time for the hour, plus distance divided by typical rider speed.
- 2Main model: gradient-boosted trees on the feature groups above, trained with quantile loss. Trees handle mixed feature types and missing values well and are fast to serve.
- 3Mention and park: a sequence model over the order's stages (prepare, pick up, travel) could predict each stage separately. It adds complexity; only worth it if error analysis shows one stage dominates.
Probe: "How often do you retrain?" Answer: start weekly and compare error on the most recent week against older weeks; if error rises quickly after training, retrain more often.
Minutes 27–35: serving
Request path: page request → feature lookup (restaurant real-time features, rider supply for the zone) → model call → estimate rounded to a displayed range. Pre-compute the restaurant-level part in batch every few minutes, and add the customer distance at request time, which keeps the online computation small.
Fallback: if the feature store or model times out, show the baseline estimate. A missing estimate is worse than a coarse one.
Minutes 35–42: evaluation and monitoring
- Offline: train on earlier weeks, test on the most recent weeks, never on a random split — random splits leak future conditions into training.
- Slice error by city, hour, restaurant size and rain versus dry; an average can hide a bad segment.
- Online: A/B test on conversion and on-time rate together, since a model that shows shorter times can raise conversion while raising late deliveries.
- Monitoring: daily error on completed orders, feature freshness (age of the real-time rows), and input drift such as a sudden change in the distance distribution after the app expands its delivery radius.
Minutes 42–45: trade-offs
Offer two or three honest ones: quantile choice trades conversion against lateness and needs a business decision; stage-level models may help but cost complexity; rainy-day error is likely the worst and deserves its own evaluation.
What interviewers mark down
Jumping to "I'd use a neural network" in minute two, never defining the label, random train-test splits on time-ordered data, and no fallback. A clear baseline and a stated loss function earn more credit than an ambitious architecture.
For the model-level questions that often precede this round, see ML fundamentals interview questions; for LLM-backed products, the AI application system design bank covers retrieval, cost and guardrails. End-to-end model deployment and monitoring are part of the capstone work in the Data Science programme.
Frequently asked questions
How is ML system design different from software system design?
Software system design focuses on services, storage and scaling. ML system design adds the problem framing, labels, features, training data, offline and online evaluation, and monitoring for drift. You still need a serving architecture, but the interviewer spends more time on data and evaluation.
Should I draw an architecture diagram in an ML system design interview?
Yes, but late. Spend the first third on the problem, the label and the metric. A diagram drawn before the objective is clear usually shows components that do not match the problem.
Pankit Kumar has 10 years in Data Science & AI, building and shipping production systems in regulated pharma and clinical environments. He is a freelance trainer at Boston Institute of Analytics, AnalytixLabs and Scaler, and has taught this material to thousands of working professionals.
This article is part of our Data Science programme — 6 months. From data foundations to machine learning, deep learning and deployment.
Explore Data Science