ML engineer role in India: shipping models, not just training them

The model is the small part

A data scientist may spend weeks improving a model's validation score. The ML engineer's questions start after that: can we compute these features at prediction time with the same logic as in training? Will it respond within the product's latency budget? What happens when a feature is missing? How do we roll back if it misbehaves? Most production ML failures come from these questions, not from the algorithm.

Batch or real time: the first design decision

Choosing online serving when batch would do is one of the most expensive mistakes in ML engineering. A good ML engineer asks this before writing code.

The artefact: a model release pull request

Model changes should go through review like any other code. Here is the description of a realistic release PR for a hypothetical search-ranking model at an Indian marketplace.

PR: ranking-model v14 -> v15 (candidate)

What changed
  - Added feature: seller_cancellation_rate_30d (from feature pipeline PR #2231)
  - Retrained on 12 weeks of data ending 2026-09-20; same hyperparameters as v14

Offline evaluation (held-out week 2026-09-14 to 2026-09-20)
  - NDCG@10: v14 0.412 -> v15 0.419
  - Slices: no slice worse than -0.005 except new sellers (<30 days): -0.012
    New sellers have null cancellation rate; imputed as 0, which ranks them higher.
    Changed imputation to the category median; slice now -0.002.

Training/serving consistency
  - Feature computed by the same library function in training and online paths
  - Replayed 10,000 logged requests: max score difference vs offline 1e-6

Performance
  - p95 inference latency: 18 ms (v14: 17 ms); budget 30 ms
  - Model artifact 41 MB (v14: 39 MB)

Rollout plan
  - Shadow mode 48 h, then 10% traffic A/B for 7 days; primary metric: orders per search
  - Guardrails: search latency p95, zero-result rate, seller complaints
  - Rollback: flip registry alias "prod" back to v14; no redeploy required

Reviewer checklist
  [ ] Eval data does not overlap training dates
  [ ] Slice regressions explained or fixed
  [ ] Rollback tested in staging

The most important section is the slice analysis. An overall improvement that hides harm to new sellers is exactly the kind of problem that reaches production when nobody checks. The replay test, comparing online and offline scores on the same requests, catches training-serving skew, a very common source of silent degradation.

All figures above are illustrative.

What fills a normal week

Implementing a feature in the pipeline for a data scientist's new model, a performance fix after latency crept up, reviewing a colleague's serving code, an incident review for a batch job that produced stale scores, and time on platform work such as improving the training pipeline's reproducibility.

Latency budgets, uptime and model impact

Reliability and latency of model services, time from approved model to production, number of incidents caused by model releases, cost of training and serving, and, jointly with data scientists, the business metric the model serves.

Data scientists, backend teams and product owners

ML engineers usually report to an ML engineering manager or engineering lead in a product team. They work with data scientists, data engineers, backend engineers who call the model, product managers who own the metric, and platform or MLOps teams.

Where the confusion lies

The title is confused with data scientist (who leads modelling and analysis) and MLOps engineer (who builds the shared platform for many teams' models; see the MLOps engineer guide). Since 2024 it is also blurred with AI engineer, which in many postings now means building on large language models rather than training your own; see the AI engineer guide.

Serving, feature stores and MLOps in postings

• "Deploy and scale ML models, REST/gRPC, Docker, Kubernetes" — true ML engineering.
• "Feature store, real-time inference" — online serving experience expected.
• "Research novel architectures, publish" — applied or research scientist role.
• "Build models in Python and present insights" — data scientist role.
• "LLM, RAG, prompt pipelines" — AI engineer or LLM application role.

Reproduce the model, then own its release

Deploy something small to production in the first month, even a minor fix, to learn the release path. In month two, add a missing test or monitor to one model service. In month three, own a model release end to end with a PR like the one above.

Senior MLE, ML platform or applied science

Senior ML engineer, ML platform engineer, MLOps lead, or a staff-level role across several ML systems. The Data Science programme covers model evaluation, slice analysis and the data engineering foundations these release reviews depend on.