MLOps engineer role: registries, drift checks and safe rollbacks

One platform, many models

Picture a hypothetical Indian insurer with fourteen models in production: claim fraud scores, renewal propensity, document classification and more, built by four teams. Without a platform, each team has its own scripts, its own way of storing model files and no consistent monitoring. When an auditor asks "which model version scored this claim on 3 March, trained on what data?", nobody can answer quickly. The MLOps engineer's job is to make that question trivial.

The lifecycle you enforce

Tools vary: MLflow, cloud-native registries in AWS, Azure or Google Cloud, Kubeflow, or in-house services. The principle does not: every production prediction should trace back to a registered version, and every registered version to code and data.

The artefact: a drift check you can run

A common monitoring task is checking whether the data a model sees in production still resembles its training data. The population stability index (PSI) is widely used for this in Indian banking and insurance. This self-contained example uses only the standard library and synthetic values.

import math
import random

random.seed(7)
# Synthetic feature: applicant monthly income in thousands of rupees
training = [random.gauss(60, 15) for _ in range(5000)]
production = [random.gauss(68, 18) for _ in range(2000)]

def quantile_edges(values, bins=10):
    s = sorted(values)
    return [s[int(len(s) * i / bins)] for i in range(1, bins)]

def bucket_shares(values, edges):
    counts = [0] * (len(edges) + 1)
    for v in values:
        i = sum(v > e for e in edges)
        counts[i] += 1
    total = len(values)
    # small floor avoids log(0) for empty buckets
    return [max(c / total, 1e-4) for c in counts]

def psi(expected, actual, bins=10):
    edges = quantile_edges(expected, bins)
    e = bucket_shares(expected, edges)
    a = bucket_shares(actual, edges)
    return sum((ai - ei) * math.log(ai / ei) for ei, ai in zip(e, a))

value = psi(training, production)
print(f"PSI = {value:.3f}")
if value < 0.1:
    print("stable")
elif value < 0.25:
    print("moderate shift: investigate")
else:
    print("significant shift: review model before relying on it")

Buckets are defined on training data deciles, so each holds about 10% of training values; PSI then measures how far production shares moved. The 0.1 and 0.25 thresholds are a common rule of thumb, not a standard, and each team should set its own after looking at history. On this synthetic data the shift is large enough to flag. A high PSI is a prompt to investigate, not proof that the model is wrong: a festive-season income mix or a new sourcing channel can shift inputs while the model still performs acceptably. Where labels arrive later, as with loan defaults, drift checks are an early warning while outcome monitoring catches up.

A week of MLOps work

Upgrading the training pipeline's base image after a security advisory, onboarding a new team's model to the registry, investigating a drift alert with the owning data scientist, cutting GPU or compute costs by scheduling training jobs, and writing the procedure for an emergency rollback. There is usually a support channel where other teams ask for help, and a surprising share of the job is answering it well.

Time to deploy, time to roll back and alert quality

Time for a team to take a model from approved to production, share of production models with monitoring and complete lineage, platform uptime, incidents caused by the platform, and compute spend. Adoption matters: a platform teams avoid is a failure however elegant it is.

Platform engineering, data science and security

MLOps engineers typically report to an ML platform lead, head of data platform or engineering manager. They work with data scientists and ML engineers (their customers), infrastructure and security teams, and, in regulated industries, model risk management and audit.

Confused with ML engineer and DevOps engineer

An ML engineer ships specific models inside a product. A DevOps or platform engineer runs general infrastructure. The MLOps engineer sits between, with ML-specific concerns such as data versioning, training reproducibility and model monitoring. In smaller companies the ML engineer does MLOps work too.

Kubeflow, MLflow and Kubernetes: which job is it?

• "MLflow, Kubeflow, SageMaker, Vertex AI pipelines" — platform MLOps.
• "Kubernetes, Terraform, CI/CD for ML" — infrastructure-heavy variant.
• "Model monitoring, drift, model risk governance" — regulated-industry flavour.
• "Train and tune models" — data scientist or ML engineer.
• "LLMOps, prompt and evaluation pipelines" — the newer LLM variant of the same discipline.

Inventory every model in production

Map every production model, its owner, where it runs and whether it is monitored; the gaps become your roadmap. Next, make one team's release path fully automated. By month three, bring a second team onto the same path and document it so a third can onboard without you.

ML platform lead or site reliability for ML

Senior MLOps or ML platform engineer, platform architect, or engineering manager for ML infrastructure. The FDE for Professionals programme, currently on a proposed schedule, covers production deployment, Kubernetes and infrastructure-as-code, and operational evaluation for experienced engineers.