Test batch and single-record prediction consistency

The same record should usually receive the same prediction whether it appears alone or inside a batch. A mismatch can expose preprocessing that computes request-batch statistics, column reordering, mutable state or threshold differences between paths.

Compare the exact service pipeline

The local deployment lab scores 32 held-out rows as one batch and then one row at a time through the same scikit-learn pipeline.

from deployment_cases import batch_consistency_case

result = batch_consistency_case()
assert result["rows"] == 32
assert result["maximum_absolute_difference"] < 1e-12
assert result["same_decisions"] is True
print(result["maximum_absolute_difference"])

The maximum probability difference is about 2.22e-16, floating-point rounding, and every threshold decision matches. Testing with exact zero would fail despite equivalent behavior, so the assertion uses a documented tolerance.

Avoid request-batch preprocessing

Scaling, imputation and category vocabularies should use stored training parameters. Calculating a mean from the current request batch means one customer’s score can change when another customer joins the batch. That is both inconsistent and hard to audit.

Preserve feature order by selecting named columns inside a packaged pipeline. Validate duplicate IDs and return outputs in an order or mapping the caller can reconcile. Test batch size one, maximum supported size and mixed valid/invalid behavior.

Include production conversions

Compare notebook predictions with the serialized model and actual service code. Format conversions, float precision, quantization or another runtime may require a wider tolerance. Set it from validated numeric behavior and confirm decisions near the threshold.

A matching batch test does not establish concurrency safety or capacity. Those need load and race-condition tests.

The Data Science course uses prediction equivalence as a release gate between experimentation and serving.

Exercise

Create rows just above and below the decision threshold. Compare dataframe, JSON single and JSON batch routes after serialization. Fail on decision disagreement and investigate numeric drift.

Continue learning

This article is part of the Model deployment and MLOps sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Serve a model with an explicit prediction schema.
• Continue with Package preprocessing with the trained model.

Reference: scikit-learn pipelines and composite estimators.