Save and reload a neural model with equivalent predictions

A model that worked in a training process is not deployable until its artifact can reconstruct the same inference function. Save all learned parameters and preprocessing, reload them in a fresh path, and compare predictions on a fixed batch before release.

Exact equivalence in the local model

The deep-learning lab trains its 45-parameter NumPy convolutional model, writes arrays to a temporary .npz artifact, reloads them and scores 60 images again.

from deep_learning_cases import reload_case

result = reload_case()
assert result["rows_compared"] == 60
assert result["maximum_prediction_difference"] == 0.0
print(result["artifact_bytes"], result["maximum_prediction_difference"])

The artifact is 1,326 bytes in this run and maximum prediction difference is exactly zero. This is possible because the same NumPy operations and float arrays are used before and after loading.

Save the whole inference contract

Production equivalence also requires input resizing, channel order, normalization, class mapping, threshold, model code or graph, package versions and output schema. Store a manifest with hashes and a model identifier. A weight file without preprocessing can produce valid-shaped but wrong predictions.

Put dropout and normalization layers in evaluation mode. Compare logits or probabilities before thresholding so small numeric differences are visible. Set tolerances according to precision and backend; exact equality may be unreasonable after format conversion or hardware changes.

Use a golden batch

Keep a small, versioned batch covering ordinary, boundary and error cases. Test model loaded in the same code path as the service. Check schema rejection, unknown categories, batch-size variation and single-item requests. A notebook-only reload is weaker evidence than an API-path reload.

Use safe, documented formats and load only trusted artifacts. Some general serialization mechanisms can execute code during deserialization. Verify provenance and integrity before loading.

The Data Science course includes artifact equivalence in the bridge from experiment to service.

Exercise

Add a manifest containing model hash, preprocessing version, class order and golden outputs. Reload in a new process and compare. Then change one normalization value and confirm the equivalence test fails.

Continue learning

This article is part of the Deep learning and computer vision sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in RNNs and LSTMs through a sequence prediction task.
• Continue with Measure inference latency with warmup and repeated trials.

References: NumPy save/load documentation and PyTorch saving and loading models.