SMOTE inside cross-validation without leakage

SMOTE creates minority-class samples by interpolating between minority neighbours. If the neighbour search sees validation rows before a cross-validation split, information from the evaluation fold influences the synthetic training set. The validation score no longer describes an untouched fold.

Put every learned transformation inside the fold

Our teaching implementation uses the 3,000-row synthetic training period and three stratified folds. For each fold it follows this order:

1. 1identify the 2,000 fit rows and 1,000 validation rows;
2. 2fit a standard scaler on the fit rows;
3. 3transform both partitions with that scaler;
4. 4generate synthetic minority points using fit-fold minority neighbours only;
5. 5fit logistic regression on the resampled fit data; and
6. 6score the original, unresampled validation rows.

The synthetic counts are 1,726, 1,726 and 1,728 across the folds. Validation average precision is 0.3850, 0.3475 and 0.3093, with mean 0.3473. These are internal training-period estimates, not a substitute for the later validation and test periods.

The imbalanced-learn pipeline example explains that samplers act during fitting and are disabled during prediction. Its validation-curve example places SMOTE in a pipeline evaluated across folds.

Verify the boundary with row identities

The local implementation returns the two original minority-parent positions for every synthetic row. It maps them back to transaction IDs and asserts that every parent belongs to the fit fold and none belongs to the validation fold.

from imbalance_cases import smote_cv_case

r = smote_cv_case()
assert len(r['folds']) == 3
for fold in r['folds']:
    assert fold['parent_subset_of_fit']
    assert fold['parent_overlap_with_validation'] == 0
    assert fold['validation_rows'] == 1000
print(r)

Run it in the imbalanced-model lab. The code implements basic interpolation for three numeric features so the boundary is inspectable. It does not claim categorical support or full compatibility with the imbalanced-learn package.

Avoid three common interpretation errors

First, synthetic points are not new independent observations. Their information comes from existing minority examples. Training counts may become balanced, but the evidence base has not grown to 5,590 independent transactions.

Second, interpolation can create implausible combinations when features are discrete, constrained or multimodal. Scaling affects neighbourhoods, and the chosen metric decides what “near” means. Validate ranges and domain rules after synthesis.

Third, cross-validation must reflect deployment. Stratification preserves label proportions, but it does not prevent the same customer appearing in fit and validation folds. Use grouped or temporal splits when that is the real generalization task, and keep resampling inside each resulting fit partition.

After model and sampler choices are fixed, refit on the permitted development data and evaluate once on original untouched rows. Never balance the test set unless the estimand explicitly concerns an artificial class mixture; doing so changes the prevalence and probability interpretation.

Exercise: intentionally run scaling and SMOTE before the three-way split. Add parent-ID tracing and count how many validation rows influence synthetic training points. Compare the contaminated score with the fold-local result, while explaining why the size of the difference does not determine whether leakage occurred.

NeuraPath's Data Science course teaches leakage prevention as an executable data-boundary check. The reliable SMOTE workflow can name exactly which original rows each synthetic training point was allowed to use.

Continue learning

This article is part of the Imbalance, calibration and decision thresholds sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Class imbalance: distinguish prevalence from model difficulty.
• Continue with Class weights versus resampling in a controlled comparison.