Audit proxy variables in a predictive model

Removing a sensitive field does not ensure a model lacks related information. Location, school, device or purchase history can correlate with a group and act as a proxy. Correlation is a review signal, not automatic proof that a feature is prohibited or harmful.

Compare one synthetic proxy intervention

Our fixture has a binary group, a proxy matching it on 92.4% of rows, a legitimate numeric signal and an authored target mechanism. We fit logistic regression on 700 rows and evaluate 300 later rows.

The signal-only model has test AUC 0.7735. Selection rates at threshold 0.5 are 29.25% for group zero and 26.14% for group one. Adding the proxy changes AUC to 0.7752 and selection rates to 28.57% and 26.80%.

The small changes in this seed do not certify the proxy as safe. They show one descriptive ablation under one mechanism.

from feature_cases import proxy_case

r = proxy_case()
assert r['proxy_group_agreement'] == .924
assert set(r['test']) == {'signal_only', 'signal_plus_proxy'}
for model in r['test'].values():
    assert 0 <= model['selection_rate_group0'] <= 1
    assert 0 <= model['selection_rate_group1'] <= 1
print(r)

Run it in the feature-engineering lab. The groups and target process are synthetic; no legal, fairness or causal conclusion applies to a real population.

Audit semantics, pathways and alternatives

Start with why the feature exists, how it is collected, who it represents and which decision it affects. Measure association with relevant groups, missingness and availability. Train ablations and compare performance, errors, calibration and selection with uncertainty.

A feature can be correlated yet necessary for accuracy or monitoring, while an apparently neutral feature can enable harmful inference in combination. Legal and domain review must interpret the use case. Fairness definitions can conflict and depend on policy goals.

Removing a proxy may reduce measured performance without removing underlying structural bias from labels or actions. Conversely, retaining sensitive attributes for audit can be essential even when they are excluded from prediction. Apply access controls and purpose limitation.

Exercise: add a second weaker proxy that becomes strong only in combination. Compare individual correlations, model ablations and group errors. Write a review record separating statistical findings, policy decision and unresolved evidence.

NeuraPath's Data Science course teaches proxy review as a multidisciplinary audit. Model metrics identify questions; they do not settle governance alone.

Continue learning

This article is part of the Feature engineering and data quality sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Build a reusable preprocessing artifact.
• Continue with Detect accidental identifiers in a training dataset.