Logistic regression: probabilities, logits and decision thresholds

Binary logistic regression forms a linear score in its input representation and maps that score through the logistic function. The score is a logit; the resulting probability and the threshold used to classify it are related but distinct quantities.

We use the original synthetic inactivity pipeline already fitted and evaluated in the project lab. This article inspects that frozen model's mathematics rather than selecting a new model from its test results.

Move between score and probability

For score z, the logistic function is 1/(1+exp(-z)). Its inverse for probabilities strictly between zero and one is log(p/(1-p)).

A zero logit corresponds to probability0.5. A positive logit corresponds to probability above0.5, and a negative logit to probability below0.5. The score itself is not a probability and is not restricted to the interval from zero to one.

The LogisticRegression reference documents decision scores and probability predictions. SciPy's expit function provides the logistic transformation used below.

Verify the fitted pipeline's relationship

import numpy as np
from scipy.special import expit
from evaluation_core import load,run,FEATURES

data = load()
test = data[data['split']=='test']
report,_,model = run()
assert model.named_steps['model'].classes_.tolist()==[0,1]
z = model.decision_function(test[FEATURES])
p = model.predict_proba(test[FEATURES])[:,1]
assert np.allclose(expit(z),p)
assert np.array_equal(z>=0,p>=.5)
threshold=.3
logit_threshold=np.log(threshold/(1-threshold))
assert np.array_equal(z>=logit_threshold,p>=threshold)
assert np.isclose(logit_threshold,-.8472978603872036)
assert int((p>=.5).sum())==23
print({'test_rows':len(p),'selected_at_0_5':int((p>=.5).sum()),
       'logit_threshold_for_0_3':float(logit_threshold)})

Class order is checked explicitly before taking probability column one. The model's positive class is label1, meaning the defined seven-day inactivity outcome. A different estimator or label encoding should not be assumed to use the same column meaning.

Interpret coefficients in the actual representation

This pipeline imputes and scales numeric inputs and one-hot encodes plan. A coefficient therefore belongs to a transformed feature, not automatically to one raw unit of the original column.

For a linear change of delta in a fitted transformed feature, holding the other model inputs fixed, the logit changes by coefficient times delta. Exponentiating that quantity gives the model's odds multiplier. It does not give a constant probability-point change because the logistic mapping is nonlinear.

Holding inputs fixed is also a model calculation, not an intervention proof. Correlated features or impossible combinations can make a coefficient interpretation poorly aligned with realistic observations.

Separate numerical probability from empirical calibration

The logistic function produces values between zero and one. That alone does not establish that cases scored0.7 are positive70% of the time in the deployment population. Calibration needs representative labelled evaluation and can change with population or process shifts.

The original test at threshold0.5 selects23 cases and has recall40%. Lowering a threshold changes which cases are selected; it does not refit the score function or prove that the new operating rule is suitable.

Exercise: compute the logit thresholds for probabilities0.2 and0.8. Then explain why a one-unit increase in logit produces different probability changes when starting near0.1 versus0.5. State which quantities are model mathematics and which require empirical validation.

NeuraPath's Data Science course connects classification equations with practical model behavior. A clear explanation distinguishes the transformed feature, logit, probability and operating threshold.

Continue learning

This article is part of the Supervised learning methods sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Linear regression with residual checks and a naive baseline.
• Continue with Ridge versus lasso when predictors are correlated.