Quantile regression for uncertainty-aware delivery estimates

A mean estimate answers an average question. Operations teams often need a different one: what duration should cover roughly 90% of comparable outcomes? Quantile regression can estimate that conditional percentile directly, but the estimate is neither a confidence interval for the mean nor a promise for every segment.

Define the estimand before choosing the model

Let q=0.9. For an observed duration y and estimate p, pinball loss penalizes underestimation nine times as strongly as equal-sized overestimation. Minimizing expected pinball loss targets the conditional 90th percentile.

Our experiment reuses the 320-row synthetic duration fixture from the supervised lab: 160 training rows, 80 validation rows and 80 test rows. The data have a curved conditional mean and noise that grows with load. A gradient-boosting quantile regressor is fixed at 100 stages, depth two and learning rate 0.05. No test value chooses these settings.

The constant baseline is the training target's 90th percentile: 36.4267 illustrative minutes. On validation, model pinball loss is 0.4598 versus 1.9662 for the constant. On test, it is 0.5151 versus 1.6714.

The HistGradientBoostingRegressor documentation is another primary reference for quantile loss. Our stored experiment uses GradientBoostingRegressor; the important contract is the declared quantile, loss and split roles.

Count coverage with its denominator

Seventy-one of 80 validation outcomes fall at or below their predicted 90th percentile, giving empirical coverage 88.75%. On test, 70 of 80 do, or 87.5%.

from decision_cases import quantile_case

r = quantile_case()
assert r['quantile'] == .9
for split in ['validation', 'test']:
    row = r['results'][split]
    assert row['empirical_coverage'] == row['covered_rows'] / row['n']
assert r['results']['test']['model_pinball'] < r['results']['test']['baseline_pinball']
print(r)

Run the code in the supervised-model lab. It recomputes predictions and losses from the fixed fixture rather than pasting a result table.

The observed 87.5% is not evidence that every type of case has 90% coverage. It is one marginal fraction from 80 synthetic test rows. Coverage can differ at low and high load, across locations, or after a process change. A prediction quantile also differs from a confidence interval around a mean parameter: one describes outcome variation; the other describes uncertainty in an estimated quantity.

Turn the estimate into an operational object

Before deployment, specify when features become available, how cancellations and censored durations are handled, which horizon the quantile serves, and whether late cases carry asymmetric cost. Assess pinball loss and empirical coverage by predeclared slices with adequate support. Compare against a simple training-quantile baseline.

Quantile crossing can occur when separate models estimate several percentiles. If a 90th-percentile prediction falls below the median, the outputs violate their intended ordering. Check this explicitly or use a method that enforces order. Distribution shift can also break historical coverage, so monitoring must include the observed denominator and outcome maturity delay.

Exercise: add median and 75th-percentile models without touching the test partition. Report pinball loss for each, count crossing rows, and explain why choosing the best-looking quantile after seeing test performance would answer a different question.

NeuraPath's Data Science course teaches learners to connect a business question to an estimand, a loss function and a verifiable assessment. That workflow makes an uncertainty estimate usable and reviewable.

Continue learning

This article is part of the Supervised learning methods sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Poisson regression for nonnegative event counts.
• Continue with Monotonic constraints: encode a justified business relationship.