Multi-output regression with shared and separate models

Multi-output regression predicts several numeric targets for each row. A shared model can exploit common structure and produce predictions together. Separate models can give each target its own representation and tuning. The choice becomes concrete when capacity and target scales are limited.

Construct two targets that need different splits

Our deterministic fixture contains a 4-by-4 grid of two features. Target one jumps from 0 to 10 when feature one becomes positive. Target two jumps from 0 to 100 when feature two becomes positive. Four new interior probes cover every sign combination.

A native multi-output decision tree limited to depth one can make only one split. In original units, the second target's hundred-unit jump dominates combined squared-error reduction, so the root uses feature two. Probe RMSE is 5 for target one and 0 for target two.

If target two is divided by 100 during fitting, the same shared tree selects feature one. After converting predictions back to original units, probe RMSE becomes 0 and 50. Changing units changed the shared optimization trade-off even though the underlying rules did not change.

The scikit-learn multi-output guide defines the task, while MultiOutputRegressor fits one regressor per target.

Give each target its own capacity

Wrapping a depth-one tree in MultiOutputRegressor creates two stumps. Their root features are zero and one respectively, and both produce zero RMSE on the four authored probes.

from decision_cases import multioutput_case

r = multioutput_case()['results']
assert r['native_original_units']['root_feature'] == 1
assert r['native_second_target_divided_by_100']['root_feature'] == 0
assert r['separate_stumps']['root_features'] == [0, 1]
assert r['separate_stumps']['rmse_original_units'] == [0, 0]
print(r)

Run this in the supervised-model lab. The comparison is deliberately capacity-matched at one split per shared tree versus one split for each separate tree. The separate approach therefore uses more total splitting capacity; its perfect result is an explanation of the construction, not a general benchmark.

Choose the architecture from the dependency you can defend

Shared modelling is attractive when targets arise from related mechanisms, training rows align, missingness is coordinated and joint predictions matter. It may regularize scarce targets through shared representation. It can also let a high-variance or large-unit target dominate an aggregate loss unless target weighting is explicit.

Separate models make per-target features, losses, refresh schedules and failure handling easier. They ignore potentially useful target dependence and can yield internally inconsistent combinations. Scaling targets for training may balance optimization, but business loss still must be evaluated in interpretable units.

Assessment should report one metric per target, relevant joint constraints and a transparent aggregation rule. Never hide a failed critical target inside a strong average. Split data by the deployment unit before fitting either architecture, and keep target-specific leakage checks: a field available for one outcome may reveal another outcome's future.

Exercise: allow a shared tree depth of two, then compare its total leaf capacity with the two separate stumps. Add correlated noise to the targets and repeat across seeds. Report per-target results rather than selecting a winner from a single averaged RMSE.

NeuraPath's Data Science course teaches multi-target work as an architecture and evaluation decision. The model interface is the easy part; target units, capacity and joint failure modes determine whether the result is useful.

Continue learning

This article is part of the Supervised learning methods sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Monotonic constraints: encode a justified business relationship.
• Continue with Cost-sensitive classification with an explicit loss table.