Build a value tree that separates adoption from model quality

A high offline model score creates no value when eligible users avoid the workflow. High adoption can also amplify poor output. A value tree keeps the causal layers separate so the team can locate the broken link.

Build the decision artifact

Start with the business outcome and current baseline. Work backward through user behavior, workflow outcomes, AI quality, operational reliability and enabling capabilities. Define a numerator and denominator for each node. Record assumptions between nodes rather than multiplying optimistic percentages into a single forecast.

The commercial leadership lab makes the artifact inspectable with authored inputs:

from leadership_cases import value_tree_case

result = value_tree_case()
assert result["adoption_rate"] == 0.75
assert result["acceptance_rate"] == 0.9
assert result["model_quality_is_adoption"] is False
assert result["fictional"] is True

In the authored tree, 600 of 800 offered users engage, an adoption rate of 75%. Of 600 used outputs, 540 are accepted with or without edit, an acceptance rate of 90%. These distinct figures prevent the quality result from being misreported as adoption.

Protect the decision from weak evidence

Vanity usage can rise while business results fall. Count eligible opportunities, successful completion, manual edits, exceptions and displacement of existing work. Use counterfactual or phased evidence where possible, and label causal claims cautiously.

Keep these artifacts for review:

• value tree with explicit causal assumptions
• metric definitions and denominators
• baseline and observation windows
• slice and counterfactual analysis plan

This practice aligns with the discovery, productisation, client enablement, technical leadership and capstone sequence in the FDE for Professionals course. The course link describes the pathway; the local scenario is fictional and does not claim a client engagement, investment result, hiring decision or certificate.

Practice task

Draw five layers from model output to one business result. For each arrow, write the observation that would show the assumed link is false.

Continue learning

This article is part of the Commercial judgement and delivery leadership sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Write an AI statement of work with measurable acceptance.
• Continue with Estimate the cost of exceptions in an automation business case.

Reference: GOV.UK Service Manual: Measuring Success.