Define an SLO for successful AI task completion

A healthy API can return fast, syntactically valid failures. For an AI workflow, availability becomes meaningful only when a user receives a correct and authorized outcome within the promised time. That end-to-end event is the service-level indicator.

Define the measurable control

Write the success contract before writing the query. Define the task boundary, eligible traffic, correctness rule, authorization rule and time limit. Give every request one terminal outcome: successful, wrong, timed out, safely refused or abandoned. Select an SLO from user need and operating constraints, then attach an error-budget response.

The AI reliability lab makes the decision reproducible with authored data:

from reliability_cases import task_slo_case

result = task_slo_case()
assert result["good"] == 944
assert result["sli"] == 0.944
assert result["target"] == 0.95
assert result["met"] is False

The fictional service completes 944 of 1,000 tasks under the contract, so its SLI is 94.4% and it misses a 95% target. A 200 response does not rescue wrong, late or abandoned tasks. The result is deliberately below target to show that the verifier tests arithmetic, not success marketing.

Challenge the result

Do not quietly exclude hard traffic after observing it. Eligibility rules should be versioned and reviewed. Slice the SLI by tenant, task type, model version and risk because an acceptable aggregate can hide a failing high-impact workflow.

Keep a reviewable evidence pack:

• written success contract and eligible-event query
• terminal-outcome reconciliation
• SLO and error-budget policy
• slice report linked to model and workflow versions

This work aligns with the evaluation, security, cloud operations, reliability and FinOps sequence in the FDE for Professionals course. The linked course describes the learning pathway; this article’s numbers are synthetic and do not report a model, client, audit or production result.

Practice task

Classify twenty recent synthetic traces into terminal outcomes. Reconcile the count to the request total and explain every exclusion.

Continue learning

This article is part of the Advanced AI reliability and assurance sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Continue with Measure retry amplification across an agent workflow.
• Then apply it in Design a circuit breaker around unreliable tools.

Reference: Google SRE Workbook: Implementing SLOs.