Set per-user quotas for expensive AI operations

One “request” might mean a short classification or a research workflow with retrieval, multiple model calls and tools. Counting both equally makes a quota easy to bypass and hard to budget.

Decide before execution

The LLMOps and security lab assigns authored operation units and checks usage by tenant-user pair.

from llmops_cases import quota_case

result = quota_case()
assert result["limits"] == {"standard": 5, "pro": 12}
assert result["denied_indices"] == [0]
assert result["decisions"][1]["allowed"] is True
assert result["decisions"][2]["allowed"] is True
assert result["tenant_user_keys"] is True

T1/U1 is rejected before two more units are spent. T1/U2 and T2/U1 keep independent buckets. The units are fixtures rather than prices or observed usage.

Meter the scarce resource

Define units from forecast cost and abuse potential: generated tokens, image resolution, tool calls, retrieved bytes or a weighted workflow estimate. Reserve the estimated amount atomically before work starts, then reconcile with actual usage. Without atomic admission, parallel requests can all pass the same remaining-balance check.

Use layered limits: per user to control misuse, per tenant for purchased capacity, per IP/device for unauthenticated abuse and a service-wide circuit breaker. Keep authentication and entitlement authoritative; do not accept a user-supplied plan name. Separate rolling rate limits from monthly spend budgets.

Return a stable error with the retry/reset time and a lower-cost allowed path where appropriate. For long workflows, propagate a remaining budget to child calls and stop cleanly when it is exhausted. A quota must not leave half-completed external writes.

Test bypass and fairness

Exercise concurrent requests, retries, multiple tenants, account switching, cancelled work and failed operations. Decide which failures receive a refund. Monitor denial rate and false positives by legitimate task class; raise capacity through reviewed policy, not hidden exceptions in prompts.

The Generative & Agentic AI course connects quotas to cost-per-success, tool-call budgets and resilient workflows.

Reproducible exercise

Build a local token-bucket or reservation ledger. Launch ten concurrent fixture requests against one budget and prove total committed units cannot exceed the limit, even when two requests retry.

Continue learning

This article is part of the LLMOps, security and operational evaluation sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Semantic caching: test incorrect cross-user reuse.
• Continue with Build a redaction check for generated output.

Reference: OWASP Top 10 for LLM and GenAI Applications (including unbounded consumption).