Per-Tenant LLM Budgets: Stop One Customer From Spending Your Whole Model Bill

Question

Picture a hypothetical B2B app with an AI assistant in every customer workspace. One customer wires it into a nightly job that loops over tens of thousands of documents, each sent with a long context and a generous max_tokens. Every request is valid and under the per-user rate limit. By morning that customer has spent more on model calls than they pay you in a year, and possibly everyone else's share too: on the Claude API, an organization that reaches its monthly spend cap gets HTTP 429 on every request until 00:00 UTC on the first day of the next month, unless it gets a higher limit sooner.

The fix is a per-tenant budget: the money each customer, and each user inside it, may spend on model calls in a window, checked before every call, settled from real usage after it, and reconciled against the provider's bill. It's the authorize-and-capture bookkeeping of card payments applied to tokens, and a standard piece of the multi-tenant AI products built in the production SaaS course at Cursuri-AI.ro.

The code uses the official anthropic Python SDK and Redis, with Postgres as the alternative. Retries, circuit breakers, agent stop conditions and tracing are separate layers and aren't repeated here. This is only about money: who spent it, how much is left, and what happens when it runs out.

Rate limits are not budgets

Most stacks already have a few kinds of limits. None of them is a budget.

The provider's rate limits protect the provider's capacity. On the Claude API they are measured in requests, input tokens and output tokens per minute, per model, for your whole organization, with a token bucket that refills continuously. They don't know which customer used the capacity, and they don't measure money: for most Claude models, cache reads don't count toward the input-tokens-per-minute limit, yet every one of them is billed.

The provider's spend limits measure money at the wrong granularity: the organization cap, or a lower one you set yourself, switches AI off for your whole product at once. Workspaces can carry their own spend and rate limits, which helps to isolate one very large customer, but prompt caches are isolated per workspace on the Claude API, so splitting traffic across workspaces also splits your cache hits.

Your own per-request rate limits bound how often someone calls, not what a call costs. Long context is billed at a flat per-token rate ("A 900k-token request is billed at the same per-token rate as a 9k-token request"), so cost scales with size. On Claude Opus 5.5, 2,000 input tokens and 500 output tokens cost just under 2 cents; about 870,000 input tokens plus the full 128K-token output cost about $6. Same endpoint, same rate-limit slot, a few hundred times the price.

A budget is an amount of money per tenant per window, reduced by what each call actually cost and checked before the next one starts.

Budget in money, not tokens

Tokens are the wrong unit because they don't have one price. A single Claude response reports up to five differently priced quantities:

Usage field	What it counts	Claude Opus 5.5, per million tokens
input_tokens	input after the last cache breakpoint	$4
cache_creation_input_tokens	input written to the prompt cache (5-minute / 1-hour TTL)	$5 / $8
cache_read_input_tokens	input read from the cache	$0.20
output_tokens	everything generated, thinking included	$20
server_tool_use.web_search_requests	web searches, if you enable the tool	$10 per 1,000

That's a 100× spread inside one model, and cache discounts differ by model: a cache hit costs 10% of the input price on most models, 5% on Claude Opus 5.5 and 2.5% on Claude Fable 5.1. Token quotas also drift. Per the pricing page, the tokenizer used by Claude 4.7 and later models "produces approximately 30% more tokens for the same text", so a per-token quota written for an older model shrinks after a migration.

Thinking is the part that catches people out. It is billed as output "even when the thinking text isn't returned to you", and with summarized thinking "the billed output token count does not match the count of tokens you see in the response" (thinking docs). Adaptive thinking is always on for Claude Opus 5.5, so never estimate cost from visible text. The Messages API reference calls usage.output_tokens "the inclusive, authoritative total used for billing", and output_tokens_details.thinking_tokens says how much of it was reasoning.

Keep budgets in integer micro-dollars. $4 per million tokens is exactly 4 micro-dollars per token, so pricing a response takes a few multiplications and one rounding step:

from decimal import Decimal, ROUND_CEILING

# USD per million tokens, from platform.claude.com/docs/en/about-claude/pricing
# (checked 2026-10-02). This is configuration, not code: reload it whenever prices or
# models change. $X per million tokens is exactly X micro-dollars per token.
_RAW_PRICES = {
    "claude-fable-5-1":  {"input": "10", "write_5m": "12.50", "write_1h": "20", "read": "0.25", "output": "50"},
    "claude-opus-5-5":   {"input": "4",  "write_5m": "5",     "write_1h": "8",  "read": "0.20", "output": "20"},
    "claude-sonnet-5-5": {"input": "2",  "write_5m": "2.50",  "write_1h": "4",  "read": "0.20", "output": "10"},
    "claude-haiku-4-5-20251001": {"input": "1", "write_5m": "1.25", "write_1h": "2", "read": "0.10", "output": "5"},
}
PRICES = {m: {k: Decimal(v) for k, v in row.items()} for m, row in _RAW_PRICES.items()}
WEB_SEARCH_MICROS = Decimal(10_000)   # $10 per 1,000 searches
US_ONLY = Decimal("1.1")              # inference_geo="us": 1.1x on every token category

def _ceil(x: Decimal) -> int:
    return int(x.to_integral_value(rounding=ROUND_CEILING))

def cost_micros(model: str, usage) -> int:
    """What one response cost, in micro-dollars, from its usage block."""
    p = PRICES[model]
    cc = getattr(usage, "cache_creation", None)
    if cc is not None:
        w5, w1h = cc.ephemeral_5m_input_tokens, cc.ephemeral_1h_input_tokens
    else:
        w5, w1h = usage.cache_creation_input_tokens or 0, 0
    tokens = (
        (usage.input_tokens or 0) * p["input"]
        + w5 * p["write_5m"]
        + w1h * p["write_1h"]
        + (usage.cache_read_input_tokens or 0) * p["read"]
        + usage.output_tokens * p["output"]          # thinking is in here
    )
    if getattr(usage, "inference_geo", None) == "us":
        tokens *= US_ONLY
    searches = usage.server_tool_use.web_search_requests if usage.server_tool_use else 0
    return _ceil(tokens + searches * WEB_SEARCH_MICROS)

The cache_creation object splits cache writes by TTL, and its values add up to cache_creation_input_tokens; without it, the function assumes 5-minute writes. The Batch API (50% off) and fast mode (its own rates) need rows of their own if you use them.

Reserve, call, settle

You can't know what a call costs until it's done, and you can't let it start unless the budget can absorb it. Card payments solved this long ago with authorization and capture:

1. Estimate the most the call can cost.
2. Reserve that amount atomically against the tenant's budget and the user's budget. If either can't absorb it, refuse now, before a single token is generated.
3. Call the model.
4. Settle: replace the reservation with the real cost from usage, which returns the unused part to the budget.

The estimate has two halves. Input comes from the token counting endpoint, which takes the same input as a Messages request, isn't billed, and has its own requests-per-minute limit. Its result is an estimate that "might differ by a small amount" and ignores caching, so price counted input at the highest rate the request can incur: the cache-write rate if it uses cache_control. Output is bounded by max_tokens, which includes all thinking in the current turn and is enforced as a strict limit. In the docs' words: "Effort is soft guidance. max_tokens is a strict limit."

import anthropic

client = anthropic.Anthropic()

COUNTABLE = {"model", "system", "messages", "tools", "tool_choice",
             "thinking", "output_config", "cache_control"}
COUNT_MARGIN = Decimal("1.02")   # the count is an estimate; the margin is your choice

def hold_micros(request: dict, cache_ttl: str | None = None, likely_cached: int = 0) -> int:
    """What to reserve for one call. With likely_cached=0 it is a true upper bound:
    every input token at the highest rate it can be billed at, plus max_tokens of output."""
    p = PRICES[request["model"]]
    counted = client.messages.count_tokens(
        **{k: v for k, v in request.items() if k in COUNTABLE}
    ).input_tokens
    in_rate = {None: p["input"], "5m": p["write_5m"], "1h": p["write_1h"]}[cache_ttl]
    fresh = max(counted - likely_cached, 0)
    return _ceil(
        fresh * COUNT_MARGIN * in_rate
        + likely_cached * p["read"]
        + request["max_tokens"] * p["output"]
    )

The counting endpoint rejects requests that use server tools (other than the advisor tool), the MCP connector, or images and documents passed by url or file; for those features, reserve from a configured ceiling. And count with the model you're about to call, since tokenizers differ between generations.

The guarded call is short; reserve and settle come next.

def call_with_budget(c: "Caller", request: dict, request_id: str, hold: int | None = None):
    keys = budget_keys(c, request_id)
    hold = hold if hold is not None else hold_micros(request)
    reserve(c, keys, hold)                    # raises BudgetExceeded or DuplicateRequest
    try:
        response = client.messages.create(**request)
    except anthropic.APIStatusError:
        settle(keys, 0)        # an error response: no message, no usage to settle
        raise
    except anthropic.APIConnectionError:
        settle(keys, hold)     # outcome unknown (timeout, dropped connection): charge the hold
        raise
    settle(keys, cost_micros(request["model"], response.usage))
    record_in_ledger(c, request_id, request["model"], response.usage)
    return response

The request_id is the idempotency key, so it must stay stable across retries of one logical request: take it from the client's idempotency header or your job ID, never from a fresh UUID per attempt. Getting usage right across caching, tools and streaming is much of the day-to-day work of running LLM integrations in production, and it's the only reliable input a budget has.

The reservation: one atomic step

The check and the decrement must be one operation. Two concurrent requests that both read "enough left" and both proceed is the classic overspend, and parallel agent calls make it common. In Redis, a Lua script does it: "Redis guarantees the script's atomic execution", and a script "can perform conditional updates across multiple keys" (Redis docs), here the tenant's budget and the user's budget in one step.

Each budget is a hash with three fields: limit, committed (settled spend) and held (open reservations). Each reservation is a small hash of its own, indexed by deadline in a per-tenant sorted set so abandoned ones can be found.

-- reserve.lua
-- KEYS[1] tenant budget   KEYS[2] user budget   KEYS[3] hold   KEYS[4] holds index
-- ARGV[1] amount in micro-USD   ARGV[2] hold deadline (unix ms)
if redis.call('EXISTS', KEYS[3]) == 1 then
  return 'duplicate'                      -- this request ID already reserved
end

local amount = tonumber(ARGV[1])
for i, key in ipairs({KEYS[1], KEYS[2]}) do
  local b = redis.call('HMGET', key, 'limit', 'committed', 'held')
  if not b[1] then
    return 'no_budget'                    -- fail closed: no limit configured
  end
  local left = tonumber(b[1]) - (tonumber(b[2]) or 0) - (tonumber(b[3]) or 0)
  if amount > left then
    if i == 1 then return 'tenant' end
    return 'user'
  end
end

redis.call('HINCRBY', KEYS[1], 'held', ARGV[1])
redis.call('HINCRBY', KEYS[2], 'held', ARGV[1])
redis.call('HSET', KEYS[3], 'amount', ARGV[1], 'state', 'held',
           'tenant_key', KEYS[1], 'user_key', KEYS[2])
redis.call('ZADD', KEYS[4], ARGV[2], KEYS[3])
return 'ok'

Three details matter. The script fails closed: a budget without a limit refuses. Every key it touches arrives through KEYS, since Redis requires that "all names of keys that a script accesses must be explicitly provided as input key arguments". And every key name carries the tenant ID as a hash tag, {tenant}, which puts all of a tenant's keys in one Redis Cluster hash slot, so the multi-key script survives sharding.

The Python side builds the keys, opens the month's budgets lazily, and turns verdicts into exceptions:

import time
from dataclasses import dataclass
from datetime import datetime
from zoneinfo import ZoneInfo

import redis

r = redis.Redis(decode_responses=True)
reserve_script = r.register_script(open("reserve.lua").read())
settle_script = r.register_script(open("settle.lua").read())

HOLD_TTL_MS = 15 * 60 * 1000    # longer than your slowest call, retries included
TOMBSTONE_S = 7 * 24 * 3600     # how long a finished request ID stays deduplicated

@dataclass
class Caller:
    tenant_id: str
    user_id: str
    tz: str               # the tenant's billing time zone, e.g. "Europe/Bucharest"
    tenant_limit: int     # micro-USD per month, from the tenant's plan
    user_limit: int       # micro-USD per month for this user; equal to tenant_limit = no user cap

class BudgetExceeded(Exception):
    def __init__(self, scope: str, resets_at: datetime | None = None):
        super().__init__(scope)
        self.scope, self.resets_at = scope, resets_at

class DuplicateRequest(Exception):
    pass

def budget_keys(c: Caller, request_id: str) -> list[str]:
    period = datetime.now(ZoneInfo(c.tz)).strftime("%Y-%m")
    t = "{" + c.tenant_id + "}"   # hash tag: one cluster slot per tenant
    return [f"budget:{t}:{period}", f"budget:{t}:user:{c.user_id}:{period}",
            f"hold:{t}:{request_id}", f"holds:{t}"]

def next_reset(tz: str) -> datetime:
    now = datetime.now(ZoneInfo(tz))
    year, month = (now.year + 1, 1) if now.month == 12 else (now.year, now.month + 1)
    return datetime(year, month, 1, tzinfo=ZoneInfo(tz))

def reserve(c: Caller, keys: list[str], amount: int) -> None:
    r.hsetnx(keys[0], "limit", c.tenant_limit)   # open this month's budgets on first use
    r.hsetnx(keys[1], "limit", c.user_limit)
    deadline = int(time.time() * 1000) + HOLD_TTL_MS
    verdict = reserve_script(keys=keys, args=[amount, deadline])
    if verdict == "duplicate":
        raise DuplicateRequest(keys[2])
    if verdict != "ok":
        raise BudgetExceeded(verdict, next_reset(c.tz))

def settle(keys: list[str], cost: int, mode: str = "settle") -> str:
    return settle_script(keys=keys, args=[cost, mode, TOMBSTONE_S])

A new month opens the first time someone spends in it, at the limit the tenant's plan has at that moment. A plan change mid-month is a plain HSET of limit.

Settlement: idempotent, and never refused

Settlement follows two rules. It must be idempotent, because whatever calls it will be retried: your queue, your web framework, a user clicking twice. And it must never refuse: by the time you settle, the money is spent, and the budget's job is to record it. If the real cost exceeds the hold (the input count was an estimate), the budget goes slightly over; the overshoot is bounded by your margin and visible in the metrics.

-- settle.lua
-- KEYS: the same four as reserve.lua
-- ARGV[1] real cost in micro-USD   ARGV[2] 'settle' or 'expire'   ARGV[3] tombstone TTL (s)
local h = redis.call('HMGET', KEYS[3], 'amount', 'state')
local held, state = tonumber(h[1]), h[2]
if not state then return 'unknown' end            -- never reserved, or tombstone gone
if state == 'settled' then return 'duplicate' end
if ARGV[2] == 'expire' and state ~= 'held' then return 'noop' end

local cost = tonumber(ARGV[1])
if ARGV[2] == 'expire' then cost = held end       -- the worker vanished: assume the worst

local delta = cost
if state == 'held' then
  redis.call('HINCRBY', KEYS[1], 'held', -held)
  redis.call('HINCRBY', KEYS[2], 'held', -held)
else                                              -- 'expired': the full hold was charged
  delta = cost - held
end
redis.call('HINCRBY', KEYS[1], 'committed', delta)
redis.call('HINCRBY', KEYS[2], 'committed', delta)
local final_state = 'settled'
if ARGV[2] == 'expire' then final_state = 'expired' end
redis.call('HSET', KEYS[3], 'state', final_state, 'cost', cost)
redis.call('ZREM', KEYS[4], KEYS[3])
redis.call('EXPIRE', KEYS[3], ARGV[3])
return 'ok'

Finished holds stay behind as tombstones with a TTL, so a duplicate settle does nothing. A settle that arrives after its hold was expired (next section) corrects the charge from the full hold down to the real cost. And because the budget keys live in the hold, a call reserved at 23:59 on the last day of the month settles against the month it was reserved in.

Keep a durable record next to the gate. The Redis Cluster specification says "usually there are small windows where acknowledged writes can be lost", which is acceptable for a gate and not for a ledger. Write every settled call to an append-only table, and rebuild committed from it if Redis is ever lost or wrong:

CREATE TABLE llm_ledger (
    request_id   TEXT        PRIMARY KEY,   -- the same ID as the Redis hold
    tenant_id    TEXT        NOT NULL,
    user_id      TEXT        NOT NULL,
    model        TEXT        NOT NULL,
    usage        JSONB       NOT NULL,      -- the raw usage block, so history can be re-priced
    cost_micros  BIGINT      NOT NULL,
    created_at   TIMESTAMPTZ NOT NULL DEFAULT now()
);
-- record_in_ledger() is INSERT ... ON CONFLICT (request_id) DO NOTHING

Storing the raw usage, not only the cost, turns a mistake in the price table into a re-computation instead of a loss.

Holds that never come back

A worker that reserves and then dies (OOM-killed, deployed over, cut off by the network) leaves a hold nobody will settle, and those holds eat the budget until a tenant is refused with money left on paper. Give each hold a deadline longer than your slowest legitimate call, retries included, and sweep:

def sweep_expired_holds(tenant_id: str, batch: int = 100) -> int:
    """Run every minute or so, for each tenant with a holds index (SCAN for holds:*)."""
    index = "holds:{" + tenant_id + "}"
    now_ms = int(time.time() * 1000)
    expired = 0
    for hold_key in r.zrangebyscore(index, "-inf", now_ms, start=0, num=batch):
        h = r.hgetall(hold_key)
        if not h:
            r.zrem(index, hold_key)
            continue
        keys = [h["tenant_key"], h["user_key"], hold_key, index]
        if settle_script(keys=keys, args=[0, "expire", TOMBSTONE_S]) == "ok":
            expired += 1
    return expired

Charging the full hold is the conservative default: the worker may have finished the call before it died, and there's no usage to prove otherwise. If you'd rather absorb that cost, release the hold at zero instead. Either way, count expired holds; a rising number means something crashes between reserve and settle.

If Postgres is your gate. The same design works without Redis. A conditional UPDATE is the check, the row lock makes it atomic, and a missing row in RETURNING means refused:

-- One row per budget: (tenant, period) and (tenant, user, period).
-- In one transaction: commit if both rows come back, roll back otherwise.
UPDATE llm_budget
   SET held = held + %(amount)s
 WHERE budget_key IN (%(tenant_key)s, %(user_key)s)
   AND committed + held + %(amount)s <= limit_micros
RETURNING budget_key;

You get durability, and the ledger insert can share the transaction. The price is that every request from one tenant serializes on that tenant's budget row, so measure before you pick a side.

Streaming: the bill arrives at the end

Streaming changes when you learn the cost, not what it is. The documented event flow is message_start, the content blocks, one or more message_delta events, then message_stop. Input usage, cache reads and writes included, sits in the Message inside message_start. The output count comes in message_delta, and "the token counts shown in the usage field of the message_delta event are cumulative." Two consequences follow.

Settle from the final usage, not the first. The SDK's accumulator overwrites its snapshot with each cumulative count, so settle on the usage of stream.get_final_message(). Input can change too: the docs' server-tool example shows a larger input_tokens in the final message_delta than in message_start.

There is no running output meter. Content deltas carry text, not token counts, and when thinking is omitted or summarized the visible text undercounts what you pay for. So you can't stop a stream "when it hits the budget" by counting what you've seen. The hard stop for one call is max_tokens, set before it starts: when a tenant is low, size max_tokens from what's left (shown below) and let the API enforce it.

def stream_with_budget(c: Caller, request: dict, request_id: str, hold: int | None = None):
    keys = budget_keys(c, request_id)
    hold = hold if hold is not None else hold_micros(request)
    reserve(c, keys, hold)
    settled = False
    try:
        with client.messages.stream(**request) as stream:
            for text in stream.text_stream:
                yield text            # a client disconnect arrives here as GeneratorExit
            final = stream.get_final_message()
        settle(keys, cost_micros(request["model"], final.usage))
        settled = True
        record_in_ledger(c, request_id, request["model"], final.usage)
    finally:
        if not settled:
            # Aborted or failed mid-stream: the final message_delta never arrived, and
            # thinking you never saw was still generated. Charge the hold; reconcile later.
            settle(keys, hold)

When the user closes the tab, you have two honest options: charge the hold (conservative) or estimate from what you streamed (generous, and blind to thinking). Choose deliberately, and let reconciliation show what the choice costs.

Agent loops: budget the step, not the session

An agent run is a series of model calls with tools in between, and its context grows at every step. Reserving for the whole session up front looks tidy and behaves badly: the pessimistic total is locked away from the tenant's other users for the whole run, and a crash leaks all of it. Reserve per step instead, each step with its own reserve, call, settle and idempotency key, and give the run a ceiling of its own on top.

STEP_MAX_TOKENS = 8_000      # per-step output cap, whatever the feature's default is

def agent_step(c: Caller, run: dict, request: dict, cache_ttl: str | None = "5m"):
    request = {**request, "max_tokens": min(request["max_tokens"], STEP_MAX_TOKENS)}
    hold = hold_micros(request, cache_ttl, likely_cached=run["cached_prefix"])
    if run["spent"] + hold > run["ceiling"]:
        raise BudgetExceeded("run")           # the run's own cap, separate from the month
    response = call_with_budget(c, request, f"{run['id']}:{run['step']}", hold=hold)
    u = response.usage
    run["spent"] += cost_micros(request["model"], u)
    run["cached_prefix"] = (u.cache_read_input_tokens or 0) + (u.cache_creation_input_tokens or 0)
    run["step"] += 1
    return response

The per-step max_tokens cap stops one runaway step from eating the run's allowance. likely_cached fixes a quieter problem: after the first step, most of an agent's input is the conversation so far, already in the cache. On Claude Opus 5.5 that prefix costs $0.20 per million tokens as a cache read and $5 as a 5-minute cache write. Reserve it at the write rate and late-run reservations come out many times too large, refusing tenants who still have budget; price it as a read and accept a small overshoot when the cache misses. The run's other stop conditions belong to the loop, which is the territory of designing agents that stay inside their limits.

Soft limits degrade, hard limits refuse

Give every budget two thresholds. Past the soft one, keep serving but spend less:

• Route new conversations to a cheaper model. Claude Sonnet 5.5 costs $2/$10 per million input/output tokens against $4/$20 for Opus 5.5, but its cache reads cost the same $0.20, so cache-heavy workloads save less than the headline suggests. Don't switch in the middle of a conversation: Sonnet 5.5 can't read Opus 5.5's thinking blocks, so the earlier turns' reasoning is dropped.
• Lower the effort. On Claude Opus 5.5 the effort docs call effort "the primary control for how much the model reasons and what a request costs". Changing it invalidates the cached message blocks, so this too is a decision for the start of a conversation.
• Shrink max_tokens to what the budget can still pay for.

A cheaper model isn't automatically a cheaper answer. Sonnet 5.5 defaults to high effort where Opus 5.5 defaults to medium, so compare cost per finished task on your own traffic, not price per token.

SOFT_LIMIT = Decimal("0.80")   # a starting point; tune it per product
CHEAPER = {"claude-fable-5-1": "claude-opus-5-5", "claude-opus-5-5": "claude-sonnet-5-5"}
MIN_USEFUL_OUTPUT = 1_024      # below this, refuse instead of returning a stub

def headroom(c: Caller, keys: list[str]) -> tuple[int, Decimal]:
    """(micro-USD left, share used) for the tighter of the two budgets. For routing only:
    the reservation script remains the only check that counts."""
    left, used = None, Decimal(0)
    for key, default in ((keys[0], c.tenant_limit), (keys[1], c.user_limit)):
        limit, committed, held = r.hmget(key, "limit", "committed", "held")
        limit = int(limit or default)
        spent = int(committed or 0) + int(held or 0)
        left = limit - spent if left is None else min(left, limit - spent)
        used = max(used, Decimal(spent) / limit if limit > 0 else Decimal(1))
    return left, used

def plan_call(c: Caller, keys: list[str], request: dict, new_conversation: bool) -> tuple[dict, int]:
    left, used = headroom(c, keys)
    if used >= SOFT_LIMIT and new_conversation and request["model"] in CHEAPER:
        request = {**request, "model": CHEAPER[request["model"]]}
    p = PRICES[request["model"]]
    input_hold = hold_micros({**request, "max_tokens": 0})
    affordable = int(Decimal(left - input_hold) / p["output"])
    # Never go below the floor: if even that doesn't fit, the reservation refuses,
    # and it knows which budget (tenant or user) ran out.
    max_tokens = max(MIN_USEFUL_OUTPUT, min(request["max_tokens"], affordable))
    return {**request, "max_tokens": max_tokens}, input_hold + _ceil(max_tokens * p["output"])

The floor matters on models with thinking always on: a call squeezed to a few hundred output tokens can spend them all reasoning and stop with stop_reason: "max_tokens" before saying anything useful. The tenant pays and gets nothing.

Past the hard threshold, refuse, clearly and honestly. Not a generic 500, not an endless spinner, not a silent switch to worse answers. A good refusal says what happened, whose budget it was, when it resets and who can change it, with an error code that tells clients not to retry:

REFUSALS = {
    "tenant": "Your organization has used this month's AI budget.",
    "user": "You've used your share of your organization's AI budget for this month.",
    "run": "This task reached its spending cap before it finished.",
}

def refusal_body(e: BudgetExceeded) -> dict:
    reason = REFUSALS.get(e.scope, REFUSALS["tenant"])
    resets = e.resets_at
    return {
        "error": "ai_budget_exhausted",     # distinct code: clients must not auto-retry
        "scope": e.scope,
        "resets_at": resets.isoformat() if resets else None,
        "message": reason + (f" It resets on {resets.day} {resets:%B %Y}." if resets else "")
                   + " An administrator can raise the limit in billing settings.",
    }

The message never implies the allowance was open-ended. The plan had an AI budget, and this is it.

Two levels, because the tenant pays and users spend. The tenant budget protects your margin; the per-user budget inside it protects the tenant's other employees from one colleague's batch job. Let the tenant's administrator set the per-user share, and check both in one atomic step, as the Lua script does, so a reservation that fits the user but not the tenant never half-commits.

Windows need a time zone. "Resets on the 1st" should mean midnight where the customer is, so the window key uses the tenant's billing time zone. Keep ledger timestamps in UTC anyway: the provider's cost buckets are UTC days.

Reconcile against the provider's numbers

Your ledger is a model of the bill, so check it against the bill. Anthropic's Usage and Cost Admin API has two endpoints. /v1/organizations/usage_report/messages reports tokens in 1-minute, 1-hour or 1-day buckets, groupable by API key, workspace, model, service tier and a few more dimensions. /v1/organizations/cost_report reports USD costs in daily buckets, groupable by workspace or by description, which carries the model. Both need Admin API credentials such as an Admin API key (sk-ant-admin01-...), they aren't available to individual accounts, and data "typically appears within 5 minutes of API request completion".

from collections import defaultdict

import httpx

def provider_cost_by_model(starting_at: str, ending_at: str, admin_key: str) -> dict[str, Decimal]:
    """Micro-dollars per model (or per non-token cost type) for whole UTC days."""
    totals: dict[str, Decimal] = defaultdict(Decimal)
    params = {"starting_at": starting_at, "ending_at": ending_at, "group_by[]": "description"}
    while True:
        resp = httpx.get(
            "https://api.anthropic.com/v1/organizations/cost_report",
            params=params,
            headers={"x-api-key": admin_key, "anthropic-version": "2023-06-01"},
        )
        resp.raise_for_status()
        body = resp.json()
        for bucket in body["data"]:
            for item in bucket["results"]:
                # "amount" is a decimal string in cents: "123.45" means $1.23
                totals[item["model"] or item["cost_type"]] += Decimal(item["amount"]) * 10_000
        if not body["has_more"]:
            return totals
        params["page"] = body["next_page"]

Compare the totals per model and UTC day with the sum of cost_micros in your ledger. None of the provider's dimensions is your customer, so reconciliation checks totals and your ledger does the attribution. When they disagree, the usual suspects are a stale price table, a code path that calls the model around the guard (background jobs and new features are the classics), aborted streams and expired holds, or a pricing modifier you didn't model. Priority Tier costs aren't in the cost endpoint; track them through the usage endpoint. Pick a drift threshold and alert on it.

Four numbers to watch

1. Settled cost per tenant, against time elapsed in the window. A tenant at 60% of the month's budget on day 5 is a conversation for this week, not day 12. That's burn rate, the idea SRE teams apply to error budgets, and operating AI workloads with SRE discipline takes it well beyond cost.
2. Refusal and degrade rates, split by scope (tenant, user, run). A rising user-scope rate inside one tenant usually means the per-user shares are wrong, not that anyone is misbehaving.
3. Reserved versus settled, per feature. Reserving ten times what you spend means refusing tenants who still have budget; tighten that feature's max_tokens rather than raising the limit.
4. Expired holds and reconciliation drift. Both should sit near zero and stay flat. Either one climbing means something crashes between reserve and settle, or calls the model around the guard.

Keep per-tenant cost in the ledger rather than as a label on a high-cardinality time series. If you trace model calls already, put the tenant ID and settled cost on the call's span, so an expensive request leads straight to its customer.

FAQ

Why not count tokens per tenant instead of money?
Because tokens don't have one price: inside one model an output token costs 100 times a cache read, and newer tokenizers change the count for the same text. Money is the only unit that matches the bill you pay.

Redis or Postgres for the budget?
Redis for the gate when request rates are high and latency matters, Postgres for the ledger in every case. Postgres alone is a fine gate while no single tenant's traffic makes its budget row a hot spot.

The short version

• Provider rate limits and spend caps protect the provider and your organization as a whole. Neither knows your customers.
• Budget in micro-dollars, priced from every usage field, with thinking counted as output.
• Reserve the worst case atomically against tenant and user, call, then settle idempotently from real usage. Settlement never refuses.
• Expire abandoned holds and keep a durable ledger behind the Redis gate.
• In streams, usage arrives at the end, so the hard stop is max_tokens sized from what's left.
• Reserve per agent step, cap each step, and price the cached prefix as a cache read.
• Degrade at the soft limit, refuse honestly at the hard one, and reconcile daily against the Cost API.

None of this makes a call cheaper. It makes every call's cost belong to someone, which you need before you can price the feature, sell it to larger customers, or sleep through a customer's nightly batch job.

---

Metering, tenancy and billing are the unglamorous half of every AI product. For the full path from prototype to paying customers, see the hands-on course on shipping a production AI SaaS on Cursuri-AI.ro, Eastern Europe's AI education platform.