EXPERIMENT_046 // TOKEN.FINOPS.PROXY
A thin proxy layer in front of @diabolicallabs/llm-client (Anthropic + OpenAI) tags every call with (agent, persona, model, scope, route, intent) via AsyncLocalStorage and persists token and cost to an embedded DuckDB store, surfacing cost by capability, by model-tier shift, and by incident class instead of a per-provider spend total.
LOADING EXPERIMENT...
Every fleet-wide cost decision — opusplan ratification, provider spend caps, model tiering across a dozen agents, a persona's Haiku-to-Sonnet promotion — gets logged as a decision, but its consequence never gets measured. A standard spend dashboard answers "what did we spend"; this experiment answers "what did the cost shape make possible". It wraps every LLM call in a tag tuple the toolkit itself has no way to infer — which agent, which persona, which capability, which route, which intent — and asks three questions no per-provider total can answer: which capability is actually expensive, what a model-tier shift cost in practice, and how autonomous-run spend compares to supervised-run spend.
HOW IT WORKS
The tag tuple (agent, persona, scope, route, intent) is bound to the current async call chain via Node's AsyncLocalStorage — not threaded through llm-client's own LlmCallOptions. llm-client's providerOptions escape hatch is provider-specific (Perplexity reads search filters from it directly, and unrecognised fields are documented as forwarded unchanged for that provider), so stashing custom tag fields there risks a leak onto the wire for a provider that doesn't ignore unknown keys. AsyncLocalStorage keeps the tag context entirely out of the request — the hook reads it, it never mutates what's sent.
createFinOpsClient() wraps llm-client's own createClient() and wires a single afterCall hook (llm-client v1.5.0+'s hooks API) that reads the ambient tag context, reads the actually-serving model and usage off the hook's own context (so provider failover is captured correctly), and writes one row to DuckDB. The write is fire-and-forget — never awaited inside the hook — so a slow or failed database write can never add latency to, or break, the LLM call the caller is waiting on. This is the proxy's error-path contract: total DuckDB failure degrades to "this call wasn't recorded", never to a broken response.
Capability cost groups by scope, not provider, so the answer to "what's expensive" is a capability name, not a vendor. Tier-shift cost groups by persona and model, split before and after a named cutover timestamp, so a Haiku-to-Sonnet promotion shows up as a real cost delta between two rows, not a single before/after number. Incident cost groups by intent — autonomous-run vs. supervised-run tags surface directly as separate rows, with a per-intent error rate riding alongside the cost. All three run as real GROUP BY queries against an embedded DuckDB file, chosen over a managed Postgres instance specifically so a Playground experiment doesn't need a new Railway service to exist.
WHAT THIS PROVES
A tag taxonomy that has no way to leak onto the wire request, because it never enters llm-client's own call options.
Cost tracking with a genuine error-path contract: proxy failure degrades to an unrecorded call, never to a broken one.
That 'cost per capability' and 'cost per provider' are different questions, and the second one is usually the wrong one to be asking.