Unified Compute Dashboard: Real-Time Token Analytics and Expenditure Insights
Empirical proof of performance for every ACE gateway optimization—caching, difficulty routing, and learned compaction—delivered through live request-stream telemetry and counterfactual FinOps attribution.
The other posts in this series describe layers that reduce a bill: a cache that skips a call, a router that selects a cheaper model, a compactor that shortens a prompt. This post covers the dashboard that reports what those layers did — a live view served by the gateway, fed by the same request stream that serves traffic.
What it shows
The first section is Proxy Savings — Savings Proof: five headline numbers — tokens saved, dollars reclaimed, cache-hit ratio, savings rate, and latency delta — each with a live trendline. Below that:
- Request pipeline — requests received, requests answered from cache, requests compacted before the upstream call, and the resulting token count change.
- Routing distribution — which requests were classified into which skill/complexity bucket, and which model endpoint served them.
- Cost attribution — spend and savings by provider, team, and feature tag, plus a run-rate projection once traffic is sufficient to extrapolate from.
- Request lifecycle — recent requests, each annotated with cache hit or miss, routing tier, compaction applied, cost, and what the same tokens would have cost against the most expensive model in the roster as an explicit counterfactual.
The dashboard renders from the gateway itself with no charting library, no JS framework and no CDN dependency, so evaluating the gateway does not require also evaluating a dashboarding stack.
Where the numbers come from
Every card on the page is a view over one event stream rather than several independently maintained counters. Each request — cache hit or miss, cheap-tier or flagship, compacted or not — produces one record carrying tokens in and out, cost, cache tier, provider, team, and latency. The dashboard, the metrics feed, and the ROI analytics view read the same record, so "tokens saved" cannot differ between them.
Recording that event is synchronous and non-blocking: the record is appended and control returns immediately, so accounting does not add latency to the response.
The "dollars saved" figure is a counterfactual. A cache hit costs effectively nothing; what it avoided must be credited as the real avoided cost rather than a capacity estimate, or two one-word exchanges can appear to save tokens that never existed. The underlying invariant is that tokens saved can never exceed tokens actually processed.
Why it is its own layer
A savings claim that cannot be observed cannot be acted on: tuning a cache threshold, detecting a routing regression, or noticing a compaction ratio trending to zero all require the per-request record. The dashboard is what makes the other layers in this series measurable against live traffic without a separate observability stack.
References
- FinOps Foundation, FinOps Framework — the industry vocabulary this dashboard's "reclaimed spend" and cost-attribution views are built around. finops.org/framework
- Prometheus, Exposition Formats — the plain-text metrics format this kind of gateway telemetry is built to be compatible with, so an existing Prometheus/Grafana setup can scrape it directly. prometheus.io/docs/instrumenting/exposition_formats
- Timescale, TimescaleDB Documentation — the class of time-series Postgres store this kind of production telemetry backend is built on. docs.timescale.com