Response Header Specification

The response body is byte-shape unmodified — a standard OpenAI chat.completion object — so existing SDKs parse it with zero changes. Everything ACE-specific rides on headers.

Vendor rate-limit headers. When a provider answered — x-ace-served-by names it — its own bucket headers come back with the values it sent, on every vendor-shaped surface: Anthropic's anthropic-ratelimit-{requests,tokens,input-tokens,output-tokens}-{limit,remaining,reset} and request-id; OpenAI's and Azure's x-ratelimit-{limit,remaining,reset}-{requests,tokens}, x-request-id, apim-request-id, x-ms-request-id; retry-after for all. They are the signal a caller spreading load across several upstream keys steers by, and ACE cannot re-derive them. Streamed responses carry them too — they are fixed at connect time, when the upstream status line arrives and before the first frame, the only moment a stream's headers can flush — and so does /anthropic/v1/messages/count_tokens, which draws on the same request bucket. They are absent on any response the vendor did not produce — a semantic-cache hit, an echo turn, an in-fleet completion — because a bucket figure from an earlier call is one a caller would steer by. Framing headers (content-length, content-encoding) are never relayed, and ACE's own 429 rate_limited (the per-tenant cap, no upstream involved) is unchanged. The list is one allowlist shared by every relay path, so it cannot drift between surfaces.

x-ace-warnings codes.

PII redaction and long prompts. pii_ner has two stages. Patterns (email, card, SSN, phone, IP) are linear and always complete. The learned stage — a local NER model for names and account numbers — costs ~28µs a token on the request path, so it is budgeted: ACE_PII_NER_BUDGET_MS (default 50; 0 lifts it) is the most it may spend on one request. Three rules make the budget safe to rely on:

• A system prompt is not scanned by default, on any surface. It is the operator's instructions, not user data; a name in "you are the assistant for Dr. Okafor's clinic" is the product. Every surface behaves the same way here — Anthropic, Bedrock and Gemini hoist the system prompt, and OpenAI and Responses match them — so the same request measures the same on every door. It is also why a 180k-character system prompt costs the skill nothing at the default. A deployment that does put user data in the system prompt can opt in with ACE_PII_SCAN_SYSTEM (default off). It is deployment-level, like ACE_CACHE_KEY_POLICY: it applies to every tenant on that process and is picked up on redeploy. Switched on, system spans are scanned and redacted on all five surfaces — parity across doors is preserved — and they are scanned BEFORE the conversation, against the same ACE_PII_NER_BUDGET_MS. Only the second property stops holding: a long system prompt now costs the skill what its length implies.
• The cut lands at the end of the conversation, never in its prefix. Turns are scanned oldest first; when the budget runs out, the turns the model finished keep its redactions and the rest carry the patterns' alone. A conversation grows at the end, so the bytes of the prefix your provider has cached do not change from one turn to the next.
• What the model has seen, it remembers. Results are memoized per span of text (offsets and labels, never the text; ACE_PII_NER_MEMO_ENTRIES, default 50,000). A replayed history is a lookup, so a 40-turn agent conversation costs the skill a few milliseconds a turn regardless of its length, and a retry of a request the budget cut picks up where the last try stopped rather than starting over.

When the budget does cut a request, the response says so — x-ace-pii-skipped: ner_model=budget_exceeded, x-ace-pii-stages: ner_regex — and the trace stage carries skipped and reason. A deployment whose prompts are routinely cut should raise the budget, pin ACE_PII_NER_THREADS, or narrow ACE_PII_NER_ENTITIES; the number to watch is the skipped header, not the latency.

The skill latency ceiling. Every heavy stage on the request path — pii_ner, semantic_cache (the embed and the vector-store query), prompt_compaction, llm_router and the injection guard's learned stage — runs under a wall-clock ceiling measured by the request itself: 250 ms for a stage the key runs in prod, 50 ms for one it runs in shadow. A stage that has not answered by then is shed: the request is served without it, the response says so on the header that stage already reports through, and the trace stage carries reason, waited_ms and ceiling_ms. It is a bound on one request's wait, not a quota. Nothing accumulates across calls, there is nothing to reset, and the next request is judged on its own clock.

• ceiling_exceeded — the stage did not answer inside its ceiling; served without it.
• overloaded — the skill worker queue was already deeper than one ceiling can drain; the stage was not attempted rather than queued to time out later.
• loop_stalled — the ceiling's timer fired late: the gateway's event loop was held by something else, and the stage may not have run at all. Read this as a gateway fault, not a slow skill — the trace's late_ms is how long the loop did not run.

GET /api/v1/settings reports the effective ceilings under skill_ceiling (per skill, acting and shadow, with the worker pool that runs them); GET /healthz reports the pool as it stands — what is queued, what is running and for how long, and everything shed so far by reason. A gateway shedding at ceiling_exceeded with nothing queued and a job running for minutes has a wedged worker; one shedding at loop_stalled has a held loop; neither is load, and neither is your key.

This header set is frozen legacy: nothing in it is going away without notice, and nothing is being designed onto it — the rows above list what the gateway already sends. A flat string cannot carry what a shadow skill measured, and on a streamed request x-ace-cost-usd is written before the first token exists — an estimate reported as fact. Both are why /v1/execute exists; new skills report on its trace.