Redacting what has no shape

Last month we argued that a gateway sitting in front of every model call is the natural place to stop PII leaving in a prompt, and we ended that post with an admission: the redactor was regex-only, and "a learned PII/NER model is the natural next step." That step has shipped. pii_ner is a per-key toggle on the settings page today.

This post is the measurement, not the announcement — including the parts where the measurement disagrees with what we shipped.

What the pattern layer could never do

The existing redactor is five deterministic rules — email, SSN, card number (Luhn-checked, so order ids and hashes don't match), phone, and optionally IPv4. It is fast, it is exact, and its limits are structural: it catches things that have a shape. A name has no shape. Neither does a street address, an IBAN, a passport number, or a plaintext password sitting in a pasted .env block.

The new stage is an Apache-2.0 bert-small token classifier (gravitee-io/bert-small-pii-detection), int8 ONNX, 28.7MB, run locally on onnxruntime — a dependency the semantic cache's embedder already brought into the tree. Nothing leaves the box. That last point is not incidental: the model we originally shortlisted was license-clean but shipped no ONNX artifact, and its package resolved to either a 2GB torch install or a client for a hosted API. Sending prompts to a third party to find the PII in them inverts the entire point of an egress redactor.

Both stages emit character spans over the same original text, and one merge pass applies them, with patterns winning every overlap — a Luhn-checked card number is a stronger claim than a softmax. With the toggle off, output is byte-identical to the pattern-only redactor.

Why we re-measured

The claim we shipped on the settings page — 93.8% entity recall, 4% of ordinary prompts altered — came from 15 labeled sentences and 25 hand-written prompts. Our own internal survey said so in its open-questions list: "the false-positive rate is measured on 25 hand-written prompts, not on traffic."

That is a fine number for a development signal and not a number to put in front of a tenant deciding whether to let a model rewrite their prompts. So we built the corpus that should have existed first.

800 prompts, 1,803 labeled spans, deliberately agent-shaped, because agent traffic is what a gateway sees and it looks nothing like newswire:

Every hand-written identifier is from a documentation range — RFC 2606 domains, 555-01xx numbers, RFC 5737 addresses, published test cards. Nothing came off a real account.

One finding shaped the whole design. The model card benchmarks against several "external" corpora — and every one of them is also listed as a training source for the checkpoint. Those published numbers are in-distribution, not held out. There is no permissively-licensed public span-annotated PII corpus that is clean for this model. We kept a public slice, marked it contaminated, and report it separately. The gap is exactly the direction you'd expect: 94.0% precision on the contaminated slice against 85.0% on the clean one.

Before and after

Same 1,803 gold spans, same corpus, relaxed span matching:

Recall of all PII in the corpus goes from 28.8% to 63.7% — the learned stage roughly doubles what gets stripped — for 6.5 points of precision.

The business case is the marginal number: 630 of 1,803 spans (34.9%) are caught by the combined redactor and missed by the regex, across 396 of the 800 prompts, with zero regressions. The gains are exactly where the pattern layer was blind — PERSON 443, licence plate 53, IBAN 39, passport 26, bank number 20, password 17.

A worked example, from a stack trace pasted into a debugging prompt:

raw   : {"error": "not_found", "hint": "did you mean Elena Petrova?", "contact": "elena.petrova@example.org"}
before: {"error": "not_found", "hint": "did you mean Elena Petrova?", "contact": "[REDACTED_EMAIL]"}
after : {"error": "not_found", "hint": "did you mean [REDACTED_PERSON]?", "contact": "[REDACTED_EMAIL]"}

The regex took the email and walked straight past the name sitting next to it.

On the shipped claim

Measured like-for-like — on the entities the default configuration actually attempts — recall is 85.1% (n=1,351 spans), against the shipped 93.8% (n=15 sentences). The claim was optimistic, but it was not fantasy; it was small.

The 63.7% above is a different and larger question: it counts DATE_TIME, LOCATION and ORGANIZATION spans that the default entity set deliberately excludes. That 452-span gap is the argument for the strict entity set, not a defect — turning it on lifts recall to 89.0% for 3.4 points of precision.

What it costs

Measured on an Apple M5 (10 cores), onnxruntime 1.28, warm cache, at the seam the gateway actually calls:

About 30µs per token, spent synchronously on the request path. Under 256 tokens — the overwhelming majority of prompts — it is unmeasurable next to a provider round-trip. Above 2k tokens it is a deliberate decision, and if your traffic is RAG-shaped it is a product decision rather than a toggle.

The number that was missing from our own copy: concurrency. Every published figure was a single request on an idle machine, which is not what a shared gateway is. At eight concurrent long prompts, the p95 a tenant sees is 445 ms — 7.6× the single-request p50. Single-request latency is not the number to plan capacity with.

What we got wrong

The eval's most useful output is the list of things it contradicted.

The narrow entity set excludes the wrong entity. ORGANIZATION is held out of the default set because its precision would "punch holes in ordinary engineering prompts." Measured across 300 clean prompts, ORGANIZATION produced 8 false-positive spans. PERSON — which is in the default set — produced 69:

raw  : Is there a managed Redis that speaks the Airflow wire protocol?
after: Is there a managed [REDACTED_PERSON] that speaks the Airflow wire protocol?

Jenkins, Redis, Watson, Jira, Kibana, Claude Opus — all redacted as people. The reasoning behind the narrow default was right. It was applied to the wrong label.

The false-positive rate depends entirely on what you count. On untargeted benign prompts, 3.8% are altered — consistent with the 4% we shipped. On prompts deliberately seeded with product and person-like names, 20.3%. Both are true; quoting either alone is misleading.

PASSWORD fails the default set's own admission test at .354 precision — worse than the ORGANIZATION figure the set exists to exclude. It tags fragments of git SHAs.

"Patterns win every overlap" backfires. The rule is right for a Luhn-checked card and wrong for the loose phone rule, which clips ten digits out of the middle of an IBAN the model had labelled correctly: NL91ABNA[REDACTED_PHONE] instead of one clean [REDACTED_IBAN_CODE]. Sixteen instances.

The pattern layer is not innocent either. It alters 18.2% of version-and-network prompts, all of it the phone rule eating 10-digit epoch timestamps. That is happening today, with the learned stage switched off.

Dense tabular pastes are the weak spot, and they are exactly the shape of agent tool output. Recall there is 53.0%, and the failures are structural rather than statistical — a span swallowed a newline and welded two CSV rows together, deleting a city and leaving the name it was meant to protect standing.

What changes

Measured, not promised:

1. PERSON needs context-word suppression or a product-name deny-list before the default set is defensible on engineering traffic. It is the largest single source of false positives by 8×.
2. PASSWORD and US_LICENSE_PLATE come out of the default set, on the same precision criterion that kept ORGANIZATION out.
3. The phone rule must lose to structured identifiers, inverting the current priority for that one pair.
4. Threshold 0.6 is the better default for prompt-preserving deployments: in-scope precision rises 86.5% → 89.7% for 5 points of recall (85.1% → 80.1%), and altered clean prompts drop from 20.3% to 16.3%.
5. The settings copy gets the corrected numbers with their sample sizes attached — including the concurrency figure, and the declared memory footprint, which describes a 29MB artifact for a process that grows ~147MB.

Honest scope

English-only. A 29MB four-layer model, not a frontier one. It raises recall on names and addresses materially; it does not make redaction complete, and nothing here should be read as if it did. The pattern layer stays in front of it and wins every overlap.

The corpus, the harness and both analyses are committed alongside the skill rather than living in someone's notebook — which is the actual lesson. A number measured during development becomes a product claim on a settings page, and nothing forces it to be re-measured at a defensible sample size before it gets there. Now something does.

---

Turn it on: Settings → Security & Compliance → PII NER stage. It runs inside the pattern redactor, so PII redaction must be enabled too. The weights are baked into the gateway image; if they can't be loaded, the redactor serves patterns-only rather than failing your request.