Privacy at the perimeter: masking GPU telemetry before it ever leaves the fleet

Selling an efficiency layer that watches someone else's GPU fleet runs into a wall that has nothing to do with GPUs. Before a customer's security team lets any telemetry leave their perimeter, they ask one question: what identifying data are you taking, and where does it go? If the honest answer is "your hostnames, your job names, your tenant and project labels, and your billing account IDs," the deal is dead — and it should be.

The design response most vendors reach for is a Data Processing Agreement and a promise. ACE's response is architectural: the raw identifiers never leave the customer VPC in the first place. The external Core Brain makes every tiering, guarding, and chargeback decision on pseudonymized tokens computed at the perimeter, under a salt the cloud side never holds. Data minimization stops being a compliance tax and becomes the thing that makes the product installable.

The current market landscape

Privacy-preserving telemetry is a mature research area, and it is worth being precise about what it does and doesn't buy you, because the marketing around "anonymized data" is notoriously loose.

• Differential privacy (Dwork & Roth, The Algorithmic Foundations of Differential Privacy, 2014 [1]) is the gold standard when you need a provable bound on what an aggregate release leaks about any one contributor. It works by injecting calibrated noise, and it is the right tool when the output is a population statistic.
• Industry put DP into practice for exactly the telemetry problem. Google's RAPPOR (Erlingsson, Pihur & Korolova, CCS 2014 [2]) collects client statistics under local differential privacy via randomized response, and Prochlo (Bittau et al., SOSP 2017 [3]) generalized this into the Encode–Shuffle–Analyze (ESA) architecture: encode to minimize, shuffle to break linkability, analyze on the anonymized batch.
• Regulation frames the same goal from the other side. GDPR Article 5(1)(c) [4] requires that data be "adequate, relevant and limited to what is necessary" — the data-minimization principle — and the research community has pushed on operationalizing purpose limitation and minimization in real data-driven systems [5].

Here is the part that matters, and that most "we anonymize your data" claims get wrong: a salted one-way hash is not anonymization. The joint EDPS/AEPD guidance on hashing as a pseudonymization technique [6] is explicit — if anyone retains the salt or a lookup table, the output is pseudonymized data, still personal data, still in scope. Anonymization is a much higher and often unachievable bar. Honesty about that distinction is a feature, not a weakness.

Where ACE is different

ACE isn't trying to release a private aggregate, so DP's noise-injection model is the wrong frame — it forwards per-node mathematical fault and utilization signals (XID codes, ECC counts, temperature, power, SM/memory utilization), not user-level counts to be summed. Its lever is the other two ideas above: minimization and perimeter-local pseudonymization. The novel move is that every policy decision the external brain needs — which tier is this owner, is it SLA-protected, what does this node's fault history imply — is computed against a salted-masked token generated inside the customer VPC, with the salt and the reverse map staying local. The cloud never sees the real name and never holds the key to recover it, yet the loop can still tier, guard, chargeback, and (once a verdict comes back) un-mask locally to actuate. The rest of this post is the mechanism.

Two things leave the perimeter, and only two

Every telemetry sample passes through a hard line at the boundary. Sensitive fields are dropped entirely; the node identifier is masked (not dropped, so cross-sample correlation survives); everything else is a non-sensitive mathematical signal that passes through untouched.

The drop list is deliberately aggressive: hostnames, IP and MAC addresses, model and job names, namespace and pod identifiers, user/owner/team/project/ tenant labels, and cloud billing metadata never cross the boundary. What remains is the fault physics — fault codes, ECC counts, temperature, power draw, and utilization fields. A GPU throwing a fault code at elevated temperature is a fact about silicon, not about a customer's product roadmap.

Free-text is the leakiest surface, so raw error strings are treated as guilty until proven innocent: every one passes through a filter that redacts absolute paths, IP addresses, MAC addresses, and interconnect hardware identifiers wholesale. It would rather redact a harmless mount point than risk leaking topology in a stack trace.

The salt stays home

Node correlation is the one identifier that can't simply be dropped — the brain has to know that the node throwing errors on Monday is the same one that flapped on Sunday. So instead of dropping it, the node identifier is replaced with a stable, one-way token: a deterministic hash salted with a value that is generated and held only inside the customer perimeter. The external brain receives tokens it can group and correlate but can never invert, because it never receives the salt that produced them.

A local reverse map, kept entirely inside the perimeter, lets an inbound verdict — which references only the masked token — be translated back to the real node for actuation. That reverse map is exactly why this is pseudonymization, not anonymization. Re-identification is not just theoretically possible, it is a designed capability — but it lives entirely inside the perimeter. The guarantee on offer is narrow and defensible: to anyone outside the boundary, the tokens are unlinkable to topology, because both the salt and the map that could reverse them never leave. The claim isn't that the cloud holds anonymous data; it's that the cloud never holds the means to de-anonymize it.

Deciding tiers without learning identities

The subtler part is policy. Reclaim and reliability logic need to be tier-aware — an SLA-protected tenant's capacity must never be proposed for reclaim, and a premium owner's faults may warrant faster isolation. That sounds like it needs to know who the owner is. It doesn't.

A specific owner or project identity collapses to a generic tier through a mapping resolved entirely at the perimeter, before anything egresses. Every downstream policy decision — "is this SLA-protected, decline the reclaim" — runs on the tier label alone. The real owner identity never has to leave the perimeter to drive the guard, and the same discipline applies to the utilization stream: identifying fields are hashed and categorized to a tier at the perimeter before egress, while the utilization numbers themselves pass through unmasked.

The system is only as private as the perimeter's handling of three things: the salt, the reverse map, and the tier mapping. Keep those local and the external brain works blind on identity while staying fully sighted on the physics and the policy tier — which is all it ever needs.

Why this is the sellable part

The efficiency numbers are what make ACE interesting to a VP of Infrastructure. This is what makes it approvable by the security org sitting next to them. Minimization at the perimeter means the vendor's blast radius, in the event of a compromise of the Core Brain, is a stream of salted tokens and GPU thermals — not a map of a customer's model portfolio. That is a materially different conversation to have in a security review, and it is designed in, not bolted on.

None of this is exotic cryptography. It's SHA-256, an environment variable, a drop list, and the discipline to keep the salt at home. The lesson from the privacy-telemetry literature above is that the hard part was never the primitive — it was deciding, up front, that identity is not yours to collect.

---

References

1. C. Dwork, A. Roth. The Algorithmic Foundations of Differential Privacy. Foundations and Trends in Theoretical Computer Science, 9(3–4):211–407, 2014. doi:10.1561/0400000042
2. Ú. Erlingsson, V. Pihur, A. Korolova. RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response. ACM CCS 2014. research.google
3. A. Bittau, Ú. Erlingsson, P. Maniatis, et al. Prochlo: Strong Privacy for Analytics in the Crowd. ACM SOSP 2017. arXiv:1710.00901
4. Regulation (EU) 2016/679 (GDPR), Article 5(1)(c) — Principles relating to processing of personal data (data minimisation). gdpr-info.eu
5. Reviving Purpose Limitation and Data Minimisation in Data-Driven Systems. 2021. arXiv:2101.06203
6. EDPS & AEPD. Introduction to the Hash Function as a Personal Data Pseudonymisation Technique. 2019. edps.europa.eu (PDF)