Blog / Engineering
Build notes
My experience masking real data before it reaches an LLM
An AI reads your email, but it never sees your name. Here's how that works, why finding the names is the hard part, and the mistake that taught me the most.
I'm building Ariadot, a small app that reads your email and sends you one short brief a day of what's open and what's coming up. The renewal you forgot, the reply you owe, the form due before term starts.
There's an obvious problem hiding in that. To do this, an AI has to read your actual email. Your name, your kids' names, account numbers, the lot. For an app whose whole pitch is "you can trust me with your inbox," that's the entire ballgame. Get it wrong and there's no product.
So the rule from day one was simple: the AI model never sees a single real name. Not "encrypted." Not "we promise not to look." The model literally never knows who you are. Here's how that works, and the mistake that taught me the most.
The basic idea
Before any of your email goes to a cloud AI, it passes through a step that swaps every personal detail for a realistic fake. A name becomes a different name. An email address becomes a different email address. A phone number becomes a fake one of the right shape.
The AI does its thinking on the fake version, which still reads like normal English. Then we swap the fakes back to the real values before anything is saved or shown to you.
You see: "remind Dana about the Henderson invoice."
The AI did real work. It just never knew who anyone was.
Finding the names is the hard part
Swapping is easy once you know what to swap. The hard bit is finding it.
Emails, phone numbers, account numbers are easy. They have a recognisable pattern, so a simple rule (a "regex") catches them every time.
Names don't have a pattern. "Sarah," "Acme Corp," "Riverton" look like any other word. To find those you need something that actually understands language. That's a thing called NER, Named Entity Recognition: a model whose one job is to read text and point at the proper nouns ("that's a person, that's a company, that's a place").
We run a small, cheap AI for this (Llama, running on Cloudflare's edge, not sent off to anyone). It reads the email and flags the names so we can mask them.
It's not perfect. Small models over-flag. Ours would sometimes decide the word "you," or the verb "make," or even the year "2026" was a person's name and mask it, which scrambles the text. So we had to add a sanity-check layer on top that throws out the obvious nonsense before masking.
The mistake that taught me the most
Here's the one I'd warn anyone else about.
Early on, I didn't fully trust the masking to catch everything. So I added a second AI step as a safety net: after masking, hand the whole document to the small model and say "rewrite this, applying these swaps, and clean up anything I missed."
Sounds sensible. It was a disaster.
The problem: when you ask a generative AI to rewrite a document, it doesn't just swap words. It rewrites. It paraphrases. It merges things. In one real case it took two completely unrelated items and fused them into one made-up company that never existed. And because the app saves some of that output and reuses it later, the corruption built up a little more every day.
I was using a "make new text" tool to do a "find and replace" job. Those are different jobs.
The fix was to delete that step entirely. I made the fake names single, unbreakable words (no spaces in them), so swapping them back is a dead-simple, exact find-and-replace that a generative model never touches. Now no AI ever rewrites the masked text. The AI is only used for the one thing it's genuinely good at: finding the names. The swapping and un-swapping is boring, deterministic code that can't get creative.
What I actually rely on now
Two rules, learned the hard way:
- Fail closed. If the name-finding step can't run properly, I do not send the email to the cloud and cross my fingers. I hold it and try again later. "Probably clean" is not a privacy promise.
- Assume you'll miss one, and watch for it. I run a simple scanner that checks everything I'm about to save or show for any fake name that should have been swapped back. If one slips through, it alerts me immediately, instead of me finding out when a user spots a stranger's name in their brief. (Crucially, this scanner is dumb string-matching, not another AI, so it can't introduce its own mess.)
That last one matters most. You can never prove masking is perfect. So you build the thing that tells you the second it isn't, and you treat every alert as real.
Still a work in progress. But the bar feels right: the AI does the thinking, and it never learns who you are.
Ariadot is in private beta. If "an AI reads my inbox" makes you nervous, good. It should, and that's exactly the part I obsess over. Request access.