VAGUE-Gate: A Plug-and-Play Local-Privacy Shield for Retrieval-Augmented Generation

When an AI assistant pulls facts from a private database to answer a question, it frequently copies confidential details into its answer word for word. We built a filter that sits between the database and the model and rephrases sensitive snippets into vague, general facts before the model ever sees them.

The usual alternatives are worse: retrain the model from scratch, or blindly delete every name and date and ruin the answer.

Why this was hard

Retrieved notes weave ordinary public facts and identifying private details into the same sentence. Existing local privacy tools use blunt entity-masking rules that censor public context and private records alike, costing up to 18 points of answer relevance on Llama-3.1-70B. Adding character-level noise turns clear phrases into gibberish. Prompt-level redaction fails outright, because models deduce hidden identities from the role descriptions and timeline patterns that survive. A workable defence has to rewrite sensitive facts with a mathematical guarantee attached, without hurting fluency or touching the retrieval system itself.

How it works

VAGUE-Gate is a client-side module that applies local differential privacy in two stages and retrains nothing. We first parse retrieved text into atomic facts with an ensemble of spaCy and Flair, pinpointing identifiers, locations and medical conditions.

In stage one, a deterministic pass at temperature zero evaluates each atomic fact against a chosen privacy budget \(\epsilon \in [0.1, 1.0]\). It drops or broadens sensitive items with probability \(p_{drop}(d;\epsilon) = 1 - \epsilon \cdot u(d)\), where \(u(d)\) is a deterministic utility weight. At \(\epsilon = 0.3\) this drops or abstracts specific facts with 70% probability while leaving public sentences intact.

Stage two applies deep-obfuscation passes with a repetition budget \(k(\epsilon) = \lceil k_{max}(1-\epsilon) \rceil\), with \(k_{max} = 4\). At low \(\epsilon\) the module triggers up to four successive high-temperature paraphrases at \(T = 0.7\), blurring clause structure and erasing residual stylistic clues. Under the post-processing property of differential privacy, those extra rounds cannot expand the budget beyond the original \(\epsilon\) bound, and we prove the composite mechanism satisfies strict \(\epsilon\)-local differential privacy for any chosen budget.

To test it under realistic conditions we built BlendPriv, a 3,000 question testbed across ten domains, annotated sentence by sentence into public, sensitive and confidential tiers. Across 32 model combinations spanning GPT-4o-mini, Llama-3.1-70B, DeepSeek-V3 and Qwen3-235B, the gate at \(\epsilon = 0.3\) cut lexical leakage by 70% and semantic leakage by 1.8 points on a 5-point scale, while retaining 91% of plain retrieval faithfulness for 240ms of added latency.

What it does not do

The gate cannot stop semantic leakage when a person holds an entirely unique role, such as a specialised clinic coordinator at one regional branch. Fluent paraphrase still leaves structural fingerprints, and a determined adversary can work back to the identity. At very tight budgets of \(\epsilon = 0.1\), dropping numeric thresholds turns precise temporal and clinical figures into generic approximations, and answer relevance falls noticeably on complex domain prompts.

Published at IJCNLP-AACL 2025. Code is on GitHub.