SurrogateShield Documentation

SurrogateShield ships in two forms:

• Interactive dashboard — a full terminal UI with chat, PII Finder, batch evaluation, and the Attacker Experiment
• pip install surrogateshield — a standalone Python library with a five-function API you can embed in any application

Installation

Requirements

• Python 3.9 or newer
• pip
• A virtual environment (strongly recommended — see below)

Step-by-step

git clone https://github.com/sherwinvishesh/SurrogateShield.git
cd SurrogateShield

# macOS / Linux
python -m venv .venv
source .venv/bin/activate

# Windows
python -m venv .venv
.venv\Scripts\activate

pip install -r requirements.txt

python -m spacy download en_core_web_lg

Full dependency list

Environment Variables

Create a .env file in the project root. You only need the key for the provider you intend to use:

# Required for Claude (default provider)
ANTHROPIC_API_KEY=sk-ant-...

# Required for Gemini
GEMINI_API_KEY=AIzaSy...

# Required for ChatGPT
OPENAI_API_KEY=sk-...

# No key needed for local Ollama

The .env file is listed in .gitignore — it will never be accidentally committed.

Quick Start

# 1. Activate your venv
source .venv/bin/activate

# 2. Add your API key
echo "ANTHROPIC_API_KEY=sk-ant-..." > .env

# 3. Launch the dashboard
./run.sh
# or: python main.py

The launcher script run.sh activates the venv and loads the .env file automatically.

First Run

On the first launch, ContextGuard downloads its model (~250 MB) from HuggingFace Hub. This may take 1–5 minutes depending on your connection. Progress is shown in the terminal. After that, the model is cached at ~/.cache/huggingface/ and loads instantly.

The interactive dashboard will appear. You will see:

• The SurrogateShield ASCII banner
• The pipeline overview (7 stages)
• A list of any saved conversations
• The action menu

Press N to start a new conversation. Type any message containing PII and watch the pipeline work.

Pipeline Overview

Every message you type goes through the following stages before it reaches the LLM. On the way back, surrogates in the response are replaced with your originals before you see anything.

ServiceQueryDetector

Not all location information is equally sensitive. A message like "restaurants near 1126 E Apache Blvd, Tempe, AZ" needs the city and state to give a useful answer — fully replacing them with fake values would destroy the response quality.

ServiceQueryDetector identifies these location queries and applies a proportional response:

• Street addresses: house number is shifted by ±2–8 (max ~100 m geographic error), street name and city are preserved
• City/state names: not replaced — they provide no meaningful re-identification risk in service queries
• Sensitive topic override: if the message contains keywords like medical, legal, shelter, or immigration, full anonymisation is forced regardless of query structure

Address existence is optionally verified via OpenStreetMap Nominatim (disable with SERVICE_QUERY_VERIFY_ADDRESSES = False in config.py for offline use).

SentinelLayer — The Detection Cascade

Three detectors run sequentially. Spans claimed by an earlier stage are not re-processed downstream, preventing double-detection. Each stage is described in detail below.

PatternScan — Stage 1

Regex-based structural detection. Runs first so all structured PII is masked before any NER model sees it. Format validators (Luhn, ABA checksum) reject false positives before any entity is emitted.

ABA routing number checksum

The ABA 9-digit checksum eliminates false positives — random 9-digit numbers almost always fail:

(3·d₀ + 7·d₁ + d₂ + 3·d₃ + 7·d₄ + d₅ + 3·d₆ + 7·d₇ + d₈) mod 10 = 0

Driver's license context gate

The driver's license regex requires a keyword (driver's license, license number, DL, D.L.) within the same phrase. Only the license value itself is marked as an entity — the keyword prefix stays in the sanitised text.

EntityTrace — Stage 2

spaCy en_core_web_lg Named Entity Recognition. Extracts PERSON, GPE, LOC, ORG, and FAC entities. Returns two confidence tiers:

EntityTrace also includes an ORG→GPE reclassifier: when a location preposition appears before an entity name (e.g. "lives in Google"), the classification is corrected from ORG to GPE.

ContextGuard — Stage 3

Transformer-based verifier using dslim/distilbert-NER (~250 MB, downloaded once from HuggingFace Hub on first run, no server required). Two jobs:

1. Confirm borderline candidates forwarded from EntityTrace (requires score ≥ 0.70)
2. Independently detect anything EntityTrace missed

Applies word-piece artefact cleaning and a blocklist of short/title tokens that commonly cause false positives (e.g. "I", "A", "The").

ContextGuard can be disabled via config.py or via the library's shield.config(context_guard_enabled=False) for faster, spaCy-only detection.

Post-processing Passes

Four additional passes run on the combined entity set from all three stages:

MimicGen — Surrogate Generation

Generates type-consistent surrogates using Faker. Each surrogate is realistic for its PII type and unique within the session (tracked via a used_surrogates set to guarantee no collisions).

Example: before and after substitution

ShadowMap — Encrypted Storage

An encrypted, per-conversation mapping of {surrogate → original}. Stored on disk; unreadable without the device secret.

ResolvePass — Response Restoration

Three-pass restoration of original values in LLM responses:

CLI Commands

# Launch the interactive dashboard (recommended)
python main.py
./run.sh

# Start a new chat conversation directly
python main.py chat

# Continue a saved conversation
python main.py chat --load <conversation-id>

# New conversation with RAG enabled
python main.py chat --rag

# List all saved conversations
python main.py list

# PII detection sandbox — no API calls, no credits
python main.py pii-finder

# Index a document into the RAG vector store
python main.py add-doc path/to/document.txt

Dashboard Keyboard Shortcuts

These shortcuts work from the main dashboard screen.

Chat Mode

Chat mode is the primary way to use SurrogateShield. Press N from the dashboard or run python main.py chat.

In each turn:

1. Type your message and press Enter
2. The full detection cascade runs on your message
3. Detected PII is replaced with surrogates
4. The sanitised message is sent to the LLM API
5. The response is returned and surrogates are restored
6. You see the response with your real values

If Detailed View is enabled (Settings → D), each turn also shows:

• Pipeline stage logs
• A per-entity PII detection table
• The API transparency panel (what was sent, what was received, the final restored output)

Type exit or press Ctrl+C to return to the dashboard. Conversations are saved automatically.

PII Finder

The PII Finder is a detection sandbox with zero API calls and zero credits spent. Press P from the dashboard or run python main.py pii-finder.

Type any text to see exactly what SurrogateShield would detect and what it would send to the LLM. The service-query path (address fuzzing) is also active, so you can test how location queries are handled.

Special commands in PII Finder:

• reset — clears the surrogate memory so you can start fresh
• exit — returns to the dashboard

If the Presidio comparison panel is enabled in Settings, the PII Finder also shows a side-by-side comparison with Microsoft Presidio's [ENTITY_TYPE] placeholder redaction below each result.

RAG Mode

RAG (Retrieval-Augmented Generation) lets you index your own documents and have them automatically retrieved as context for your questions. Backed by ChromaDB and all-MiniLM-L6-v2 embeddings — no server required, runs in-process.

Indexing a document

python main.py add-doc path/to/your/document.txt

Documents are anonymised through the full SentinelLayer pipeline before indexing. Real PII never enters the vector store. Surrogate mappings from indexed documents are stored in a shared rag_global ShadowMap.

Starting a RAG conversation

Press R from the dashboard, or run python main.py chat --rag. Retrieved context is prepended to your sanitised message before the LLM call.

Configuration

JSON Batch Test

Batch-process a list of questions through the full pipeline. Press J from the dashboard.

Input format

Create a JSON file in the experiment/ directory:

[
  { "input": "My name is Sherwin and my SSN is 544-87-2944. What are Wyoming's tax benefits?" },
  { "input": "My email is sjathann@asu.edu and phone is 480-555-1234. Draft a resignation letter." }
]

Running

1. Press J in the dashboard
2. Enter the filename (e.g. test for experiment/test.json)
3. Select which output fields to capture (toggle with number keys)
4. Press Enter to run

Output is saved to experiment/<name>_answers.json. Results are flushed every 25 questions — safe to interrupt and resume.

Available output fields

Evaluation Mode

Score a completed answers file against a ground-truth key. Press E from the dashboard.

Ground-truth key format

[
  {
    "Question": "My name is Sherwin and my SSN is 544-87-2944...",
    "Answer-Key": {
      "name": "Sherwin",
      "ssn": "544-87-2944"
    }
  }
]

Save this as experiment/<name>_key.json. The evaluator accepts flexible label names — see the supported labels below.

Supported answer-key labels

Metrics reported

Ablation study

The ablation study quantifies how much each detection stage contributes to overall F1. It is computed post-hoc from the existing answers file — no pipeline re-run needed. Each answer records which stage detected each entity via the source field in pii_detail.

To generate ablation data, enable pattern_scan_pii, entity_trace_pii, context_guard_pii, and confirmed_pii in JSON Test before running (all four are on by default).

Attacker Experiment

Simulates an informed adversary who intercepts the sanitised text SurrogateShield sends to the LLM API and actively tries to recover the original PII values. Press A from the dashboard.

What it tests

Two variants run on each question from an existing answers file:

Expected result

0% recovery for both systems. Surrogates have no cryptographic or statistical relationship to the original values. This proves SurrogateShield achieves equivalent inference resistance to blunt placeholder redaction, while preserving significantly higher semantic utility (as measured by BERTScore).

Running the experiment

Press A in the dashboard. The four-screen flow will:

1. Explain the experiment
2. Prompt for an existing answers file from experiment/ (must contain surrogate_map, sanitized_input, and optionally presidio_sanitized_input / presidio_found_piis)
3. Show question counts and estimated API calls, then run
4. Display a results summary on completion

The experiment supports resume — if interrupted, re-running with the same answers file picks up where it left off.

Output files

Required fields in answers file

PII Types Detected

SurrogateShield detects 20 PII types across three detection mechanisms:

PII Examples

Structured PII (PatternScan)

# SSN
544-87-2944  →  317-42-8806

# Email
sjathann@asu.edu  →  jdoe@example.net

# Phone US
+1-480-555-1234  →  +1-602-555-9871

# Credit card (Luhn-valid)
4111 1111 1111 1111  →  5234 5678 9012 3456

# Bitcoin wallet
1A1zP1eP5QGefi2DMPTfTL5SLmv7Divf  →  1BvBMSEYstWetqTFn5Au4m4GFg7xJaNVN2

# ABA routing number
021000021  →  071000013

# Driver's license (context-gated)
My DL number is B7654321  →  My DL number is F2390481

Named entities (EntityTrace + ContextGuard)

# Person
Sherwin  →  Sarah Mitchell

# City / GPE
Phoenix, AZ  →  Springfield, IL (in sensitive contexts)
# Note: well-known cities are often preserved in service queries

# Organisation
Arizona State University  →  Lakewood Industries

# Facility
Banner Health Hospital  →  Riverside Medical Center

Quasi-Identifier Risk

Based on Sweeney's k-anonymity research. Even without traditional PII, certain combinations of fields can uniquely re-identify individuals. SurrogateShield detects these and warns you before the message is sent.

Warnings appear in the terminal when these combinations are detected. Enable Detailed View (Settings → D) to always show quasi-identifier analysis.

Runtime Settings

Changed interactively from inside the app (press S → Settings). Persisted across sessions in ~/.surrogateshield/settings.json.

Advanced Configuration (config.py)

Hard-coded thresholds and flags. Edit the file directly to change them. No restart required for PII Finder; restart required for chat sessions.

LLM Providers

Switch providers from the Settings menu (press S inside the dashboard).

Enabling Presidio comparison

The Presidio comparison panel is off by default. To enable it:

1. Activate your venv, then install: pip install presidio-analyzer presidio-anonymizer
2. Download the spaCy model (if not already done): python -m spacy download en_core_web_lg
3. Open the app and press S → C (Settings → Presidio Comparison) to toggle it on

Python Library — Installation

SurrogateShield is also available as a standalone pip package. No dashboard, no CLI — just the core detection and masking pipeline as a clean Python API.

pip install surrogateshield

Both the spaCy model (en_core_web_lg) and the ContextGuard transformer model (dslim/distilbert-NER) install automatically — no separate download step needed.

Python Library — Quick Start

import surrogateshield as shield

# Silence the default Rich tables (recommended for production)
shield.config(detailed_view=False)

user_text = (
    "Hi, I'm Sarah Mitchell. My email is sarah.mitchell@gmail.com, "
    "my SSN is 123-45-6789, and I was born on 04/12/1990."
)

sanitized = shield.mask(user_text)
# "Hi, I'm Rachel Torres. My email is torresrachel@yahoo.com,
#  my SSN is 876-32-1045, and I was born on 09/27/1983."

response = any_llm.chat(sanitized)   # send surrogates, not real PII
restored = shield.unmask(response)   # real values restored from the session map
shield.flush()                       # clear session before the next conversation

Python Library — Public API

scan() — Inspect Without Masking

scan() runs the full detection cascade and returns a {value: type} dict, but it never modifies the session or updates the ShadowMap. Safe to call in a read-only context.

found = shield.scan(
    "Contact Alice Nguyen at alice@corp.com or call +1-415-555-0198."
)
# {"Alice Nguyen": "PERSON", "alice@corp.com": "email", "+1-415-555-0198": "phone_us"}

for value, pii_type in found.items():
    print(f"{pii_type:15s}  {value}")

scan() always returns every detected entity regardless of pii_off settings.

pii_off — Suppress Specific Types

Sometimes you want to detect certain PII types but not replace them. Use pii_off to suppress substitution for specific types while still detecting them.

shield.config(pii_off=["location", "org"])

sanitized = shield.mask(
    "Emma Johnson works at Deloitte in New York, email emma@deloitte.com."
)
# "Emma Johnson"      → replaced (PERSON not in pii_off)
# "emma@deloitte.com" → replaced (email not in pii_off)
# "Deloitte"          → kept  (org in pii_off)
# "New York"          → kept  (location in pii_off)

Available aliases for pii_off

phone, name, location, org, email, ssn, dob, address, zip, postcode, postal_code, credit_card, ip_address, api_key, crypto, bank, license, gender_indicator

Raw type strings ("PERSON", "GPE", etc.) also accepted.

Persistent ShadowMap

By default the session mapping is in-memory only. For web servers or multi-process deployments, point pii_mem at a directory and the map is encrypted to disk with AES-256-GCM.

import os, surrogateshield as shield

os.makedirs("/var/app/shadowmaps", exist_ok=True)
shield.config(pii_mem="/var/app/shadowmaps")

sanitized = shield.mask("My name is Clara Oswald, phone 555-123-4567.")
restored  = shield.unmask(llm.chat(sanitized))
shield.flush()   # deletes the .shadowmap and .key files for this session

When pii_mem is a directory path, each session gets its own .shadowmap file encrypted with AES-256-GCM. shield.flush() deletes the files and generates a new session ID.

Provider Examples

unmask() accepts native SDK response objects directly — no manual text extraction needed.

Anthropic / Claude

import anthropic
import surrogateshield as shield

client = anthropic.Anthropic()
shield.config(detailed_view=False)

sanitized = shield.mask(user_text)
response = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    messages=[{"role": "user", "content": sanitized}]
)
print(shield.unmask(response))   # passes Anthropic Message object directly
shield.flush()

OpenAI / ChatGPT

from openai import OpenAI
import surrogateshield as shield

client = OpenAI()
shield.config(detailed_view=False)

sanitized = shield.mask(user_text)
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": sanitized}]
)
print(shield.unmask(response))   # passes ChatCompletion object directly
shield.flush()

Google Gemini

import google.generativeai as genai
import surrogateshield as shield

model = genai.GenerativeModel("gemini-1.5-flash")
shield.config(detailed_view=False)

sanitized = shield.mask(user_text)
response = model.generate_content(sanitized)
print(shield.unmask(response))   # passes GenerateContentResponse directly
shield.flush()

Ollama / plain string

import surrogateshield as shield

shield.config(detailed_view=False)
sanitized = shield.mask(user_text)
raw_reply = ask_ollama(sanitized)      # plain string
print(shield.unmask(raw_reply))        # plain strings work too
shield.flush()

Library Config Parameters

Security Design

Privacy Guarantees

• No PII crosses the API boundary. Every entity confirmed by SentinelLayer is replaced before the HTTP request is made.
• Service queries get proportional protection. A restaurant search near your home address gets the house number shifted; the city name is preserved so the answer is useful. Sensitive topic overrides (medical, legal, shelter) force full anonymisation regardless.
• Geographic generality is preserved. US states, countries, and major cities are never replaced — they provide no meaningful re-identification risk and destroying them would break answer quality.
• Quasi-identifier risks are surfaced. If your message contains combinations like ZIP+DOB+gender that are statistically re-identifying even without traditional PII, you are warned before the message is sent.
• RAG documents are anonymised at index time. Real PII never enters the vector store. Retrieval and context injection all operate on surrogates.
• Financial and identity credentials are protected. Bitcoin/Ethereum wallet addresses, ABA routing numbers, and driver's license numbers are detected and replaced alongside traditional PII.

Project Structure

SurrogateShield/
├── main.py                  # CLI entry point and interactive dashboard
├── pipeline.py              # End-to-end message pipeline orchestration
├── config.py                # All constants and thresholds (single source of truth)
├── util.py                  # Shared dataclasses (DetectedEntity, Conversation)
├── settings_manager.py      # Persistent user settings (~/.surrogateshield/settings.json)
├── evaluator.py             # Precision/recall/F1 evaluation logic + Presidio/BERTScore/ablation
├── json_tester.py           # Batch JSON question processing
├── attacker.py              # Adversarial PII recovery experiment
├── run.sh                   # Launcher script (venv activation, .env loading)
│
├── detection/               # SentinelLayer — three-stage PII detection cascade
│   ├── logic.py             # Cascade orchestration + post-processing passes A–D
│   ├── pattern_scan.py      # PatternScan — regex-based structured PII detection
│   ├── entity_trace.py      # EntityTrace — spaCy NER (en_core_web_lg)
│   ├── context_guard.py     # ContextGuard — distilbert-NER (dslim/distilbert-NER)
│   ├── service_query.py     # ServiceQueryDetector — address fuzzing for location queries
│   ├── quasi_identifier.py  # Quasi-identifier combination risk scorer (k-anonymity)
│   └── geo_data.py          # Geographic pass-through whitelist (US states, countries)
│
├── generation/
│   └── logic.py             # MimicGen — type-consistent surrogate generation (Faker)
│
├── storage/
│   └── logic.py             # ShadowMap — AES-256-GCM encrypted surrogate mapping store
│
├── reconstruction/
│   └── logic.py             # ResolvePass — three-pass surrogate→original restoration
│
├── presidio/                # Presidio integration layer (optional comparison feature)
│   ├── engine.py            # Lazy singleton AnalyzerEngine wrapper
│   ├── detect.py            # detect(text) → list[PresidioEntity]
│   └── redact.py            # redact(text, entities) → [TYPE]-placeholder string
│
├── chatbot/
│   ├── chat.py              # Multi-provider LLM conversation handler
│   └── rag.py               # RAGStore — ChromaDB + sentence-transformers vector store
│
├── experiment/              # Batch test input/output files
│   ├── example.json         # Sample question file
│   ├── example_key.json     # Sample ground-truth answer key
│   └── example_answers.json # Sample output from JSON Test
│
├── tests/                   # Test suite (no API key required)
│   ├── test1.py             # PatternScan, cascade, MimicGen, ShadowMap, ResolvePass
│   ├── test2.py             # Additional detection and generation tests
│   ├── test3.py             # Additional pipeline and storage tests
│   ├── test6.py             # Attacker Experiment — all mocked, no API key needed
│   └── test7.py             # Python library — 134 checks, no API key required
│
└── python-library/          # Standalone pip package (surrogateshield)
    ├── pyproject.toml
    └── surrogateshield/     # Public API: config, scan, mask, unmask, flush

Running Tests

# Activate your venv first
source .venv/bin/activate

# Core pipeline tests (no API key required)
python tests/test1.py
python tests/test2.py
python tests/test3.py

# Attacker Experiment tests (all API calls mocked, no API key required)
python tests/test6.py

# Python library tests (run from repo root)
python tests/test7.py

Troubleshooting

Presidio shows "not installed"

This almost always means the packages were installed into a different Python than the one running the app. Fix:

# Activate your venv first, then:
pip install presidio-analyzer presidio-anonymizer
python -m spacy download en_core_web_lg

"Package not found" at runtime

You ran pip install without activating the venv first. The packages went into the system Python. Fix:

source .venv/bin/activate   # activate venv
pip install -r requirements.txt   # reinstall into venv

ContextGuard model download is slow

The dslim/distilbert-NER model is ~250 MB. On a slow connection, the first download can take 5–15 minutes. Progress is shown in the terminal. Once downloaded, it is cached at ~/.cache/huggingface/.

BERTScore takes very long

BERTScore uses roberta-large (~1.4 GB). On CPU, scoring a batch of 50 questions can take 15–30 minutes. The model is only downloaded if you enable BERTScore fields in JSON Test. If you have a GPU, it will be used automatically.

Address verification fails in offline environments

ServiceQueryDetector optionally verifies fuzzed addresses via OpenStreetMap Nominatim. In offline environments, disable this:

# In config.py
SERVICE_QUERY_VERIFY_ADDRESSES = False

Conversations are not found after moving the project

Conversations are stored in conversations/ relative to the project root. If you move the project directory, the existing .shadowmap files are no longer decryptable with the device key at ~/.surrogateshield/device.key — but the device key itself stays in place, so new conversations in the new location will work normally.

spaCy model not found

# With venv active:
python -m spacy download en_core_web_lg