Privacy Research Engineer Safeguards

Privacy Research Engineer, Safeguards

When to Use

• Frame privacy research questions for safeguard and moderation stacks
• Design PII detection/redaction benchmarks — precision/recall, re-identification risk
• Evaluate de-identification techniques (mask, tokenize, synthetic replace) on realistic prompts
• Study memorization and extraction — can models or logs leak user content?
• Curate privacy-sensitive datasets — synthetic data, consent boundaries, labeling rules
• Run ablations on detector architecture, threshold, or post-processing
• Define logging minimization — what safety systems may store vs must discard
• Write research memos with privacy–utility trade-offs and production recommendations
• Specify promotion criteria for privacy mitigations before prod rollout

When NOT to Use

• Audit evidence pipelines for GDPR/SOC 2 attestations → compliance-engineer
• Legal DPIA, acceptable-use policy, regulatory mapping → ai-risk-governance
• Harm categories, jailbreak benchmarks, toxic classifiers → ml-research-engineer-safeguards
• Deploy gateways, canaries, safety-path SLOs → ml-infrastructure-engineer-safeguards
• Red-team attack campaigns → ai-redteam
• Enterprise data governance architecture → data-architect
• Human-data platform product ethics (contributor labor) → product-management-human-data-platform
• General literature review unrelated to privacy in ML → ai-researcher

Related skills

Core Workflows

1. Privacy research framing

Threat model, metrics, baselines.

See `references/privacy_research_framing.md`.

2. PII detection and redaction research

Detectors, redaction quality, evals.

See `references/pii_detection_redaction_research.md`.

3. Memorization and extraction

Leakage studies, attack surfaces.

See `references/memorization_and_extraction.md`.

4. Privacy benchmarks and datasets

Corpora, labeling, versioning.

See `references/privacy_benchmarks_datasets.md`.

5. Logging and retention minimization

Safety observability without over-collection.

See `references/logging_retention_minimization.md`.

6. Handoff to production

Promotion bar, monitoring hooks.

See `references/privacy_to_production_handoff.md`.

Outputs

• Threat model — assets, adversaries, failure modes for privacy in safeguards
• Benchmark spec — PII types, locales, adversarial variants
• Results table — detection/redaction metrics by slice (language, format)
• Leakage study report — methodology, findings, confidence
• Logging policy draft — fields allowed, TTL, access controls (engineering input to legal)
• Promotion recommendation — go/no-go with privacy–utility summary

Principles

• Minimize data — collect and retain only what eval and ops truly need
• Separate privacy from safety metrics — low PII leak rate is not interchangeable with low toxicity FN
• Locale and format matter — email in one language ≠ global PII detector
• Synthetic ≠ risk-free — synthetic PII can still encode patterns; document limits
• Legal review for human data — research plans involving real user content need governance sign-off

Logging and retention minimization

Table of contents

1. Data minimization principles 2. Safety pipeline fields 3. Retention tiers 4. Research observability

Data minimization principles

• Log decision codes and scores, not raw user text, when policy allows
• Hash or tokenize stable identifiers for correlation
• Separate security incident retention from product analytics
• Default TTL; no infinite raw prompt stores for safety tuning

Engineering draft ≠ legal retention schedule — align with compliance-engineer and counsel.

Safety pipeline fields

Review each new log field in privacy review checklist before prod.

Retention tiers

Automate deletion jobs; verify with sampling audit.

Research observability

Researchers need signal without prod dumps:

• Stratified samples via privacy-preserving export pipeline
• Differential privacy on aggregate block rates (if used, document epsilon)
• Synthetic replay environments mirroring format but not content

Document what researchers may not do (paste prod logs in external tools).

Memorization and extraction

Table of contents

1. Attack surfaces 2. Study designs 3. Mitigations research 4. Ethics

Attack surfaces

Study designs

• Canary strings — unique tokens inserted in train; probe generation frequency
• Membership inference — research-only; document limitations and ethics
• Prompt repetition — ask model to continue known prefixes from suspected leaks
• Log replay — verify redaction before indexing for analytics

Use synthetic canaries when possible; prod canaries only with governance approval.

Mitigations research

Evaluate:

• Differential privacy training (utility cost curve)
• Data deduplication and near-duplicate removal before train
• Train only on redacted corpora
• Output filters blocking high-entropy repeated n-grams
• Tenant isolation in retrieval and fine-tune pipelines

Report measurable utility delta on primary safety metrics.

Ethics

• No publishing raw leaked strings externally
• Aggregate findings in memos
• Stop study if unintended real PII discovered — incident path via ai-lead-ops

Pair offensive testing of extraction with ai-redteam scope clarity — research measures, red team attacks prod.

PII detection and redaction research

Table of contents

1. Entity taxonomy 2. Detection approaches 3. Redaction strategies 4. Evaluation protocol

Entity taxonomy

Align types with policy and locale:

• Direct identifiers: email, phone, government ID, payment card
• Quasi-identifiers: name, address, employer, rare combinations
• Special categories (jurisdiction-dependent): health, biometric references in text
• Contextual: username, session token, internal employee ID

Document overlap with harm categories (e.g. self-harm content is not PII).

Detection approaches {#detection-approaches}

Redaction strategies

Measure downstream harm classifier change after redaction — privacy must not silently break safety.

Evaluation protocol

1. Frozen benchmark version with human-adjudicated spans 2. Report per-type P/R and macro averages 3. Hard negative set (benign numbers, product IDs) 4. Adversarial formats: obfuscated email, split across messages, homoglyphs 5. Error analysis clusters for next labeling round

Hand promotion thresholds to ml-research-engineer-safeguards when shared classifier stack.

Privacy benchmarks and datasets

Table of contents

1. Dataset sources 2. Synthetic data 3. Labeling 4. Versioning and access

Dataset sources

Never merge prod dumps into train without de-ID pipeline sign-off.

Synthetic data

Pros: no real subjects. Cons: unrealistic distribution, missed edge formats.

• Vary formats (international phone, unicode names)
• Inject hard negatives (order numbers, public figures)
• Document generator version in dataset card

Labeling

• Span-level PII labels with guidelines per type
• Ambiguous bucket — do not force wrong certainty
• Inter-annotator agreement on subset
• Annotator access: VPN, no local export, DLP where required

Versioning and access

Dataset card:

• dataset_id, version, languages
• Provenance and license
• Known gaps (e.g. no APAC addresses)
• Access group (research vs production)

Rotate credentials; audit downloads.

For enterprise lineage standards → data-architect.

Privacy research framing

Table of contents

1. Threat model 2. Research questions 3. Metrics 4. Baselines

Threat model

Adversaries: insider, compromised log store, cross-tenant bug, model inversion (research threat).

Coordinate legal/processing basis with ai-risk-governance before using production data.

Research questions

Examples:

• Does redaction before logging reduce FN on downstream harm detection?
• What false negative rate on phone/email across top 10 locales at FP budget X?
• After fine-tune on safety data, does the model regurgitate rare strings from training?
• Can aggregated safety metrics be published without re-identifying users?

Pre-register primary metric and holdout set.

Metrics

Baselines

Compare:

1. Production detector/redaction stack 2. Regex/heuristic baseline 3. Vendor API (document data handling terms) 4. Previous research champion

Report privacy–utility Pareto — not accuracy alone.

Privacy handoff to production

Table of contents

1. Promotion criteria 2. Deliverables 3. Monitoring in prod 4. Incident triggers

Promotion criteria

Minimum before enabling new PII/redaction stack:

Sign-off: privacy research + ml-research-engineer-safeguards if shared model + governance for high-risk tiers.

Deliverables

Package for ml-infrastructure-engineer-safeguards:

• Detector version, threshold table per locale/type
• Redaction mode configuration
• Prohibited logging flags
• Runbook: false negative PII escalation
• Eval report JSON attached to release ticket

Monitoring in prod

Ongoing metrics (aggregated):

• PII detector score distribution drift
• Manual sample audit rate (reviewer findings)
• Privacy-related tickets / escalations
• Failed redaction jobs

Alert on spike in raw text log volume — config regression.

Incident triggers

Immediate review if:

• Suspected cross-tenant data in safety queue
• Bulk export of reviewer data
• Model outputs repeating unique user strings
• Public dataset release containing prompts

Route to ai-lead-ops incident process; preserve forensics per legal hold guidance.