OpenAI's privacy-filter PII masking
model (1.5B-param MoE, 50M active) reimplemented as one C file, tuned for Apple
Silicon. No dependencies beyond libm; weights are mmap'd bf16 straight out of
model.safetensors.
$ ./pf "Call Maya Chen at +1 (415) 555-0124 or maya@example.com"
Call <PRIVATE_PERSON> at <PRIVATE_PHONE> or <PRIVATE_EMAIL>
Output is byte-identical to the Python reference (opf --device cpu) on every
input tested — including windowed long inputs — at ~1,650 tok/s warm on an M2
Air (~65× the reference, ~166 GFLOPS sustained).
- o200k_base tokenizer: hand-compiled scanner for the pretokenizer regex
(Unicode tables generated by
gen_unicode.py), rank-based BPE, bit-exact with tiktoken. - Forward pass: 8 pre-norm blocks; GQA (14Q/2KV) with interleaved-pair YaRN RoPE; ±128 banded bidirectional attention with log2-space sink logits; 128-expert top-4 MoE with gpt-oss SwiGLU (clamps, +1 linear bias).
- Kernels: weights stay bf16 in the mmap;
BFDOT(ARMv8.6 bf16 dot product) GEMM with pair-interleaved k-tiles reused across 16-token blocks; expert work split into(expert, ≤16 tokens)chunks dispatched via libdispatch to dodge router skew; NEON exp/sigmoid; parallel page pre-touch. - Decoding: fp32 log-softmax → constrained BIOES Viterbi (33 states, the
shipped calibration biases are all zero) → span merge, UTF-8 snap,
whitespace trim, per-label overlap discard →
<LABEL>masking.
Same machine (MacBook Air M2, 8 cores, 24GB), same checkpoint, both on CPU,
byte-identical outputs. "Reference" is the opf CLI / Python API from
openai/privacy-filter at default
settings (PyTorch CPU path, no torch.compile); pf timings include its
mmap + page pre-touch load unless noted.
| Workload | reference opf |
pf |
speedup |
|---|---|---|---|
| One email (107 tok), end-to-end | 7.7 s | 1.0 s | ~8× |
| One 2,821-token doc, end-to-end | 80.5 s | 3.0 s | ~27× |
| Same doc, inference only | ~73 s (39 tok/s) | 1.9 s (1,659 tok/s) | ~38× |
| 144 short docs, one API call each (model preloaded) | 252.7 s (0.6 docs/s) | — | |
156 short docs, pf --lines (one packed forward), end-to-end |
— | 3.8 s (41 docs/s) | ~70× |
| Short docs via HTTP server, 32 concurrent clients | n/a (no server) | 55 docs/s |
Notes: the reference has no batch or server mode, so the many-docs rows
compare its natural usage (one call per document, ~1.75 s each of which is
per-call overhead) against pf's. Load alone: reference ≈ 6.5 s
(torch import + module build) vs pf ≈ 0.25 s warm page cache / ~3 s cold.
Forward-only throughput on long inputs is ~1,400–1,550 tok/s (~166 GFLOPS
sustained; the MoE kernel runs at ~86 of a measured ~91 GMAC/s practical
ceiling). Fanless M2 thermals add ±5% on sustained runs. The precision story:
pf accumulates in fp32 over the same bf16 weights and rounds activations to
bf16 at matmul inputs exactly where the reference does — Viterbi label
streams, and therefore outputs, matched the reference on every input tested.
make # clang -O3 -mcpu=native; needs Apple Silicon for the fast path
mkdir -p model && cd model
curl -LO https://huggingface.co/openai/privacy-filter/resolve/main/original/config.json
curl -LO https://huggingface.co/openai/privacy-filter/resolve/main/original/model.safetensors
python3 -c "import tiktoken,base64; e=tiktoken.get_encoding('o200k_base'); \
open('o200k_base.tiktoken','wb').writelines(base64.b64encode(t)+b' '+str(r).encode()+b'\n' \
for t,r in sorted(e._mergeable_ranks.items(),key=lambda kv:kv[1]))"./pf "text" # mask PII, print to stdout
./pf -f file # from file ("-" = stdin), or pipe to stdin
./pf -j "text" # JSON spans (byte offsets)
./pf --lines -f file # batch mode: one document per input line, one output line each
./pf --batch N # token budget per packed batch (default 16384, min = ctx)
./pf --argmax # per-token argmax instead of Viterbi
./pf --ctx N # context window (default 4096, matching reference CPU)
./pf --stats # timing breakdown on stderr
PF_MODEL=/path/to/model # checkpoint dir (default ./model)
Batching packs many documents (and the windows of long documents) into one
flat token stream with per-segment attention bounds and RoPE positions — the
banded attention makes this exact, not an approximation — so a pile of short
docs runs one big forward instead of many small ones. Weights are pre-touched
in parallel and pinned with best-effort mlock at load.
./pf --serve 8080 [--batch N] [--ctx N]
Loads once, pins weights, then answers on localhost. Each route is an adapter: a JSON-path pattern set naming which string values carry user text. The response is the request payload, unchanged except those values are masked — so it drops in front of an LLM API call as a sanitizing proxy step.
GET /health -> {"status":"ok"}
POST /redact -> masks "text" and "texts[*]"
POST /redact/openai -> masks messages[*].content (string or parts[].text)
POST /redact/anthropic -> masks system / system[*].text / messages[*].content
curl -s localhost:8080/redact/anthropic -d '{
"model":"claude-fable-5",
"messages":[{"role":"user","content":"my api key is sk-live-99fKq2"}]}'
# -> ... "content":"my api key is <SECRET>" ...All text fields in one request are masked in a single packed forward, and
concurrent requests are dynamically batched: connection threads tokenize
and enqueue, a batcher thread drains the queue after a linger window
(--linger MS, default 8) and runs one forward over everything in flight.
At 32 concurrent single-doc clients this roughly doubles throughput over
sequential handling (~29 → ~55 docs/s on an M2 Air) with identical outputs;
solo-request latency pays only the linger. Adding a format = one row in the
ADAPTERS table (route + path patterns).
One small kernel interface, three implementations, chosen at compile time:
- NEON + BFDOT — Apple Silicon and AWS Graviton 3/4 (
-mcpu=native) - AVX2 + FMA — Intel / AMD (
-mavx2 -mfma), same shift-widen bf16 trick - scalar C — anything else
All three produce byte-identical output (the AVX2 and scalar builds are
verified under Rosetta 2). Threading uses libdispatch on macOS and a built-in
pthread pool elsewhere; no other dependencies. Linux build:
cc -O3 -std=c11 -march=native -pthread -o pf pf.c -lm
--skip-beta B enables dynamic expert skipping (Lu et al. 2024): expert
slots whose routing weight is below B x the top slot's are dropped and the
rest renormalized. Measured on this model: it is not output-preserving at
any useful threshold (even B=0.02 flips ~8% of dense-PII docs, mostly span
boundaries on near-tie tokens; B=0.6 is ~30% faster and flips ~8% of short
docs). Default off; treat it as an accuracy/speed dial and evaluate on your
own corpus before enabling.
Categories: account_number, private_address, private_date,
private_email, private_person, private_phone, private_url, secret.
Not an anonymization guarantee — see OpenAI's model card for limitations. Apache 2.0, same as the model.