SEC: canonical detector catches glued-prefix credentials at egress sinks

[thesongzhu]

Defect (SEC-SECRET-GLUED-PREFIX-001 — P1 correction)

The canonical secret detector findSecretShapeSpans / redactSecretShapesInString
(src/security/friday-secret-shape-redactor.ts) kept a leading \b on the WHOLE
GitHub-classic and AWS alternations. That protected the benign word/base32 fragments
in those alternations — but it over-protected the SAFE members, so a real
GitHub-classic PAT and AWS access-key glued after a word char were left UNMATCHED:

• keyghp_<36 base62> and keyAKIA<16> (no boundary before the prefix) survived
  verbatim through the detector → realtime redactor → memory output filter →
  persisted audit JSONL.

An "accepted gap" for a real credential prefix does not satisfy
SEC-EVENT-REDACTION-001 (the canary must be ABSENT from every raw sink readback). The
prior PR body also mis-described these two families as already "de-\b'd", which
they were not — corrected below.

All egress sinks funnel through this single canonical detector — audit-read
(friday-hub-audit-log-writer.ts), realtime on-wire
(friday-event-payload-redactor.ts), memory read-back
(friday-memory-output-filter.ts + friday-memory-pii-guard.ts), plus realtime
persistence/bus, legacy rewrite, WebSocket egress, execution-control audit, and
agent-run persistence — so detector + those 3 sinks + the audit-JSONL cross-sink parity
proof are representative of the whole egress surface.

Fix — split each alternation by prefix; catch a glued credential ONLY on the provably-unambiguous members

Each member of the GitHub-classic set {gho,ghp,ghr,ghs,ghu} and the AWS set
{AKIA,ASIA,AGPA,AIDA,AROA,AIPA,ANPA,ANVA} is classified with three criteria:
(a) English word-ending (grep /usr/share/dict/words) or a common proper-noun,
(b) Crockford-base32 / ULID-constructible (ULID charset = 0-9 A-Z minus I,L,O,U;
a member of only Crockford chars can sit mid-ULID), (c) a common all-caps acronym /
base32 fragment. A member catches a glued credential ONLY if it fails all three. The
GitHub body is MIXED-CASE base62 after _, so a lowercase snake_case identifier can
collide → even obscure word-endings count. The AWS body is [0-9A-Z]{16} UPPERCASE+digit,
so a member only collides inside an all-caps / alphanumeric 20+ contiguous run.

Per-member classification

Member	(a) word/proper-noun	(b) ULID	(c) all-caps acronym	Catches glued	Mechanism
ghp	no (0 dict)	— (lowercase)	no	YES ✅	plain de-\b
ghr	no (0 dict)	—	no	YES ✅	plain de-\b
ghs	yes (walkthrou-GHS…)	—	—	no	keep \b
gho	yes (sor-GHO, San-GHO)	—	—	no	keep \b
ghu	yes (Ra-GHU)	—	—	no	keep \b
github_pat_	yes ("github")	—	—	no	keep \b
AKIA	country suffix SLOV-AKIA / CZECHOSLOV-AKIA (all-caps)	no (has I)	no	YES ✅	(?<![A-Z0-9]) lookbehind
ASIA	yes (Eur-ASIA, Austral-ASIA)	no (has I)	yes (AUSTRALASIA/EURASIA)	no	keep \b
AGPA	no	yes (Crockford)	—	no	keep \b
ANPA	no	yes (Crockford)	—	no	keep \b
ANVA	no	yes (Crockford)	yes (C-ANVA-S)	no	keep \b
AIDA	Haida/Maida	no (has I)	yes (AIDA model / opera / PROJECT-AIDA)	no	keep \b
AROA	Aotearoa	no (has O)	yes (AOTE-AROA)	no	keep \b
AIPA	Arivaipa/Caraipa	no (has I)	yes (prefix of AIPAC)	no	keep \b

AKIA — country-word suffix → context lookbehind (round-5 correction). The prior
round claimed AKIA had "no common all-caps word source"; that was wrong — AKIA is the
suffix of SLOVAKIA and CZECHOSLOVAKIA (proper nouns absent from the dict, which
the per-word grep missed). A plain leading-\b drop over-redacts an all-caps region
constant (SLOVAKIAREGIONCODE… → SLOV[REDACTED]). So AKIA is de-bounded with a
context-sensitive negative-lookbehind (?<![A-Z0-9]), NOT a plain \b-drop: it
matches only when AKIA starts a fresh token (after a lowercase word char / symbol /
whitespace / string start — the credential-glue case), not when embedded after an
UPPERCASE letter or DIGIT (mid-all-caps-word SLOVAKIA…, mid-alphanumeric PROJECT2AKIA…).
The lookbehind is zero-width, so the redacted span is still exactly AKIA<16>. ghp/ghr
need no lookbehind — a repo-wide scan confirms neither is a substring of any all-caps word.

Resulting GitHub + AWS patterns

GitHub catch-glued  (de-`\b`):    /gh[pr]_[A-Za-z0-9]{16,}\b/gu
GitHub keep `\b`:                  /\b(?:gh[osu]_[A-Za-z0-9]{16,}|github_pat_[A-Za-z0-9_]{16,})\b/gu
AWS    catch-glued  (lookbehind):  /(?<![A-Z0-9])AKIA[0-9A-Z]{16}\b/gu
AWS    keep `\b`:                   /\bA(?:SIA|GPA|IDA|ROA|IPA|NPA|NVA)[0-9A-Z]{16}\b/gu

Bodies are unchanged (base62 / [0-9A-Z]{16}); the open-ended {16,} is never
tightened to an exact length, so a longer real token cannot slip the trailing \b and
leak. The previously-shipped 9 de-\b'd vendor prefixes (hf_, gsk_, npm_,
dop_v1_, GOCSPX-, glpat-, SG.<22>.<43>, sq0atp-/sq0csp-, Slack xox[abprs]-),
the AWS keep-\b branch, and the kept-\b shapes (sk-/rk-/ak-/xai-, Stripe
underscore, Slack xapp-, AIza, ya29., JWT eyJ, Bearer / assignment) are
UNCHANGED.

Red-first + NO-DEGRADE proof

• RED on 7021926c (base): glued keyghp_<36>, keyghr_<…>, keyAKIA<16> return
  verbatim, 0 spans at the detector and survive all sinks.
• RED on 2c83630f (plain de-\b AKIA): SLOVAKIAREGIONCODE2024AB →
  SLOV[REDACTED_SECRET] (and the free-text embedding) — the country-suffix over-redaction.
• GREEN after: each glued canary → key[REDACTED_SECRET] / aws[REDACTED_SECRET]
  (credential subspan only, benign lead preserved) at the detector, event-payload
  redactor (on-wire), memory output filter (content / metadata / tags legs), and the
  audit-JSONL sink (cross-sink parity: memory egress == audit sink). SLOVAKIA… /
  CZECHOSLOVAKIA… / PROJECT2AKIA… are unchanged through the detector and all 3 sinks.
• AMBIGUOUS members still correct: every {gho,ghs,ghu} / {ASIA,AGPA,AIDA,AROA,AIPA, ANPA,ANVA} prefix standalone/delimited still redacts, but its glued form remains
  an accepted gap (keeps \b).
• NO-DEGRADE benign corpus — byte-identical vs base (0 over-redaction): ghs-words
  (walkthroughs_completedThisWeek, coughs_<hash>, breakthroughs_*), gho/ghu
  (sorgho_yieldPerHectare2024xx, raghu_…), ULID/all-caps (012345AGPA…, AUSTRALASIA…,
  EURASIA…, PROJECTAIDA…), the (c) negative controls (AIPACPOLICYCONFERENCE…,
  AOTEAROANEWZEALAND…, OPENCANVAS…), the AKIA country-suffix controls
  (SLOVAKIAREGIONCODE…, CZECHOSLOVAKIAREGIONCODE…), UUID/hex/base64url — all return
  byte-identical through the detector and the 3 sinks.

Gates

• npx tsc --noEmit = 0 (V8 + tsconfig support the lookbehind + u flag);
  npm run lint = 0 errors.
• Secret-redactor suite + all 3 sink suites + cross-sink parity + broad test/unit/security
  • test/unit/memory/guard/services + test/unit/api/realtime — 60 files / 1167 tests
    green, no .only/.skip.
• detect-secrets-hook v1.5.0 (exact CI command) → no new secrets (new PLACEHOLDER /
  AKIA-shaped fixture literals are seg()-built and/or carry inline pragma: allowlist secret).
• Diff = source + tests only. No migration / lockfile / .secrets.baseline / exclude-glob
  change.

Scope / honesty

Bounded to the leading-\b over-protection on the GitHub-classic and AWS alternations,
plus the AKIA country-suffix (SLOVAKIA/CZECHOSLOVAKIA) NO-DEGRADE follow-up. The prior
"GitHub classic and AWS are de-\b'd", "ghp_ + AKIA glued is an accepted gap", and "AKIA
has no common all-caps word source" claims are all removed. ghp/ghr are plain de-\b'd;
AKIA is de-bounded via a context-sensitive (?<![A-Z0-9]) lookbehind that preserves
SLOVAKIA/CZECHOSLOVAKIA; the word/ULID/acronym members keep \b.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JhCDTLngxAf6MDh4Bfcd48