Skip to content

fix: guard unsupported Presidio entities#3085

Open
vishalg0wda wants to merge 7 commits into
mainfrom
vishal/age-2576-bug-audit-all-advertised-risk-analysis-detection-rules-TXqb
Open

fix: guard unsupported Presidio entities#3085
vishalg0wda wants to merge 7 commits into
mainfrom
vishal/age-2576-bug-audit-all-advertised-risk-analysis-detection-rules-TXqb

Conversation

@vishalg0wda
Copy link
Copy Markdown
Member

@vishalg0wda vishalg0wda commented May 28, 2026

Summary

  • add a server-side Presidio allowlist so unsupported entities are dropped before scan requests
  • shrink the Presidio support map to the empirically verified runtime set from the live 2.2.362 container
  • simplify the frontend rule catalog by removing unsupported Presidio rules in-place instead of carrying available flags and special-case UI logic
  • revert the recent presidiotest sample churn so tests stay focused on client behavior rather than mirroring runtime probes

Validation

  • rtk go test ./server/internal/background/activities/risk_analysis/...
  • cd client/dashboard && npx tsc --noEmit
  • probed live Presidio API/logs on http://127.0.0.1:5050

Notes

  • this is still a shrink, not an allowlist expansion: DOMAIN_NAME, SG_NRIC_FIN, and MEDICAL_LICENSE remain removed from the supported production set
  • cd client/dashboard && pnpm build still fails on a pre-existing unresolved import from client/dashboard/src/contexts/Sdk.tsx
  • the isolated Presidio lab experiment lives in separate draft PR wip: isolated presidio lab experiment #3101
  • production healthcare/image work is intentionally not included in this PR

@vercel
Copy link
Copy Markdown

vercel Bot commented May 28, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
gram-docs-redirect Ready Ready Preview, Comment May 29, 2026 12:58pm

Request Review

@linear-code
Copy link
Copy Markdown

linear-code Bot commented May 28, 2026

AGE-2576

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented May 28, 2026

⚠️ No Changeset found

Latest commit: 13fd2f4

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 28, 2026

Prompt-injection risk report

Corpus: 933 cases (384 malicious / 549 benign)
This PR: 3e7dfe8c · 2026-05-29T12:59:53Z

No main baseline artifact found yet; this comment shows the current run only.

Operational Modes

mode status total TP FP FN precision recall F1 FP-rate
L0 only ok 933 76 0 308 1 0.1979 0.3304 0
L0 + L1 opt-in skipped - - - - - - - -

L1 opt-in was not evaluated in this run: classifier URL is not set.

Generated by .github/scripts/risk-metrics-comment.py. Full source/rule breakdown and samples are in the risk-accuracy-metrics artifact.

@github-actions github-actions Bot added the preview Spawn a preview environment label May 28, 2026
@speakeasybot
Copy link
Copy Markdown
Collaborator

speakeasybot commented May 28, 2026

🚀 Preview Environment (PR #3085)

Preview URL: https://pr-3085.dev.getgram.ai

Component Status Details Updated (UTC)
✅ Database Ready Existing database reused 2026-05-29 13:05:33.
✅ Images Available Container images ready 2026-05-29 13:05:15.

Gram Preview Bot

@vishalg0wda vishalg0wda changed the title fix: guard unsupported Presidio entities fix: guard unsupported Presidio entities and add presidio lab May 28, 2026
@vishalg0wda vishalg0wda force-pushed the vishal/age-2576-bug-audit-all-advertised-risk-analysis-detection-rules-TXqb branch from da8dd4b to 6c174c2 Compare May 29, 2026 10:31
@vishalg0wda vishalg0wda changed the title fix: guard unsupported Presidio entities and add presidio lab fix: guard unsupported Presidio entities May 29, 2026
@vishalg0wda vishalg0wda marked this pull request as ready for review May 29, 2026 11:31
@vishalg0wda vishalg0wda requested review from a team as code owners May 29, 2026 11:31
Copy link
Copy Markdown

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 6 files

Re-trigger cubic

{name: "uk nhs", entity: "UK_NHS", text: "NHS number 943 476 5919"},
{name: "url", entity: "URL", text: "Visit https://example.com/path"},
{name: "us bank", entity: "US_BANK_NUMBER", text: "Bank account 123456789012"},
{name: "us driver license", entity: "US_DRIVER_LICENSE", text: "Driver license D1234567"},
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I gather you are trying to verify that presidio is able to handle the various detection rules/entities that you put here and detect stuff. However if I'm not mistaken the test runs against a mock so we never actually verify that?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry yeah. This test is stupid.

if _, blocked := presidioEntityBlacklist[e]; blocked {
continue
}
if _, supported := presidioSupportedEntities[e]; !supported {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm not mistaken, this will not break presidio sending unknown entities will simply be ignored. Now if we add detection for a new entity we'll have to update several places

  1. Add to UI
  2. Remove from UI blacklist
  3. Remove from request blacklist
  4. Add to presidio

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. Is it a concern needing to update multiple places?

Comment thread client/dashboard/src/pages/security/PolicyCenter.tsx Outdated
Comment thread client/dashboard/src/pages/security/policy-data.ts Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

preview Spawn a preview environment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants