Skip to content

Enforce selected-text disposal and explicit model-egress privacy controls #103

Description

@jmcte

Problem

Expired cache entries are only marked with evicted_at; cached_text remains in SQLite indefinitely. The table also lacks the documented purpose, redaction state, provenance, and review fields.

Optional cloud LLM extraction sends sender, subject, folder, and date metadata without a central redaction/pseudonymization stage or explicit provider-mode policy.

Relevant code

  • src/mailplus_intelligence/cache.py:43-117
  • src/mailplus_intelligence/llm_extractor.py:82-99,145-225
  • src/mailplus_intelligence/migrations/003_cache_and_queue.sql:3-15
  • docs/selected-message-text-cache-policy.md
  • docs/privacy-redaction-boundaries.md:24-65,158-177
  • docs/operational-safety.md

Scope

Cache lifecycle

  • Physically delete, overwrite, or cryptographically destroy expired selected text.
  • Bound TTL values and reject invalid/unsafe retention.
  • Record purpose, class, redaction state, provenance, creation/expiry, and review requirement.
  • Add privacy-safe durable events for write, read, miss, denial, expiry, and disposal.
  • Document WAL/backup implications and verify owner-only permissions for database sidecars where practical.

Model egress

  • Add a provider policy such as disabled | local | cloud, with cloud disabled by default.
  • Require explicit operator configuration for cloud use.
  • Redact or pseudonymize sensitive subject/sender/folder values before cloud requests where the task does not require them.
  • Record what data classes and provider/model were used without logging payload content.
  • Fail closed when policy and requested extractor disagree.

Non-goals

  • No bulk body cache.
  • No live mailbox fetch in this issue.
  • No claim of secure deletion from unmanaged filesystem snapshots/backups; document the boundary.
  • No provider-specific secrets in config files or logs.

Acceptance criteria

  • Disposed entries no longer retain recoverable cached_text in active database rows.
  • Invalid, negative, or excessive TTLs are rejected or bounded explicitly.
  • Cache rows contain all required lifecycle metadata.
  • Audit events never contain cached text or secrets.
  • Cloud extraction is disabled by default and requires explicit opt-in.
  • Redaction/pseudonymization is tested for sensitive metadata.
  • Provider/model/data-class audit metadata is persisted without prompt payloads.
  • Tests cover expiration, disposal, WAL-aware operational guidance, denial, and egress-policy failure.

Validation

PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python3.12 -m unittest tests.test_cache tests.test_llm_extractor -v
bash scripts/check-detect-secrets.sh --all-files
bash scripts/ci/run-fast-checks.sh

Parent

Part of #98.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:dataData, schema, storage, privacy, or migration surface.area:securitySecurity, auth, secret, permission, or policy surface.bugSomething isn't workingpriority:P1Codex Connector P1; blocks execution until Athena and Ares validate.ready-for-agentIssue is approved for worker-lane execution.risk:securitySecurity-sensitive change.status:ready-for-agentIssue is approved for worker-lane execution.

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions