Skip to content

Implement an operator-selected transient source-fetch boundary #109

Description

@jmcte

Objective

Enable evidence-grounded commitment, decision, and follow-up extraction without turning MailPlus Intelligence into a second raw-mail archive.

Bulk/live sync remains metadata-only. Exact source text is fetched only for an explicitly selected high-value message or thread, minimized/redacted, used for a bounded operation, and discarded unless a separately approved cache policy applies.

Product rationale

Subject/sender metadata alone cannot reliably answer “Did I commit to this?” or “What follow-up is pending?” The existing architecture already describes on-demand fetch and a selected-text policy; this issue implements that boundary safely.

Relevant code/docs

  • docs/raw-fetch-interface.md
  • docs/selected-message-text-cache-policy.md
  • docs/privacy-redaction-boundaries.md
  • docs/candidate-extraction.md
  • src/mailplus_intelligence/cache.py
  • src/mailplus_intelligence/extractor.py
  • src/mailplus_intelligence/llm_extractor.py
  • src/mailplus_intelligence/queue.py

Scope

  • Define typed fetch requests/responses keyed by exact MailPlus locator.
  • Implement fixture/fake backend first and a live read-only backend through the approved adapter boundary.
  • Require explicit operator selection, purpose, and allowed data class.
  • Return transient content in memory; do not log or persist it implicitly.
  • Add minimization/redaction for secrets, reset/payment/tracking links, signatures, quoted history, and unnecessary recipients.
  • Feed only minimized evidence into deterministic/local/cloud extraction according to provider policy.
  • Preserve evidence field references/hashes and source locators without copying raw text into durable audit logs.
  • Dispose of transient buffers and document process/crash/core-dump limitations.
  • Make any cache write a separate explicit action governed by the selected-text policy.

Non-goals

  • No bulk full-text indexing.
  • No attachment content extraction.
  • No implicit caching.
  • No automatic promotion or reminder creation.
  • No cloud egress without the explicit provider policy from the privacy issue.
  • No claim that Python memory zeroization is perfect; document realistic boundaries.

Acceptance criteria

  • Fetch requires an exact, authorized locator and explicit purpose.
  • Missing, moved, ambiguous, credential-gated, and backend-unavailable failures are typed.
  • Metadata sync never enters this fetch path.
  • Raw source content is absent from SQLite, logs, audit events, test reports, and export artifacts unless an explicit permitted cache write occurs.
  • Redaction/minimization covers documented secret/link/signature/quoted-history classes.
  • Extraction candidates include complete provenance and evidence references.
  • Provider policy is enforced before any model request.
  • Fixture and fake-live tests prove fetch, redaction, extraction, disposal, and failure behavior.
  • Operator docs explain the exact approval and disposal boundary.

Validation

Include privacy-focused tests that scan produced logs, databases, reports, and artifacts for unique sentinel secrets from the transient source fixture.

Dependency notes

Blocked on the provenance/review, cache/egress, live adapter, and unified ingest issues linked from #98. Interface and fixture work may begin earlier.

Exact dependencies

Blocked on #102, #103, #106, and #107. Fixture interface and redaction test design may begin earlier; no live content path may merge before all blockers close.

Parent

Part of #98.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:aiAI/model/agent behavior surface.area:securitySecurity, auth, secret, permission, or policy surface.featureNew feature or requestpriority:P1Codex Connector P1; blocks execution until Athena and Ares validate.ready-for-agentIssue is approved for worker-lane execution.risk:securitySecurity-sensitive change.status:ready-for-agentIssue is approved for worker-lane execution.

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions