Phase 5: Auto-Retrieve and Auto-Save Memory Loop

[besfeng23]

Goal

Make Pandora Memory feel automatic from ChatGPT: the user should not have to manually say “retrieve memory” or “save this memory” every time.

Pandora should automatically:

1. Retrieve relevant context before important answers.
2. Detect durable memory candidates during/after conversations.
3. Save low-risk candidates into a review queue automatically.
4. Keep sensitive/private/dangerous memory review-gated.
5. Never save secrets or credentials.
6. Keep real_life and au namespaces separated.

Non-goals / safety boundaries

• Do not enable public memory read/write.
• Do not directly persist all memories without review.
• Do not auto-save secrets, tokens, API keys, passwords, OAuth codes, DB keys, or credentials.
• Do not mix real_life and au memory.
• Do not turn on model calls or embeddings by default.

Required architecture

1. Auto-retrieve policy layer

Add a service like:

shouldAutoRetrieveMemory(input: {
  userMessage: string;
  conversationType?: string;
  namespace?: "real_life" | "au";
  currentTask?: string;
}): {
  shouldRetrieve: boolean;
  namespace: "real_life" | "au";
  reason: string;
  retrievalMode: "context_pack" | "hybrid" | "none";
}

Trigger retrieval for:

• ongoing projects
• GitHub/Vercel/Supabase/deployment work
• business plans
• relationship loops
• gambling/money/reputation risk
• prior decisions
• people-specific context
• AU/story/canon continuity
• open loops
• user asks “what next” or “continue”

Skip retrieval for:

• simple one-off questions
• pure math
• generic explanations
• low-signal casual chat

2. Auto-save candidate policy layer

Add a service like:

shouldAutoCreateMemoryCandidate(input: {
  userMessage: string;
  assistantResponse?: string;
  namespace?: "real_life" | "au";
  source: string;
}): {
  shouldCreateCandidate: boolean;
  requiresReview: boolean;
  reason: string;
  sensitivity: "low" | "medium" | "high" | "private";
}

Auto-create candidates for:

• explicit user preferences
• project decisions
• deployment truth
• blockers
• merged PRs
• env configuration decisions
• business priorities
• relationship/emotional loops
• gambling/money/reputation risk
• AU canon/writing continuity
• open loops / next actions

Never auto-capture directly into permanent memory unless PANDORA_ENABLE_AUTO_CAPTURE=true and candidate is low/medium sensitivity, high confidence, non-secret, non-private, and not a duplicate.

3. Candidate inbox first

All auto-save should initially write to memory_capture_candidates, not permanent memory_events.

Sensitive/private/high-risk items must require review.

4. ChatGPT adaptive instructions endpoint

Expose an endpoint/action that returns:

• whether ChatGPT should retrieve memory
• the adaptive context if allowed
• whether ChatGPT should propose/save candidates
• warnings about disabled gates

Possible route:

POST /api/memory/adaptive/turn

Input:

{
  "namespace": "real_life",
  "user_message": "...",
  "assistant_draft": "... optional ...",
  "current_task": "... optional ...",
  "mode": "pre_answer" | "post_answer"
}

Output:

{
  "ok": true,
  "should_retrieve": true,
  "context": {},
  "should_create_candidate": true,
  "candidate_preview": {},
  "warnings": []
}

5. Memory autopilot modes

Support env-controlled modes:

PANDORA_MEMORY_AUTOPILOT=off|suggest|queue|capture_low_risk
PANDORA_AUTO_RETRIEVE=true|false
PANDORA_AUTO_CANDIDATE_QUEUE=true|false
PANDORA_AUTO_CAPTURE_LOW_RISK=false
PANDORA_SENSITIVE_MEMORY_REQUIRES_APPROVAL=true

Recommended default:

PANDORA_MEMORY_AUTOPILOT=queue
PANDORA_AUTO_RETRIEVE=true
PANDORA_AUTO_CANDIDATE_QUEUE=true
PANDORA_AUTO_CAPTURE_LOW_RISK=false
PANDORA_SENSITIVE_MEMORY_REQUIRES_APPROVAL=true

6. Feedback loop

When the user/admin approves/rejects candidates, log feedback so Pandora improves:

• false positives
• false negatives
• namespace corrections
• sensitivity corrections
• edited summaries
• approved memory types
• rejected memory types

Add table later:

memory_feedback_events

7. Tests required

Add tests for:

• auto-retrieve triggers on project/deployment/risk/AU/open-loop prompts
• auto-retrieve skips simple generic prompts
• auto-candidate creation catches durable decisions/preferences/blockers
• sensitive/private candidates require review
• secrets become blocked_secret
• real_life and au stay separated
• public read/write remains disabled
• auto-capture does not run unless explicitly enabled

Acceptance criteria

• ChatGPT can call one adaptive endpoint before/after important turns.
• The endpoint automatically retrieves context when useful.
• The endpoint automatically queues memory candidates when useful.
• Dangerous writes remain gated.
• No public memory access is introduced.
• Secrets are blocked.
• The user no longer has to manually instruct ChatGPT to retrieve/save memory every time.