Skip to content

Cross-run memory: org/repo profile that accumulates across runs #75

@andthattoo

Description

@andthattoo

Summary

Each run is currently a blank slate. The state manager persists ExploitRecord, FixRecord, and RunRecord per run, but nothing accumulates across runs. An org/repo profile that grows over time would let the agent leverage historical context instead of starting from scratch.

Goals

  • Build an OrgProfile model that aggregates data across runs for each org/repo
  • Feed the profile into root agent context so findings are informed by history
  • Make the agent aware of the team's tech stack, common patterns, and past vulnerabilities

Proposed Changes

1. OrgProfile model

A persistent model that grows across runs:

OrgProfile
+-- tech_stack: ["Solidity 0.8.x", "Foundry", "OpenZeppelin"]
+-- vulnerability_history: [{class, count, avg_severity, fp_rate}]
+-- suppression_rules: ["skip gas optimizations", ...]
+-- codebase_patterns: ["proxy/upgradeable", "governor/timelock"]

2. Profile accumulation

After each run completes, update the OrgProfile with:

  • Detected tech stack and patterns
  • New vulnerability findings (class, severity)
  • FP rates from user feedback (depends on feedback loop feature)
  • Which findings were acted on vs dismissed

3. Profile injection into run_exploit() context

The root prompt already accepts prior_findings and instructions. Add an org_context field populated from the profile:

  • Root prompt: "This org typically uses OpenZeppelin's ReentrancyGuard, flag cases where they forgot it"
  • Analyzer prompt: "Skip these known patterns, focus on deviations"
  • Verifier prompt: "Severity calibration based on historical data for this org"

Implementation Notes

  • Mostly a state schema change + prompt injection at run_exploit() time
  • The LLM does the heavy lifting of using the context
  • Depends on the feedback loop feature for FP rates and suppression rules, but tech stack / vulnerability history can be built independently

Context

  • State infrastructure: LocalStateManager, ExploitRecord, RunRecord already exist
  • See docs/product-learning.md for full design rationale

Metadata

Metadata

Assignees

Labels

No labels
No labels
No fields configured for Feature.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions