Summary
Each run is currently a blank slate. The state manager persists ExploitRecord, FixRecord, and RunRecord per run, but nothing accumulates across runs. An org/repo profile that grows over time would let the agent leverage historical context instead of starting from scratch.
Goals
- Build an
OrgProfile model that aggregates data across runs for each org/repo
- Feed the profile into root agent context so findings are informed by history
- Make the agent aware of the team's tech stack, common patterns, and past vulnerabilities
Proposed Changes
1. OrgProfile model
A persistent model that grows across runs:
OrgProfile
+-- tech_stack: ["Solidity 0.8.x", "Foundry", "OpenZeppelin"]
+-- vulnerability_history: [{class, count, avg_severity, fp_rate}]
+-- suppression_rules: ["skip gas optimizations", ...]
+-- codebase_patterns: ["proxy/upgradeable", "governor/timelock"]
2. Profile accumulation
After each run completes, update the OrgProfile with:
- Detected tech stack and patterns
- New vulnerability findings (class, severity)
- FP rates from user feedback (depends on feedback loop feature)
- Which findings were acted on vs dismissed
3. Profile injection into run_exploit() context
The root prompt already accepts prior_findings and instructions. Add an org_context field populated from the profile:
- Root prompt: "This org typically uses OpenZeppelin's ReentrancyGuard, flag cases where they forgot it"
- Analyzer prompt: "Skip these known patterns, focus on deviations"
- Verifier prompt: "Severity calibration based on historical data for this org"
Implementation Notes
- Mostly a state schema change + prompt injection at
run_exploit() time
- The LLM does the heavy lifting of using the context
- Depends on the feedback loop feature for FP rates and suppression rules, but tech stack / vulnerability history can be built independently
Context
- State infrastructure:
LocalStateManager, ExploitRecord, RunRecord already exist
- See
docs/product-learning.md for full design rationale
Summary
Each run is currently a blank slate. The state manager persists
ExploitRecord,FixRecord, andRunRecordper run, but nothing accumulates across runs. An org/repo profile that grows over time would let the agent leverage historical context instead of starting from scratch.Goals
OrgProfilemodel that aggregates data across runs for each org/repoProposed Changes
1.
OrgProfilemodelA persistent model that grows across runs:
2. Profile accumulation
After each run completes, update the
OrgProfilewith:3. Profile injection into
run_exploit()contextThe root prompt already accepts
prior_findingsandinstructions. Add anorg_contextfield populated from the profile:Implementation Notes
run_exploit()timeContext
LocalStateManager,ExploitRecord,RunRecordalready existdocs/product-learning.mdfor full design rationale