FORGE

FORGE is a runnable research workbench aligned with the DARPA AI Forge roadmap. It maps the 15 public Critical AI Challenges into concrete subprojects with executable prototypes, measurable evidence artifacts, and scale-up paths for larger models, secure testbeds, and mission simulations.

Sources:

• DARPA AI Forge program page: https://www.darpa.mil/research/programs/ai-forge
• DARPA AI Forge Critical AI Challenges report: https://www.darpa.mil/sites/default/files/attachment/2026-06/ai-forge-report.pdf

Quick Start

python3 -m ai_forge_workbench.cli list
python3 -m ai_forge_workbench.cli run I1
python3 -m ai_forge_workbench.cli run-all --out reports/latest
python3 -m ai_forge_workbench.cli integrated-demo --out reports/latest/integrated-assurance.json
python3 -m ai_forge_workbench.cli portfolio --reports reports/latest --out reports/portfolio
python3 -m pytest

No third-party dependencies are required for the first pass.

If pytest is not installed, run the dependency-free smoke test:

python3 tests/smoke.py

Challenge Coverage

ID	Thrust	Subproject	Local prototype
I1	AI Interpretability	Scaling causal interpretability	Causal intervention diagnostics under stress
I2	AI Interpretability	Long-horizon black-box accountability	Trace recorder plus temporal delta debugging
I3	AI Interpretability	Automated interpretability at scale	Role-tailored explanations with faithfulness checks
I4	AI Interpretability	Agentic AI auditability	Goal, plan, tool, and memory event fingerprints
I5	AI Interpretability	Scientific discovery evaluation	Evidence package scoring and judge calibration
C1	AI Control	Verifiable steerability	Uncertainty-aware deference controller
C2	AI Control	AI provenance	Hash-linked manifest and behavior diff tests
C3	AI Control	Secure agent sandbox	Risk-tiered policy and information-flow engine
C4	AI Control	Runtime intervention	Low-latency monitor with revocation and overrides
C5	AI Control	Mission assurance evaluation	Dynamic perturbation bench and rare-failure estimate
R1	Adversarial Robustness	Training data compromise	Poisoned-shard simulator and trigger tests
R2	Adversarial Robustness	Adaptive interactive defense	Attacker/defender loop with countermeasure costs
R3	Adversarial Robustness	Multi-agent active defense	Byzantine-tolerant consensus and isolation
R4	Adversarial Robustness	Continual learning hardening	Drift-vs-poison quarantine and unlearning hooks
R5	Adversarial Robustness	Robustness benchmarking	Operational taxonomy and certification evidence matrix

Architecture

FORGE is organized around three workstreams:

• Interpretability modules produce causal diagnostics, trace attributions, explanation checks, and auditable agent records.
• Control modules cover provenance, sandboxing, runtime interventions, steerability checks, and mission-readiness cases.
• Robustness modules simulate training-data compromise, adaptive adversaries, subverted multi-agent systems, continual-learning attacks, and benchmark evidence.
• The integrated demo combines the workstreams into one cross-track evidence artifact.

Generated Artifacts

• reports/portfolio/AI_FORGE_PORTFOLIO.md
• reports/portfolio/index.html
• reports/latest/integrated-assurance.json