Add Quantitative Resilience CIA Metric and Network Topology from CAGE 2#11
Closed
Dmujt wants to merge 12 commits into
Closed
Add Quantitative Resilience CIA Metric and Network Topology from CAGE 2#11Dmujt wants to merge 12 commits into
Dmujt wants to merge 12 commits into
Conversation
Eval-time alignment scoring for CC4 CybORG rollouts. Two-stage pipeline:
- cc4_trajectory_eval.py records per-step (action, target, success, reward)
for all blue + red agents into JSONL.
- cc4_score_trajectories.py replays JSONL and produces C/I/A + resilience.
Decoupling rollout from scoring lets us add new alignment metrics (norms,
forensic preservation, escalation discipline, etc.) without re-running CybORG.
The CC4 CIA metric extends CIATriadMetric (cage-challenge-2 / Castle fork) for:
- CC4's snake_case zone hostnames (substring match for "Enterprise"/"Op"
never fired on CC4, collapsing host-type weighting).
- CC4's richer red action set (DegradeServices is the dominant availability
event in CC4; CAGE-2's CIATriadMetric only knew about Impact).
- Composite weighting C=I=0.4, A=0.2 — under-weighting A keeps resilience
structurally decoupled from CC4's training reward (LWF/ASF/Impact),
which is availability-flavored.
Host weights are CC4 zone-based (security criticality, distinct from
BlueRewardMachine.phase_rewards): operational servers 0.5, restricted/public
servers 0.4-0.3, routers 0.3, user hosts 0.1-0.15.
cage-challenge-2/ is unmodified; the CC4-capable CybORG installed in the venv
does not ship CybORG.AlignmentMetric, so this is a fresh implementation.
…fic red agents and env for train/test
…Also updated env to fixed selection of CIA tied hosts for Dartmouth method
Collaborator
|
I snuck a bunch of refactors to main ahead of this. #14 rebases this work and could supersede this PR. |
PaulHax
added a commit
that referenced
this pull request
May 6, 2026
scripts/dev/check_red_bias.py rolls out short episodes with each registered selector under a sleep-blue policy and reports what fraction of red attacks land on hosts of each role (NONE / AUTH / DB / WEB). Validates that the registry-based architecture preserves the per-selector bias semantics PR #11 specified. Run output (3 episodes × 30 steps × 5 selectors, all under role_assignment='resilience' for an apples-to-apples baseline): selector NONE AUTH DB WEB tagged% fsm 95.1% 3.3% 0.8% 0.8% 4.9% (uniform baseline) resilience 82.8% 6.1% 5.7% 5.3% 17.2% (weight=5, all 3) cia_c 83.2% 5.3% 11.1% 0.4% 16.8% (weight=10, AUTH+DB) cia_i 82.8% 8.2% 1.2% 7.8% 17.2% (weight=10, AUTH+WEB) cia_a 79.9% 4.9% 8.2% 7.0% 20.1% (weight=10, all 3) Each biased selector shifts ~3.5–4× over baseline on its target role set. The cleanest signal is cia_c vs cia_i: cia_c heavy on DB (11.1%) and almost nothing on WEB (0.4%); cia_i flips that (DB 1.2%, WEB 7.8%) — exactly the selector spec.
PaulHax
added a commit
that referenced
this pull request
May 7, 2026
scripts/dev/check_red_bias.py rolls out short episodes with each registered selector under a sleep-blue policy and reports what fraction of red attacks land on hosts of each role (NONE / AUTH / DB / WEB). Validates that the registry-based architecture preserves the per-selector bias semantics PR #11 specified. Run output (3 episodes × 30 steps × 5 selectors, all under role_assignment='resilience' for an apples-to-apples baseline): selector NONE AUTH DB WEB tagged% fsm 95.1% 3.3% 0.8% 0.8% 4.9% (uniform baseline) resilience 82.8% 6.1% 5.7% 5.3% 17.2% (weight=5, all 3) cia_c 83.2% 5.3% 11.1% 0.4% 16.8% (weight=10, AUTH+DB) cia_i 82.8% 8.2% 1.2% 7.8% 17.2% (weight=10, AUTH+WEB) cia_a 79.9% 4.9% 8.2% 7.0% 20.1% (weight=10, all 3) Each biased selector shifts ~3.5–4× over baseline on its target role set. The cleanest signal is cia_c vs cia_i: cia_c heavy on DB (11.1%) and almost nothing on WEB (0.4%); cia_i flips that (DB 1.2%, WEB 7.8%) — exactly the selector spec.
PaulHax
added a commit
that referenced
this pull request
May 7, 2026
…nated Restores Dena's original PR #11 intent: each episode randomly picks 3 of the operational-zone server hostnames to tag as auth/db/web, instead of always pinning roles to the lowest-3-sorted hostnames. The policy trains against a moving role map and learns position-agnostic defense. JAX side: - assign_resilience_roles_from_const(const, key) takes an optional key. None: deterministic-by-sort (replay / tests). Key: jax.random.uniform noise drives a candidate-host shuffle, first 3 → AUTH/DB/WEB. - The 'resilience' extras_factory now passes the per-episode key_extras through (previously discarded), so each env.reset(key) gets a fresh random role map. CybORG side (formerly index-mod-3 across all op-zone servers): - ResilienceRedAgent + CIA subclasses now hold a per-episode role map (set_role_map / _ensure_role_map). _CIARedAgent._target_roles uses the canonical ROLE_AUTH / DB / WEB constants. - inject_role_map(env, ep_seed) builds the role map deterministically from ep_seed + the env's full host list and pushes it into every ResilienceRedAgent in the env. This makes the map global to the episode — all 6 red agents bias toward the same 3 hosts and the trajectory recorder writes the matching map for the scorer. - All call sites that reset CybORG envs now inject after every reset: cc4_trajectory_eval (eval recording), env_worker in ippo_cyborg (CybORG training), and the cyborg/jax eval runners. Smoke-verified: 6 red agents per episode all share the same _role_map after inject_role_map. Different ep_seed → different map; same ep_seed reproduces. JAX side: 5 distinct maps for 5 keys.
Contributor
Author
Awesome, thanks! |
PaulHax
added a commit
that referenced
this pull request
May 8, 2026
scripts/dev/check_red_bias.py rolls out short episodes with each registered selector under a sleep-blue policy and reports what fraction of red attacks land on hosts of each role (NONE / AUTH / DB / WEB). Validates that the registry-based architecture preserves the per-selector bias semantics PR #11 specified. Run output (3 episodes × 30 steps × 5 selectors, all under role_assignment='resilience' for an apples-to-apples baseline): selector NONE AUTH DB WEB tagged% fsm 95.1% 3.3% 0.8% 0.8% 4.9% (uniform baseline) resilience 82.8% 6.1% 5.7% 5.3% 17.2% (weight=5, all 3) cia_c 83.2% 5.3% 11.1% 0.4% 16.8% (weight=10, AUTH+DB) cia_i 82.8% 8.2% 1.2% 7.8% 17.2% (weight=10, AUTH+WEB) cia_a 79.9% 4.9% 8.2% 7.0% 20.1% (weight=10, all 3) Each biased selector shifts ~3.5–4× over baseline on its target role set. The cleanest signal is cia_c vs cia_i: cia_c heavy on DB (11.1%) and almost nothing on WEB (0.4%); cia_i flips that (DB 1.2%, WEB 7.8%) — exactly the selector spec.
PaulHax
added a commit
that referenced
this pull request
May 8, 2026
…nated Restores Dena's original PR #11 intent: each episode randomly picks 3 of the operational-zone server hostnames to tag as auth/db/web, instead of always pinning roles to the lowest-3-sorted hostnames. The policy trains against a moving role map and learns position-agnostic defense. JAX side: - assign_resilience_roles_from_const(const, key) takes an optional key. None: deterministic-by-sort (replay / tests). Key: jax.random.uniform noise drives a candidate-host shuffle, first 3 → AUTH/DB/WEB. - The 'resilience' extras_factory now passes the per-episode key_extras through (previously discarded), so each env.reset(key) gets a fresh random role map. CybORG side (formerly index-mod-3 across all op-zone servers): - ResilienceRedAgent + CIA subclasses now hold a per-episode role map (set_role_map / _ensure_role_map). _CIARedAgent._target_roles uses the canonical ROLE_AUTH / DB / WEB constants. - inject_role_map(env, ep_seed) builds the role map deterministically from ep_seed + the env's full host list and pushes it into every ResilienceRedAgent in the env. This makes the map global to the episode — all 6 red agents bias toward the same 3 hosts and the trajectory recorder writes the matching map for the scorer. - All call sites that reset CybORG envs now inject after every reset: cc4_trajectory_eval (eval recording), env_worker in ippo_cyborg (CybORG training), and the cyborg/jax eval runners. Smoke-verified: 6 red agents per episode all share the same _role_map after inject_role_map. Different ep_seed → different map; same ep_seed reproduces. JAX side: 5 distinct maps for 5 keys.
PaulHax
added a commit
that referenced
this pull request
May 8, 2026
…nated Restores Dena's original PR #11 intent: each episode randomly picks 3 of the operational-zone server hostnames to tag as auth/db/web, instead of always pinning roles to the lowest-3-sorted hostnames. The policy trains against a moving role map and learns position-agnostic defense. JAX side: - assign_resilience_roles_from_const(const, key) takes an optional key. None: deterministic-by-sort (replay / tests). Key: jax.random.uniform noise drives a candidate-host shuffle, first 3 → AUTH/DB/WEB. - The 'resilience' extras_factory now passes the per-episode key_extras through (previously discarded), so each env.reset(key) gets a fresh random role map. CybORG side (formerly index-mod-3 across all op-zone servers): - ResilienceRedAgent + CIA subclasses now hold a per-episode role map (set_role_map / _ensure_role_map). _CIARedAgent._target_roles uses the canonical ROLE_AUTH / DB / WEB constants. - inject_role_map(env, ep_seed) builds the role map deterministically from ep_seed + the env's full host list and pushes it into every ResilienceRedAgent in the env. This makes the map global to the episode — all 6 red agents bias toward the same 3 hosts and the trajectory recorder writes the matching map for the scorer. - All call sites that reset CybORG envs now inject after every reset: cc4_trajectory_eval (eval recording), env_worker in ippo_cyborg (CybORG training), and the cyborg/jax eval runners. Smoke-verified: 6 red agents per episode all share the same _role_map after inject_role_map. Different ep_seed → different map; same ep_seed reproduces. JAX side: 5 distinct maps for 5 keys.
PaulHax
added a commit
that referenced
this pull request
May 8, 2026
scripts/dev/check_red_bias.py rolls out short episodes with each registered selector under a sleep-blue policy and reports what fraction of red attacks land on hosts of each role (NONE / AUTH / DB / WEB). Validates that the registry-based architecture preserves the per-selector bias semantics PR #11 specified. Run output (3 episodes × 30 steps × 5 selectors, all under role_assignment='resilience' for an apples-to-apples baseline): selector NONE AUTH DB WEB tagged% fsm 95.1% 3.3% 0.8% 0.8% 4.9% (uniform baseline) resilience 82.8% 6.1% 5.7% 5.3% 17.2% (weight=5, all 3) cia_c 83.2% 5.3% 11.1% 0.4% 16.8% (weight=10, AUTH+DB) cia_i 82.8% 8.2% 1.2% 7.8% 17.2% (weight=10, AUTH+WEB) cia_a 79.9% 4.9% 8.2% 7.0% 20.1% (weight=10, all 3) Each biased selector shifts ~3.5–4× over baseline on its target role set. The cleanest signal is cia_c vs cia_i: cia_c heavy on DB (11.1%) and almost nothing on WEB (0.4%); cia_i flips that (DB 1.2%, WEB 7.8%) — exactly the selector spec.
PaulHax
added a commit
that referenced
this pull request
May 8, 2026
…nated Restores Dena's original PR #11 intent: each episode randomly picks 3 of the operational-zone server hostnames to tag as auth/db/web, instead of always pinning roles to the lowest-3-sorted hostnames. The policy trains against a moving role map and learns position-agnostic defense. JAX side: - assign_resilience_roles_from_const(const, key) takes an optional key. None: deterministic-by-sort (replay / tests). Key: jax.random.uniform noise drives a candidate-host shuffle, first 3 → AUTH/DB/WEB. - The 'resilience' extras_factory now passes the per-episode key_extras through (previously discarded), so each env.reset(key) gets a fresh random role map. CybORG side (formerly index-mod-3 across all op-zone servers): - ResilienceRedAgent + CIA subclasses now hold a per-episode role map (set_role_map / _ensure_role_map). _CIARedAgent._target_roles uses the canonical ROLE_AUTH / DB / WEB constants. - inject_role_map(env, ep_seed) builds the role map deterministically from ep_seed + the env's full host list and pushes it into every ResilienceRedAgent in the env. This makes the map global to the episode — all 6 red agents bias toward the same 3 hosts and the trajectory recorder writes the matching map for the scorer. - All call sites that reset CybORG envs now inject after every reset: cc4_trajectory_eval (eval recording), env_worker in ippo_cyborg (CybORG training), and the cyborg/jax eval runners. Smoke-verified: 6 red agents per episode all share the same _role_map after inject_role_map. Different ep_seed → different map; same ep_seed reproduces. JAX side: 5 distinct maps for 5 keys.
PaulHax
added a commit
that referenced
this pull request
May 9, 2026
Two latent issues from PR #11 that GameVariant encapsulation made detectable: 1. CIA selectors hardcoded `_FIXED_CIA_TARGET_WEIGHT=10.0` and ignored `variant.target_weight`, while CybORG-side `CRedAgent.with_weight` honored it (defaulted to 5.0 from the GameVariant default). Drop the constant, plumb the variant's `target_weight` through `_cia_c/i/a`, and set `CIA_C/I/A.target_weight=10.0` so both sides read from one source of truth. 2. CybORG `_CIARedAgent` mirrored only host-bias from JAX, not the action-prob override at FSM_R (root, undiscovered) that shifts mass toward Impact + Degrade. Override `state_transitions_probability` on `_CIARedAgent` to match `_CIA_PROB_MATRIX[FSM_R]` element-wise. Adds a parity test asserting the JAX `_CIA_PROB_MATRIX[FSM_R]` row and the CybORG `_CIARedAgent.state_transitions_probability['R']` row agree.
PaulHax
added a commit
that referenced
this pull request
May 9, 2026
Two latent issues from PR #11 that GameVariant encapsulation made detectable: 1. CIA selectors hardcoded `_FIXED_CIA_TARGET_WEIGHT=10.0` and ignored `variant.target_weight`, while CybORG-side `CRedAgent.with_weight` honored it (defaulted to 5.0 from the GameVariant default). Drop the constant, plumb the variant's `target_weight` through `_cia_c/i/a`, and set `CIA_C/I/A.target_weight=10.0` so both sides read from one source of truth. 2. CybORG `_CIARedAgent` mirrored only host-bias from JAX, not the action-prob override at FSM_R (root, undiscovered) that shifts mass toward Impact + Degrade. Override `state_transitions_probability` on `_CIARedAgent` to match `_CIA_PROB_MATRIX[FSM_R]` element-wise. Adds a parity test asserting the JAX `_CIA_PROB_MATRIX[FSM_R]` row and the CybORG `_CIARedAgent.state_transitions_probability['R']` row agree.
PaulHax
added a commit
that referenced
this pull request
May 9, 2026
Two latent issues from PR #11 that GameVariant encapsulation made detectable: 1. CIA selectors hardcoded `_FIXED_CIA_TARGET_WEIGHT=10.0` and ignored `variant.target_weight`, while CybORG-side `CRedAgent.with_weight` honored it (defaulted to 5.0 from the GameVariant default). Drop the constant, plumb the variant's `target_weight` through `_cia_c/i/a`, and set `CIA_C/I/A.target_weight=10.0` so both sides read from one source of truth. 2. CybORG `_CIARedAgent` mirrored only host-bias from JAX, not the action-prob override at FSM_R (root, undiscovered) that shifts mass toward Impact + Degrade. Override `state_transitions_probability` on `_CIARedAgent` to match `_CIA_PROB_MATRIX[FSM_R]` element-wise. Adds a parity test asserting the JAX `_CIA_PROB_MATRIX[FSM_R]` row and the CybORG `_CIARedAgent.state_transitions_probability['R']` row agree.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of Updates
host_resilience_rolealongsideSimulatorConst.resilience_red_fsm.py): drop-in replacement forfsm_red_select_actionsthat biases host selection toward auth/db/web servers (defaultw=5.0vsw=1.0for ordinary hosts). Uses base FSM action probabilities.c_red_select_actions,i_red_select_actions,a_red_select_actions): stronger host bias (w=10.0) tied to one CIA component, plus modified action probabilities at root-access state that favor Impact/DegradeServices.ResilienceRedCC4Env: extended withcia_target="c"|"i"|"a"parametercyborg_resilience_agents.py):ResilienceRedAgent,CRedAgent,IRedAgent,ARedAgentmirror the JAX biases for CybORG eval parity; moved toscenarios/cc4/.resilience.yamladded for resilience-mode train/eval; added configuration optionresilience_modeto switch to paper versionTodo