Skip to content

usability: document intentional AgentShield surface and clean legacy names #174

@changliu2

Description

@changliu2

Reporter perspective

As a first-time user auditing the ASSERT repo after the rename, I grepped for AgentShield, Agent Shield, and agent-shield. Most hits are intentional because the incident-triage example is a joint AgentShield + ASSERT demo. A few surfaces could still use cleanup or an explicit note so new users do not mistake the references for stale branding.

Evidence and recommended action by group

Keep: joint incident-triage case study references

These are intentional and should stay, but the docs should make the intent clear near the first mention.

  • examples/README.md:42 — examples table describes the joint AgentShield + ASSERT demo. Action: keep; forward-slash path cleanup is tracked separately.
  • examples/incident_triage_agent/README.md:3, :24, :156, :451, :463, :482, :505, :513, :581, :696, :894, :896, :909, :926, :1015, :1037, :1057, :1145-1147, :1155, :1177, :1203, :1232-1233, :1245, :1250, :1260, :1275, :1289 — narrative, install, provenance, and demo instructions. Action: keep; ensure opening paragraph says AgentShield is an external runtime guardrail used by this example.
  • examples/incident_triage_agent/SOP.md:6, :64 — SOP references the guardrail policy. Action: keep.
  • examples/incident_triage_agent/agent_guarded.py:4, :10, :27, :57, :64, :76, :85, :175, :178, :183 — guarded target implementation. Action: keep.
  • examples/incident_triage_agent/agent.py:408 — notes the guarded variant wraps with AgentShield. Action: keep.
  • examples/incident_triage_agent/eval_config_baseline.yaml:1, :4, :8, :144; eval_config_guarded.yaml:1, :4, :8, :144; eval_config_guarded_gepa.yaml:1-2, :143; eval_config_naive_prompt.yaml:1, :146. Action: keep.
  • examples/incident_triage_agent/incident-triage.guardrails.yaml:214 and prompt files under prompts/. Action: keep.
  • .github/workflows/regression.yml:21 and tests/test_incident_triage_smoke.py:4, :9, :70, :99. Action: keep as CI/test context.

Clarify: website co-hosted Agent Shield routes

  • website/tsconfig.json:18-19 maps @/components/* and @/lib/* to app/agent-shield/....
  • website/app/Terminal.tsx:100 defaults the title to Agent Shield.
  • website/app/page.tsx:90, :95, :100, :105 link to /agent-shield/docs/....
  • website/docs/design-migration.md:77, :84 refers to Agent Shield docs.
  • website/.eslintrc.json:5 scopes rules to app/agent-shield/**/*.

Action: if the website intentionally co-hosts Agent Shield docs/components, add a short README/note in the website package. If not, rename paths/titles to the ASSERT surface.

Clean or confirm: legacy suite names

  • scripts/render_trade_off.py:7, :21, :87 still reference bank-manager-agent-shield / examples/bank_manager_agent_shield/... while noting incident-triage-agent-v1 is newer.

Action: remove old bank-manager defaults if that demo is gone, or mark them as legacy compatibility in the script help.

Recommended fix

  • Add one explicit sentence to examples/incident_triage_agent/README.md and/or examples/README.md: AgentShield is an external runtime guardrail intentionally used by this one case study.
  • Clarify website agent-shield route/component ownership.
  • Clean or clearly label scripts/render_trade_off.py legacy bank-manager-agent-shield references.

Slice rollups

Found by slice 4 and confirmed by slice 5:

  • C:\Users\changliu2\.copilot\session-state\3714f9ab-3680-4990-a750-a80c932203f2\files\usability-slice-4-rollup.md
  • C:\Users\changliu2\.copilot\session-state\3714f9ab-3680-4990-a750-a80c932203f2\files\usability-slice-5-rollup.md

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationenhancementNew feature or requestshould-fixConfusing or visibly rough but not launch-blocking

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions