Skip to content

feat: add agent eval reporting - #70

Merged
jlitola merged 2 commits into
mainfrom
fuchsia-twig
May 7, 2026
Merged

feat: add agent eval reporting#70
jlitola merged 2 commits into
mainfrom
fuchsia-twig

Conversation

@jlitola

@jlitola jlitola commented May 7, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add derived agent eval reports with console summaries, JSON output, and before/after comparison.
  • Add follow-up guidance so agents can reopen summaries, compare against baselines, and inspect raw tool calls quickly.
  • Harden report artifact loading against traversal and symlink escapes, and cover workload docs sync.

Validation

  • bun test
  • bun run typecheck
  • bun run format:check
  • Live Claude and Codex package-overview-vulnerabilities workloads

jlitola added 2 commits May 7, 2026 13:35
Adds a derived report command and console summary so agent eval runs are easier to inspect and compare without relying on raw artifact spelunking.
Documents the new package vulnerability filter workload so the agent eval workload coverage check stays in sync after rebasing onto main.
@jlitola
jlitola merged commit 38a9f65 into main May 7, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant