Skip to content

fix(e2e): capture platform failure diagnostics#123

Merged
rowan-stein merged 1 commit into
mainfrom
casey/chat-app-broad-e2e-diagnostics
May 20, 2026
Merged

fix(e2e): capture platform failure diagnostics#123
rowan-stein merged 1 commit into
mainfrom
casey/chat-app-broad-e2e-diagnostics

Conversation

@casey-brooks
Copy link
Copy Markdown
Contributor

Summary

  • Capture dependent platform service logs when an E2E suite fails.
  • Include agyn-workloads pod status/events/logs so agent startup and Ziti sidecar issues are visible.
  • Copy Playwright test-results and playwright-report into artifacts for screenshot/trace inspection.

Context

This supports root-cause investigation for agynio/chat-app#95 broad service: chat_app failures without narrowing selectors, pinning refs, or injecting caller env vars. The latest artifacts from run 26132904397 only had E2E pod state and an empty runner log; they did not include llm-proxy/llm/agents-orchestrator/tracing/authorization/Ziti/OpenFGA logs needed to confirm or reject the suspected 403/terminator/tuple readiness issue.

Validation

  • bash -n scripts/run-pipeline.sh
  • shellcheck scripts/run-pipeline.sh
  • nix shell --impure --expr 'with import <nixpkgs> {}; python3.withPackages (ps: [ ps.pyyaml ])' -c python3 scripts/parse_suite.py suites/playwright-chat-app/suite.yaml /tmp/e2e-parse-suite-check

@casey-brooks
Copy link
Copy Markdown
Contributor Author

Test & Lint Summary

  • bash -n scripts/run-pipeline.sh — passed (syntax: 1/1)
  • shellcheck scripts/run-pipeline.sh — passed with no warnings/errors
  • nix shell --impure --expr 'with import <nixpkgs> {}; python3.withPackages (ps: [ ps.pyyaml ])' -c python3 scripts/parse_suite.py suites/playwright-chat-app/suite.yaml /tmp/e2e-parse-suite-check — passed (suite parse: 1/1)

Copy link
Copy Markdown

@noa-lucent noa-lucent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the diagnostic/artifact collection changes in scripts/run-pipeline.sh hunk-by-hunk. The added helpers preserve the existing failure semantics, collect the requested platform/workload diagnostics only after failures, and copy Playwright artifacts when present without breaking non-Playwright suites.

No blocking issues found.

@rowan-stein rowan-stein merged commit 11ce033 into main May 20, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants