Skip to content

feat: add Datadog triage workflow - #111

Merged
egdbear merged 1 commit into
mainfrom
egdbear/add-datadog-triage-workflow
Aug 6, 2026
Merged

feat: add Datadog triage workflow#111
egdbear merged 1 commit into
mainfrom
egdbear/add-datadog-triage-workflow

Conversation

@egdbear

@egdbear egdbear commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

This adds a reusable workflow that looks at a repo's production errors in Datadog and opens a single pre-triaged GitHub issue for the top untracked crash, plus a copy-paste caller template for opting a repo in.

It runs the datadog-triage skill from artsy/agent-tooling#10, pinned to the artsy-artnet-v1.4.0, so a skill edit can't quietly change what runs against production overnight. The job is read-only which means that its only write is gh issue, enforced by a deny by default settings file and a hook that blocks credential reads (more about security in this artsy/agent-tooling#10).

Assisted-by: Claude:Opus-5

Comment thread .github/workflows/daily-datadog-triage.yml Outdated
Comment thread .github/workflows/daily-datadog-triage.yml Outdated
Comment thread .github/workflows/daily-datadog-triage.yml Outdated
Comment thread .github/workflows/daily-datadog-triage.yml Outdated
Comment thread .github/workflows/run-daily-datadog-triage.yml Outdated

@MounirDhahri MounirDhahri left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks good to me. I don't know though if a daily run makes sense here.
Given that this might be opening PRs here and there, I would also share a note about this running as an experiment so more folks are aware of it. If it works great here, we can bring it to other apps as well

@egdbear
egdbear force-pushed the egdbear/add-datadog-triage-workflow branch from 6ea8723 to b193dad Compare August 5, 2026 14:05
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

Detected new or modified files in .github/workflows/. Please ensure you update the relevant documentation alongside these workflow changes.

@github-actions github-actions Bot deleted a comment from claude Bot Aug 5, 2026
Comment thread .github/workflows/daily-datadog-triage.yml Outdated
Comment thread .github/workflows/daily-datadog-triage.yml Outdated
@egdbear

egdbear commented Aug 5, 2026

Copy link
Copy Markdown
Contributor Author

this looks good to me. I don't know though if a daily run makes sense here. Given that this might be opening PRs here and there, I would also share a note about this running as an experiment so more folks are aware of it. If it works great here, we can bring it to other apps as well

🙏 Quick clarification: no PRs, just triaging and opening issues. Example here: https://github.com/artsy/gravity/issues/20411

I'll spread the word once it's hooked up!

@MounirDhahri

Copy link
Copy Markdown
Member

Nice! great

@egdbear
egdbear force-pushed the egdbear/add-datadog-triage-workflow branch from b193dad to d2b01e8 Compare August 6, 2026 08:22
@github-actions github-actions Bot deleted a comment from claude Bot Aug 6, 2026
Comment thread .github/workflows/daily-datadog-triage.yml Outdated
@egdbear
egdbear force-pushed the egdbear/add-datadog-triage-workflow branch from d2b01e8 to 2281a6d Compare August 6, 2026 10:11
@github-actions github-actions Bot deleted a comment from claude Bot Aug 6, 2026
@claude

claude Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Code Review

Summary
Adds a reusable daily-datadog-triage.yml workflow that scans a private repo's Datadog production errors and opens at most one pre-triaged GitHub issue per run, plus a caller template and docs. The job is read-only: its only write is gh issue, scoped to a dedicated triage-issues-token while GITHUB_TOKEN gets contents: read.

Issues Found
No blocking or important issues. The workflow is careful and the reasoning is documented inline where it matters.

A few things I checked and confirmed are correct:

  • The public-repo gate assigns IS_PRIVATE on its own line before the if, so a failing gh aborts instead of being read as "public" — the comment calls this out and the code matches.
  • pup is pinned by digest and verified with sha256sum -c, then pup --version confirms the binary runs.
  • datadog-labs/agent-skills is checked out at a pinned SHA, not npx skills add.
  • Tool policy is derived once from the skill's settings.example.json via jq and restated as --allowedTools/--disallowedTools because claude_args takes precedence over --settings — consistent with how claude-review.yml already passes tools.
  • The __SKILL_DIR__ substitution has a grep guard so an unsubstituted placeholder fails closed rather than leaving the PreToolUse hook pointing at a nonexistent command.
  • Empty allow/deny lists abort. GH_TOKEN for the preflight is github.token; only the triage step swaps in triage-issues-token.

Areas Reviewed

  • Security: Deny-by-default policy, credential-read hook, least-privilege permissions: contents: read, and separate issues token all look right. Secrets are declared required in workflow_call.
  • Bugs & edge cases: set -euo pipefail throughout; the jq -e + set -e combination aborts on missing keys. Concurrency queues rather than cancels to avoid paying token cost for a half-finished run.
  • Docs: docs/actions.md and the template match the workflow's inputs, secrets, and behavior. No drift found.

Questions for Author

  1. Skill discovery paths. dd-pup/dd-logs/dd-apm are copied to $HOME/.claude/skills/, while datadog-triage is read from the checkout at $GITHUB_WORKSPACE/.claude/skills/. Does claude-code-action@v1 run claude with a HOME and working directory such that both the user-level and project-level skill dirs are discovered? If the action ever runs claude under a different HOME, the $HOME-installed dd skills would silently not load, and unlike the datadog-triage skill there is no abort guard for their absence.
  2. Pinned agent-skills SHA reachability. git clone (default refs) then checkout <SHA> works only while DD_SKILLS_SHA stays reachable from a fetched branch. If upstream force-pushes and the commit becomes unreachable, checkout fails — a fail-closed error, but worth confirming the pin is on main's history so this does not surprise a scheduled run.

@egdbear
egdbear merged commit 81b4304 into main Aug 6, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants