Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 19 additions & 16 deletions .github/prompts/investigate.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,13 @@

You are investigating a CockroachDB test failure. You are running
autonomously in a GitHub Action — there is no interactive
back-and-forth with a user. You must complete the full investigation
and write your findings to `artifacts/findings.md` (a later workflow
step posts it as a comment on the issue). Create the directory first
back-and-forth with a user. Your sole deliverable is a written
investigation report in the file `artifacts/findings.md`. Complete the
full investigation and write that file; producing it is the entire job,
and nothing you do outside it has any effect. Create the directory first
with `mkdir -p artifacts`.

The issue and your findings comment live in `ISSUE_REPO` (passed in the
The issue you are investigating lives in `ISSUE_REPO` (passed in the
prompt); `gh` defaults to it, so use plain `gh issue`/`gh pr`/`gh search`
commands. The working tree is checked out from `CODE_REPO`; use that
when building source links (blob/permalink URLs).
Expand Down Expand Up @@ -59,11 +60,11 @@ Key tools at your disposal:
`$(go env GOMODCACHE)/<module>@<version>/`
- **Output**: Write tool (to the workspace directory and below)

NOT available: curl (use fetch-url or WebFetch instead), gh api,
gh issue comment, gh pr comment, rm, sed, xargs, or any
repo-modifying tool. Write your findings to `artifacts/findings.md`
(create the directory with `mkdir -p artifacts` first) — do not
attempt to post comments directly.
This is a read-only environment. NOT available: curl (use fetch-url or
WebFetch instead), gh api, rm, sed, xargs, or any repo-modifying tool.
Anything that would write back to GitHub is blocked. Your only durable
output is the `artifacts/findings.md` file you write with the `Write`
tool (create the directory with `mkdir -p artifacts` first).

## Confidence and Tone

Expand Down Expand Up @@ -395,10 +396,6 @@ valuable during this investigation but were not available. Examples:
a specific command that was blocked, a log or artifact source that
was inaccessible, an API you couldn't query, etc. This helps
improve future investigation runs.

---

Was this investigation helpful? Leave a 👍 or 👎 on this comment.
```

Important:
Expand All @@ -410,6 +407,12 @@ Important:
ones).
- If you cannot determine the failure SHA, investigate using the
default branch and note this limitation.
- End the findings with a link to the workflow run (from the
WORKFLOW RUN variable passed in the prompt), followed by the
feedback footer (the `---` / reaction prompt from the template).

Finishing up — the report file is the whole point:

- Writing `artifacts/findings.md` with the `Write` tool is your last and
most important action. The report only counts if it is in that file;
a report saved anywhere else (e.g. under `/tmp`), or left only in your
final message, does not count and is discarded.
- Before finishing, confirm the file is in place, e.g. `ls -l
artifacts/findings.md`.
56 changes: 54 additions & 2 deletions .github/workflows/investigate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,7 @@ jobs:
fi

- name: Investigate
id: investigate
uses: cockroachdb/claude-code-action@v1
env:
ANTHROPIC_VERTEX_PROJECT_ID: ${{ env.HAS_API_KEY != 'true' && 'vertex-model-runners' || '' }}
Expand Down Expand Up @@ -299,6 +300,48 @@ jobs:
at the default-branch tip instead. Do not run `git checkout` — the
agent's git credentials cannot fetch from CODE_REPO.

# The agent is instructed to write its report to artifacts/findings.md,
# which the steps below upload and post. It occasionally finishes a
# successful investigation without writing the file — e.g. it tries to
# post the comment itself, which the sandbox denies, and gives up. Its
# final summary is still captured in the action's execution-output JSON,
# so recover it into artifacts/findings.md as a fallback. This keeps a
# single source of truth for the upload and post steps, and means a
# missing file reflects a genuinely empty investigation rather than the
# agent delivering its findings the wrong way.
- name: Recover findings from agent output
if: always()
env:
OUTPUT_FILE: ${{ steps.investigate.outputs.execution_file }}
run: |
set -uo pipefail

if [ -s artifacts/findings.md ]; then
exit 0
fi
if [ -z "${OUTPUT_FILE:-}" ] || [ ! -s "$OUTPUT_FILE" ]; then
echo "No execution output file to recover findings from."
exit 0
fi

# The file is a JSON array of turn objects; the agent's final summary
# is the `result` field of the terminating `result` turn.
summary=$(jq -r '[.[] | select(.type? == "result") | .result? // empty] | last // empty' \
"$OUTPUT_FILE" 2>/dev/null || true)
if [ -z "$summary" ]; then
echo "Execution output contained no result summary to recover."
exit 0
fi

mkdir -p artifacts
{
echo "> **Note:** Recovered from the agent's final message; the"
echo "> investigation did not write \`artifacts/findings.md\` directly."
echo
echo "$summary"
} > artifacts/findings.md
echo "Recovered findings from execution output."

- name: Upload findings
if: always()
uses: actions/upload-artifact@v4
Expand All @@ -312,14 +355,23 @@ jobs:
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
run_url="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
if [ -s artifacts/findings.md ]; then
# Append the run link and feedback footer here rather than asking
# the agent to write them, so the prompt never has to describe the
# report as a posted comment (which tempted it to post directly).
{
cat artifacts/findings.md
printf '\n\n---\n\n'
printf '[Workflow run](%s) · Was this investigation helpful? Leave a 👍 or 👎 on this comment.\n' "$run_url"
} > "${RUNNER_TEMP:-/tmp}/comment_body.md"
gh issue comment "$ISSUE_NUMBER" \
--repo ${{ github.repository }} \
--body-file artifacts/findings.md
--body-file "${RUNNER_TEMP:-/tmp}/comment_body.md"
else
gh issue comment "$ISSUE_NUMBER" \
--repo ${{ github.repository }} \
--body "Investigation did not produce findings. Check the [workflow run](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}) for details."
--body "Investigation did not produce findings. Check the [workflow run]($run_url) for details."
fi

- name: Clean up EngFlow certificates
Expand Down
Loading