fix(ci): resolve ci-self-test file issue and live eval failures#39
Closed
YiWang24 wants to merge 0 commit into
Closed
fix(ci): resolve ci-self-test file issue and live eval failures#39YiWang24 wants to merge 0 commit into
YiWang24 wants to merge 0 commit into
Conversation
ⓘ You've reached your Qodo monthly free-tier limit. Reviews pause until next month — upgrade your plan to continue now, or link your paid account if you already have one. |
📝 WalkthroughWalkthroughUpdate many GitHub Actions reusable-workflow pins to a new commit SHA, adjust one checkout step to pass an explicit token, add Anthropic env/model fallback in live eval steps, modify runner/label expectations and test assertions, tighten .claude allowed commands, and fix small test script shell issues. ChangesReusable workflow pins, resolve-openci, and manifest
Local CI config, runner labels, tests, and scripts
Sequence Diagram(s)(Skipped — changes are configuration, pinning, tests, and small scripting fixes without a new multi-component runtime control flow that requires visualization.) Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
⚔️ Resolve merge conflicts
Review rate limit: 9/10 reviews remaining, refill in 6 minutes. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/ci-self-test.yml:
- Line 45: The reusable workflow reference in the `uses` step points to a
subdirectory path which GitHub Actions disallows; move the reusable workflow
file into the top-level workflows directory and update the `uses:
YiAgent/OpenCI/.github/workflows/reusable/self-test.yml@be43e4efd2f14f2a3da7d5264356a9e6774c8ef1`
entry to reference the new top-level file name (e.g., `uses:
YiAgent/OpenCI/.github/workflows/self-test.yml@...`) so the workflow is
loadable; keep the same commit/sha portion and ensure the moved file name
matches the `uses` entry.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: d59ea458-652a-40b1-82d3-936da213cc32
📒 Files selected for processing (4)
.github/workflows/ci-self-test.yml.github/workflows/on-main-bump-sha.yml.github/workflows/test.ymltests/actions/self-test-routing.bats
YiWang24
force-pushed
the
fix/reusable-workflow-paths
branch
from
32995c0 to
7d191a7
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.claude/settings.local.json:
- Around line 74-85: The wildcard entry "Bash(bash *)" in the list loosens the
policy and nullifies the previous tight allowlist (e.g., "Bash(bash
/Users/.../bump-self-sha.sh --dry-run)" and "Bash(bash
.github/scripts/verify-sha-consistency.sh)"); remove the wildcard or replace it
with explicit allowed invocations (exact commands or a precise path pattern) so
only the intended scripts (bump-self-sha.sh, verify-sha-consistency.sh, etc.)
are permitted and arbitrary repo scripts or inline shells cannot be run.
- Line 86: The entry "Bash(gh secret *)" grants too-broad GitHub CLI
permissions; change this to "Bash(gh secret list)" so only read/list access is
allowed. Locate the settings entry containing the string "Bash(gh secret *)" and
replace it with "Bash(gh secret list)" to restrict the scope to listing secrets
only.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 6364a42f-1ac9-4ae0-b773-a9ea1be45f02
📒 Files selected for processing (22)
.claude/settings.local.json.github/actionlint.yaml.github/workflows/agent.yml.github/workflows/ci-self-test.yml.github/workflows/ci.yml.github/workflows/dependencies.yml.github/workflows/deploy.yml.github/workflows/docs.yml.github/workflows/issue-ops.yml.github/workflows/observability.yml.github/workflows/on-main-bump-sha.yml.github/workflows/on-maintenance.yml.github/workflows/pull-request.yml.github/workflows/release.yml.github/workflows/reusable/ci.yml.github/workflows/test.ymlmanifest.ymltests/actions/on-issue-routing.batstests/actions/on-pr-routing.batstests/actions/self-test-routing.batstests/e2e/live-e2e-verify.shtests/e2e/run-live-tests.sh
💤 Files with no reviewable changes (1)
- tests/e2e/run-live-tests.sh
✅ Files skipped from review due to trivial changes (7)
- .github/workflows/pull-request.yml
- .github/workflows/docs.yml
- .github/workflows/ci-self-test.yml
- .github/workflows/ci.yml
- .github/workflows/on-main-bump-sha.yml
- .github/workflows/dependencies.yml
- tests/e2e/live-e2e-verify.sh
🚧 Files skipped from review as they are similar to previous changes (1)
- .github/workflows/test.yml
Comment on lines
+74
to
+85
| "Bash(chmod +x /Users/wy/projects/yiagent/OpenCI/scripts/bump-self-sha.sh)", | ||
| "Bash(bash /Users/wy/projects/yiagent/OpenCI/scripts/bump-self-sha.sh --dry-run)", | ||
| "Bash(sed -i s/ebe8fca3260dce68d34d51b74703169e776bc72d/__TRACKED_VAR__/ manifest.yml)", | ||
| "Bash(bash .github/scripts/verify-sha-consistency.sh)", | ||
| "Bash(grep -n \"^jobs:\\\\|^ [a-z].*:\\\\s*$\\\\|needs:\" /Users/wy/projects/yiagent/OpenCI/.github/workflows/reusable/pr.yml)", | ||
| "Bash(grep -n \"^ [a-z].*:\\\\s*$\\\\|needs:\\\\|gate\\\\|summary\" /Users/wy/projects/yiagent/OpenCI/.github/workflows/reusable/pr.yml)", | ||
| "Bash(command -v actionlint)", | ||
| "Bash(echo \"exit: $?\")", | ||
| "Bash(sed -n '25,45p' .github/workflows/docs.yml)", | ||
| "Bash(sed -n '35,65p' .github/workflows/deploy.yml)", | ||
| "Bash(sed -n '115,130p' .github/workflows/on-maintenance.yml)", | ||
| "Bash(bash *)", |
There was a problem hiding this comment.
Bash(bash *) undoes the tightening in this block.
The explicit bump-self-sha.sh --dry-run and verify-sha-consistency.sh entries above become moot once arbitrary bash invocations are allowed again, because this wildcard can run any repo script or inline shell.
Suggested change
- "Bash(bash *)",📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| "Bash(chmod +x /Users/wy/projects/yiagent/OpenCI/scripts/bump-self-sha.sh)", | |
| "Bash(bash /Users/wy/projects/yiagent/OpenCI/scripts/bump-self-sha.sh --dry-run)", | |
| "Bash(sed -i s/ebe8fca3260dce68d34d51b74703169e776bc72d/__TRACKED_VAR__/ manifest.yml)", | |
| "Bash(bash .github/scripts/verify-sha-consistency.sh)", | |
| "Bash(grep -n \"^jobs:\\\\|^ [a-z].*:\\\\s*$\\\\|needs:\" /Users/wy/projects/yiagent/OpenCI/.github/workflows/reusable/pr.yml)", | |
| "Bash(grep -n \"^ [a-z].*:\\\\s*$\\\\|needs:\\\\|gate\\\\|summary\" /Users/wy/projects/yiagent/OpenCI/.github/workflows/reusable/pr.yml)", | |
| "Bash(command -v actionlint)", | |
| "Bash(echo \"exit: $?\")", | |
| "Bash(sed -n '25,45p' .github/workflows/docs.yml)", | |
| "Bash(sed -n '35,65p' .github/workflows/deploy.yml)", | |
| "Bash(sed -n '115,130p' .github/workflows/on-maintenance.yml)", | |
| "Bash(bash *)", | |
| "Bash(chmod +x /Users/wy/projects/yiagent/OpenCI/scripts/bump-self-sha.sh)", | |
| "Bash(bash /Users/wy/projects/yiagent/OpenCI/scripts/bump-self-sha.sh --dry-run)", | |
| "Bash(sed -i s/ebe8fca3260dce68d34d51b74703169e776bc72d/__TRACKED_VAR__/ manifest.yml)", | |
| "Bash(bash .github/scripts/verify-sha-consistency.sh)", | |
| "Bash(grep -n \"^jobs:\\\\|^ [a-z].*:\\\\s*$\\\\|needs:\" /Users/wy/projects/yiagent/OpenCI/.github/workflows/reusable/pr.yml)", | |
| "Bash(grep -n \"^ [a-z].*:\\\\s*$\\\\|needs:\\\\|gate\\\\|summary\" /Users/wy/projects/yiagent/OpenCI/.github/workflows/reusable/pr.yml)", | |
| "Bash(command -v actionlint)", | |
| "Bash(echo \"exit: $?\")", | |
| "Bash(sed -n '25,45p' .github/workflows/docs.yml)", | |
| "Bash(sed -n '35,65p' .github/workflows/deploy.yml)", | |
| "Bash(sed -n '115,130p' .github/workflows/on-maintenance.yml)", |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.claude/settings.local.json around lines 74 - 85, The wildcard entry
"Bash(bash *)" in the list loosens the policy and nullifies the previous tight
allowlist (e.g., "Bash(bash /Users/.../bump-self-sha.sh --dry-run)" and
"Bash(bash .github/scripts/verify-sha-consistency.sh)"); remove the wildcard or
replace it with explicit allowed invocations (exact commands or a precise path
pattern) so only the intended scripts (bump-self-sha.sh,
verify-sha-consistency.sh, etc.) are permitted and arbitrary repo scripts or
inline shells cannot be run.
| "Bash(sed -n '35,65p' .github/workflows/deploy.yml)", | ||
| "Bash(sed -n '115,130p' .github/workflows/on-maintenance.yml)", | ||
| "Bash(bash *)", | ||
| "Bash(gh secret *)", |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify which `gh secret` subcommands are available before narrowing this entry.
gh secret --help
gh secret list --help
gh secret set --help
gh secret delete --helpRepository: YiAgent/OpenCI
Length of output: 6740
Scope gh secret down to list only.
gh secret * permits both set and delete operations, which exceed the read-only scope needed for workflow/SHA inspection. Use gh secret list instead.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.claude/settings.local.json at line 86, The entry "Bash(gh secret *)" grants
too-broad GitHub CLI permissions; change this to "Bash(gh secret list)" so only
read/list access is allowed. Locate the settings entry containing the string
"Bash(gh secret *)" and replace it with "Bash(gh secret list)" to restrict the
scope to listing secrets only.
7d191a7 to
bd3b300
Compare
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
uses: ./refs for top-level.github/workflows/files, not subdirectories. Replaceduses: ./.github/workflows/reusable/self-test.ymlwith the external SHA ref — same behavior, actually parseable by GitHub.Agentic › Live Claude Evalwas hittingapi.anthropic.comwith the GLM key and getting 401. AddedANTHROPIC_BASE_URLto both live eval env blocks so the GLM proxy is used. Also fall back tovars.AI_MODELfor the eval model name.workflowspermission".GITHUB_TOKENcannot push workflow file changes; switched checkout to useMY_GITHUB_TOKEN(PAT withworkflowscope, sourced from Dopplerinfraproject).Test plan
ci-self-testworkflow now runs successfully on push/PR (instead of "workflow file issue" at 0s)Agentic › Live Claude Evalpasses on push to main using GLM 5.1 viaANTHROPIC_BASE_URLon-main-bump-shaSHA bump workflow can push workflow file changesself-test-routing.batstest for the new external ref pattern)Need help on this PR? Tag
@codesmithwith what you need.Summary by CodeRabbit
Tests
Chores