Skip to content

ci: 让 codecov 上传失败不再拖垮测试 gate#459

Merged
suyiiyii merged 2 commits into
mainfrom
fix/codecov-non-blocking-upload
Jun 28, 2026
Merged

ci: 让 codecov 上传失败不再拖垮测试 gate#459
suyiiyii merged 2 commits into
mainfrom
fix/codecov-non-blocking-upload

Conversation

@suyiiyii

Copy link
Copy Markdown
Owner

问题

所有 dependabot/renovate 的依赖更新 PR 在 CI 上都显示「失败」,但测试本身是通过的。真正失败的是 codecov 上传步骤:

HTTP Error 400 -- {"message":"Token required because branch is protected"}
==> Failed to run upload-coverage  (exit code 1)

机器人发起的 PR 拿不到 CODECOV_TOKEN secret,codecov 上传返回 400,配合 fail_ci_if_error: trueUnit + Contract / Integration / Backend E2E 三个 coverage job 全部变红,整条依赖更新 PR 队列被卡住无法合并。

改动

codecov.yml 里三处 fail_ci_if_error: true 改为 false,与已有的 coverage-e2e-frontend.yml(本就是 false,所以它在依赖 PR 上一直是绿的)保持一致。

覆盖率仍会正常上传到 codecov,只是上传失败不再阻断测试 gate——测试通过与否仍由 pytest 决定。

影响

解锁被卡住的依赖更新 PR 队列:#458 #457 #454 #453 #451 #450 #448 #430 #435 等。

机器人(dependabot/renovate)发起的 PR 拿不到 CODECOV_TOKEN secret,
codecov 上传返回 400 'Token required because branch is protected',
配合 fail_ci_if_error: true 导致 Unit/Integration/Backend E2E 三个
coverage job 全部变红, 整条依赖更新 PR 队列被卡住无法合并.

将三处 fail_ci_if_error 改为 false, 与已有的
coverage-e2e-frontend.yml 写法保持一致. 覆盖率上传仍会执行,
失败只是不再阻断测试 gate.
@vercel

vercel Bot commented Jun 21, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
autoglm-gui Ready Ready Preview, Comment Jun 21, 2026 4:58pm

@claude

claude Bot commented Jun 21, 2026

Copy link
Copy Markdown

Claude finished @suyiiyii's task in 59s —— View job


Review progress

  • Read CLAUDE.md and PR context
  • Review diff and surrounding workflow files
  • Post review feedback

Findings

No blocking issues found.

The change is minimal, well-scoped, and matches the stated problem: CODECOV_TOKEN is unavailable to Dependabot/Renovate PRs, so codecov/codecov-action@v6 returns 400 and, with fail_ci_if_error: true, turns the coverage jobs red even though the actual tests passed. Switching the three backend coverage jobs to fail_ci_if_error: false aligns them with the existing frontend coverage workflow (.github/workflows/coverage-e2e-frontend.yml:78), which already uses the non-blocking setting.

Optional improvement (P3)

If you want to keep the safety net for human/internal PRs while still allowing bot PRs to pass, consider making the failure behavior conditional rather than uniformly false:

fail_ci_if_error: ${{ secrets.CODECOV_TOKEN != '' }}

This way, when the token is present and upload fails (expired token, codecov outage, misconfiguration), the job still fails and you notice it. When the token is absent (Dependabot/Renovate/forks), upload failure is silently tolerated. Using the token presence check is more robust than actor-name checks because it also covers external forks where secrets are not available.

Residual risk

With fail_ci_if_error: false, a prolonged codecov outage or token misconfiguration on regular PRs will not be caught by CI. Coverage data may be missing from codecov without anyone noticing until they check the dashboard. Given the current pain point with the dependency-update queue, this is a reasonable trade-off.


@codecov

codecov Bot commented Jun 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.93%. Comparing base (c5ffef2) to head (22e9754).
⚠️ Report is 1 commits behind head on main.

@@            Coverage Diff             @@
##             main     #459      +/-   ##
==========================================
- Coverage   85.19%   82.93%   -2.27%     
==========================================
  Files         114      116       +2     
  Lines       11561    12250     +689     
==========================================
+ Hits         9849    10159     +310     
- Misses       1712     2091     +379     
Flag Coverage Δ
e2e-backend 33.35% <ø> (-0.21%) ⬇️
e2e-frontend 0.61% <ø> (-0.04%) ⬇️
integration 39.10% <ø> (-0.16%) ⬇️
unit 81.31% <ø> (-2.43%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 24 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c5ffef2...22e9754. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

🚀 Lint & Format Check Results: ✅ PASSED

What was checked:

  • 🐍 Backend: Ruff linting + formatting
  • 📱 Frontend: ESLint + Prettier + TypeScript types
  • 🔧 Unified: Custom lint script validation

✅ All checks passed!

Your code follows the project's style guidelines.

Checks performed:

  • Backend Ruff lint ✅
  • Backend Ruff format ✅
  • Frontend ESLint ✅
  • Frontend Prettier format ✅
  • Frontend TypeScript types ✅
  • Unified lint script ✅

This is an automated check. For questions, ask in the PR discussion.

abort 用例注释假设 'mock LLM 永不 finish 所以任务一直 RUNNING',
但测试 config 没设 default_max_steps, 后端默认上限为 100 步.
mock LLM 是本地 round-robin, 每步几乎瞬时, 100 步会很快跑完,
任务到达终态后 loading 置 false, Abort 按钮(loading && 渲染)消失.
慢的 Windows runner 上, 'Step 1 出现'到'断言按钮'的窗口里时常
正好撞上任务跑完 -> getByTitle('Abort Chat') element not found.

修复: 在 config 里设 default_max_steps=null(不限制), 任务持续
RUNNING 直到被 abort, 符合用例本意; 按钮断言超时 5s -> 15s 给慢
runner 留余量.
@claude

claude Bot commented Jun 21, 2026

Copy link
Copy Markdown

Claude finished @suyiiyii's task in 2m 13s —— View job


Review progress

  • Read CLAUDE.md and PR context
  • Review diff and surrounding workflow files
  • Review updated frontend/e2e/abort-task.spec.ts
  • Post review feedback

Findings

No blocking issues found.

The PR contains two independent, well-scoped changes:

  1. CI change (.github/workflows/codecov.yml:42, :76, :113)
    Changes fail_ci_if_error: truefalse for the three backend coverage upload jobs. This directly addresses the problem described in the PR body: Dependabot/Renovate PRs cannot access CODECOV_TOKEN, so codecov returns 400 and the coverage jobs turn red even though pytest passed. This now matches the existing frontend coverage workflow (.github/workflows/coverage-e2e-frontend.yml), which already uses fail_ci_if_error: false.

  2. E2E test hardening (frontend/e2e/abort-task.spec.ts:168-174, :218)

    • Adds default_max_steps: null to the config payload. The schema accepts int | None (AutoGLM_GUI/schemas.py:313) and None means unlimited (config_manager.py treats None as no cap). This prevents the mock LLM's round-robin tap loop from hitting the default 100-step cap on fast runners and reaching a terminal status before the abort button is clicked, which was the source of the Windows flakiness.
    • Increases the abort-button visibility timeout from 5000 ms to 15000 ms, giving slower/Windows runners enough time for the UI to render the loading state.

Both changes are minimal and correct. The test still has bounded runtime: waitForTask enforces a 30-second deadline, and Playwright's own test timeout will catch any runaway scenario.

Residual risk

  • With fail_ci_if_error: false, a real codecov outage or token misconfiguration on regular PRs will no longer fail CI, so missing coverage uploads could go unnoticed until someone checks the codecov dashboard. This is a reasonable trade-off for unblocking the dependency-update queue.
  • Setting default_max_steps: null means the task will never self-terminate via the step cap; the test relies entirely on the abort path and the existing timeouts to finish. If the abort mechanism regressed, the test would fail via timeout rather than a quick step-cap exit, slightly increasing failure diagnosis time.

@suyiiyii
suyiiyii merged commit 8ff2857 into main Jun 28, 2026
18 checks passed
@suyiiyii
suyiiyii deleted the fix/codecov-non-blocking-upload branch June 28, 2026 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant