Skip to content

test: stabilize ci benchmark baseline#5908

Open
killagu wants to merge 7 commits intonextfrom
agent/egg-dev/8ea216e0
Open

test: stabilize ci benchmark baseline#5908
killagu wants to merge 7 commits intonextfrom
agent/egg-dev/8ea216e0

Conversation

@killagu
Copy link
Copy Markdown
Contributor

@killagu killagu commented May 1, 2026

Summary

  • make root clean/pretest scripts compatible with utoo workspace execution
  • stabilize cluster app-listen-path socket readiness and cleanup
  • widen schedule stop fixture timing and add tegg manifest fallback generation

Validation

  • ut run clean --workspaces --if-present
  • ut run pretest --workspaces --if-present
  • pnpm exec oxlint --type-aware --type-check --quiet package.json packages/cluster/test/app_worker.test.ts plugins/schedule/test/stop.test.ts tegg/plugin/tegg/test/ManifestCollection.test.ts
  • git diff --check
  • CI=true pnpm exec vitest run tegg/plugin/tegg/test/ManifestCollection.test.ts --bail 1 --retry 0 --testTimeout 20000 --hookTimeout 20000
  • CI=true pnpm exec vitest run plugins/schedule/test/stop.test.ts --bail 1 --retry 0 --testTimeout 30000 --hookTimeout 30000
  • CI=true pnpm exec vitest run packages/cluster/test/app_worker.test.ts --bail 1 --retry 0 --testTimeout 30000 --hookTimeout 30000
  • env CI=true /usr/bin/time -p ut run ci

Summary by CodeRabbit

  • New Features

    • Listen-path socket can be overridden via an environment variable.
  • Tests

    • Improved stability: isolated per-test socket paths, readiness polling for sockets, log-polling helper, null-safe cleanup, centralized manifest lookup, and faster schedule interval (10s → 5s).
  • Chores

    • Added a workspace-level clean script and simplified pretest to run only the local clean step.

Copilot AI review requested due to automatic review settings May 1, 2026 17:16
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 1, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds a workspace script runner and top-level clean script; updates pretest. Tests: cluster tests use unique temp Unix sockets and env-driven socket override with readiness polling; schedule interval shortened and stop test now polls logs and preserves pre-close counts; tegg tests use a memoized manifest-extension helper.

Changes

Cohort / File(s) Summary
Top-level scripts
package.json, scripts/run-workspace-scripts.mjs
Add scripts/run-workspace-scripts.mjs and top-level clean script delegating to it; update pretest to only run clean-dist (remove workspace ut run ... --if-present).
Cluster listen path & tests
packages/cluster/test/app_worker.test.ts, packages/cluster/test/fixtures/apps/app-listen-path/config/config.default.js
Tests now generate unique temp Unix socket paths, remove preexisting socket, inject path via EGG_APP_LISTEN_PATH_SOCKET, and wait for socket readiness before HTTP+UNIX requests; config reads socket path from that env var when present.
Schedule plugin tests
plugins/schedule/test/fixtures/stop/app/schedule/interval.js, plugins/schedule/test/stop.test.ts
Schedule interval changed from 100005000. Stop test replaced direct post-stop interval assertion with a waitForLog poll, records pre-close 'interval' occurrences, closes app (app?.close()), waits, then asserts the count unchanged.
Tegg manifest tests
tegg/plugin/tegg/test/ManifestCollection.test.ts
Introduce memoized getTeggManifestExtension() helper that falls back to app.loader.generateManifest() if loader cache lacks the extension; reset helper cache during setup and update assertions to use helper.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Dev as Developer
  participant Runner as run-workspace-scripts.mjs
  participant WS as pnpm-workspace.yaml
  participant FS as Filesystem
  participant Sub as Subprocess/npm

  Dev->>Runner: invoke script name (e.g. clean)
  Runner->>WS: read workspace patterns
  Runner->>FS: expand patterns & enumerate package dirs
  Runner->>FS: read each workspace `package.json`
  alt workspace has script
    Runner->>Sub: spawn `npm run --silent <script>` (inherit stdio)
    Sub-->>Runner: exit code
    Runner->>Dev: propagate non-zero exit code (fail early)
  else no script found
    Runner->>Dev: print "No workspace scripts found for \"<script>\""
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested labels

chore: gitAction

Suggested reviewers

  • fengmk2
  • jerryliang64

Poem

🐰 I hopped through sockets, scripts, and log,
Clean called runners, tests now wait and knock.
Manifests peek where caches once were thin,
Intervals quicker, logs checked safe within.
A joyful twitch — the rabbit commits with a hop!

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title 'test: stabilize ci benchmark baseline' is vague and does not clearly convey the actual changes made in this PR, which include script reorganization, test fixture modifications, and CI stability improvements. Consider a more specific title like 'test: improve CI test stability and workspace compatibility' that better reflects the actual changes across multiple areas (scripts, cluster tests, schedule tests, tegg tests).
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch agent/egg-dev/8ea216e0

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented May 1, 2026

Deploying egg with  Cloudflare Pages  Cloudflare Pages

Latest commit: f9bd11d
Status: ✅  Deploy successful!
Preview URL: https://2260a69f.egg-cci.pages.dev
Branch Preview URL: https://agent-egg-dev-8ea216e0.egg-cci.pages.dev

View logs

@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented May 1, 2026

Deploying egg-v3 with  Cloudflare Pages  Cloudflare Pages

Latest commit: f9bd11d
Status: ✅  Deploy successful!
Preview URL: https://2bb84efc.egg-v3.pages.dev
Branch Preview URL: https://agent-egg-dev-8ea216e0.egg-v3.pages.dev

View logs

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors test scripts and improves test reliability across several packages, notably by implementing dynamic socket paths in cluster tests and adding a manifest retrieval helper in tegg tests. Feedback recommends caching the manifest extension in the new helper to prevent redundant, expensive generation during test execution.

Comment thread tegg/plugin/tegg/test/ManifestCollection.test.ts
@codecov
Copy link
Copy Markdown

codecov Bot commented May 1, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.03%. Comparing base (101ab97) to head (f9bd11d).

Additional details and impacted files
@@            Coverage Diff             @@
##             next    #5908      +/-   ##
==========================================
- Coverage   85.04%   85.03%   -0.02%     
==========================================
  Files         665      665              
  Lines       19100    19100              
  Branches     3716     3716              
==========================================
- Hits        16244    16242       -2     
- Misses       2463     2465       +2     
  Partials      393      393              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Stabilizes CI benchmark-related tests and workspace scripts by reducing timing flakiness and improving determinism around manifest generation and unix-socket readiness.

Changes:

  • Update root clean / pretest scripts to run reliably under workspace runners by delegating via pnpm -r (excluding the workspace root).
  • Make cluster unix-socket listen-path test deterministic by using a unique tmp socket path and waiting until the socket is connectable before issuing requests.
  • Reduce schedule stop-test flakiness by widening the interval fixture timing and avoiding failures when the log file is not created.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
package.json Adds root clean and adjusts pretest to delegate across workspaces via pnpm -r excluding @eggjs/monorepo.
packages/cluster/test/app_worker.test.ts Adds unix-socket readiness polling and uses a unique tmp socket path for the listen-path test.
packages/cluster/test/fixtures/apps/app-listen-path/config/config.default.js Allows overriding the socket path via EGG_APP_LISTEN_PATH_SOCKET to support deterministic tests.
plugins/schedule/test/stop.test.ts Avoids reading a non-existent log file by checking existence first.
plugins/schedule/test/fixtures/stop/app/schedule/interval.js Increases schedule interval to reduce timing-related flakes.
tegg/plugin/tegg/test/ManifestCollection.test.ts Adds a fallback to generateManifest() when the tegg manifest extension is not present in the loaded manifest.

Copilot AI review requested due to automatic review settings May 1, 2026 17:32
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
scripts/run-workspace-scripts.mjs (1)

42-44: ⚡ Quick win

Make workspace traversal order deterministic

readdirSync() order can vary by filesystem. Sorting here makes clean/pretest execution stable across CI environments.

Deterministic ordering patch
   return readdirSync(baseDir, { withFileTypes: true })
     .filter((entry) => entry.isDirectory())
+    .sort((a, b) => a.name.localeCompare(b.name))
     .map((entry) => join(baseDir, entry.name));
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@scripts/run-workspace-scripts.mjs` around lines 42 - 44, The directory list
produced by readdirSync is non-deterministic across filesystems; in the block
that calls readdirSync(baseDir, { withFileTypes: true }).filter(...).map(...),
sort the Dirent entries by entry.name (e.g., using localeCompare) before mapping
to join(baseDir, entry.name) so workspace traversal (and tasks like
clean/pretest) runs in a stable, deterministic order across CI.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@scripts/run-workspace-scripts.mjs`:
- Around line 42-44: The directory list produced by readdirSync is
non-deterministic across filesystems; in the block that calls
readdirSync(baseDir, { withFileTypes: true }).filter(...).map(...), sort the
Dirent entries by entry.name (e.g., using localeCompare) before mapping to
join(baseDir, entry.name) so workspace traversal (and tasks like clean/pretest)
runs in a stable, deterministic order across CI.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b7649e46-0eab-4be8-ab25-46939e3e87b7

📥 Commits

Reviewing files that changed from the base of the PR and between c1db165 and 9b7597f.

📒 Files selected for processing (3)
  • package.json
  • scripts/run-workspace-scripts.mjs
  • tegg/plugin/tegg/test/ManifestCollection.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • package.json
  • tegg/plugin/tegg/test/ManifestCollection.test.ts

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

Comment thread scripts/run-workspace-scripts.mjs
Comment thread packages/cluster/test/app_worker.test.ts Outdated
Comment thread plugins/schedule/test/stop.test.ts Outdated
Comment thread plugins/schedule/test/fixtures/stop/app/schedule/interval.js Outdated
Comment thread tegg/plugin/tegg/test/ManifestCollection.test.ts
Copilot AI review requested due to automatic review settings May 1, 2026 18:14
Comment thread plugins/schedule/test/stop.test.ts Fixed
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@plugins/schedule/test/stop.test.ts`:
- Around line 34-46: The test captures a baseline before the interval callback
runs because it waits for 'egg-schedule.log' loader entry; change it to wait for
the actual "interval" log in stop-web.log before closing the app so
beforeCloseCount reflects a real interval execution (use waitForLog(logPath,
'interval', longerTimeout) or equivalent), then proceed to call await
app!.close(); keep the later sleep/assert logic but increase the polling/timeout
value to be longer than the 5s schedule interval to avoid flaky failures.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 51db55da-83a7-4c0e-89c8-d19abd113800

📥 Commits

Reviewing files that changed from the base of the PR and between c2cec14 and a50a553.

📒 Files selected for processing (4)
  • packages/cluster/test/app_worker.test.ts
  • plugins/schedule/test/fixtures/stop/app/schedule/interval.js
  • plugins/schedule/test/stop.test.ts
  • scripts/run-workspace-scripts.mjs
🚧 Files skipped from review as they are similar to previous changes (2)
  • plugins/schedule/test/fixtures/stop/app/schedule/interval.js
  • scripts/run-workspace-scripts.mjs

Comment thread plugins/schedule/test/stop.test.ts Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Comment thread plugins/schedule/test/stop.test.ts Outdated
Copilot AI review requested due to automatic review settings May 1, 2026 18:40
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants