e2e: add test labels and profiles for selective execution - #273
Conversation
Add category/speed/scenario labels to all e2e tests for selective filtering. Introduce profile system with priority-based registration for downstream overrides. Labels: category (lifecycle, configuration, resilience, networking, observability), speed (fast, moderate, slow), scenario (crud, spec-update, drift, ownership, storage, port, security, metadata, failure, recovery). Profiles: smoke (lifecycle+configuration/fast), extended (+resilience/+moderate). Speed labels validated against CI execution times from PR kubernetes-sigs#271. Assisted-by: 🤖 claude-opus-4-6@default
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: cardil The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
kubernetes-prow
Bot
added
needs-ok-to-test
|
Hi @cardil. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Tip We noticed you've done this a few times! Consider joining the org to skip this step and gain Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
✅ Deploy Preview for mcp-lifecycle-operator ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
kubernetes-prow
Bot
added
the
size/L
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughAdds standardized category, scenario, and speed labels to e2e tests, introduces smoke and extended profile selection, and applies selected profile labels during test initialization. ChangesE2E label selection
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant TestMain
participant ResolveProfile
participant ProfileRegistry
participant E2EConfig
TestMain->>ResolveProfile: resolve -profile
ResolveProfile->>ProfileRegistry: lookup selected profile
ProfileRegistry-->>ResolveProfile: return label selectors
ResolveProfile-->>TestMain: return labels
TestMain->>E2EConfig: apply labels
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #273 +/- ##
=======================================
Coverage ? 85.36%
=======================================
Files ? 14
Lines ? 1578
Branches ? 0
=======================================
Hits ? 1347
Misses ? 188
Partials ? 43 🚀 New features to boost your workflow:
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/e2e/lifecycle_test.go`:
- Around line 115-119: Update the scenario label in TestMCPServerUpdatePort from
scenario.CRUD to scenario.SpecUpdate, since the test modifies
s.Spec.Config.Port. Keep the existing feature, category, and speed labels
unchanged.
In `@test/e2e/manager_test.go`:
- Around line 53-56: The affected features lack scenario-dimension labels,
preventing scenario-based selection. Add semantically appropriate scenario.Label
values to TestManagerPodRunning and TestMetricsEndpoint in
test/e2e/manager_test.go, TestMCPHandshake in test/e2e/mcp_handshake_test.go,
and TestNetworkPolicyCreated, TestNetworkPolicyPortUpdate, and
TestNetworkPolicyGarbageCollected in test/e2e/networkpolicy_test.go; extend
test/e2e/framework/labels/scenario/scenario.go only if no existing taxonomy
value fits.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 828b655c-dd41-426b-842c-baabf60dfd2c
📒 Files selected for processing (13)
test/e2e/configuration_test.gotest/e2e/failure_scenarios_test.gotest/e2e/framework/labels/category/category.gotest/e2e/framework/labels/scenario/scenario.gotest/e2e/framework/labels/speed/speed.gotest/e2e/framework/profiles.gotest/e2e/framework/profiles_defaults.gotest/e2e/lifecycle_test.gotest/e2e/main_test.gotest/e2e/manager_test.gotest/e2e/mcp_handshake_test.gotest/e2e/networkpolicy_test.gotest/e2e/reconciliation_test.go
Rename scenario.CRUD to scenario.Deploy for clarity. CRUD is too broad. Deploy means create+verify. Fix TestMCPServerUpdatePort to use scenario.SpecUpdate instead of scenario.Deploy since it modifies Spec.Config.Port. Add package doc to scenario explaining it is an optional secondary label for slicing larger categories. Assisted-by: 🤖 claude-opus-4-6@default
1 participant
Add category/speed/scenario labels to all e2e tests for selective
filtering. Introduce profile system with priority-based registration
for downstream overrides.
Labels (3 dimensions):
category: lifecycle, configuration, resilience, networking, observabilityspeed: fast, moderate, slowscenario: crud, spec-update, drift, ownership, storage, port, security, metadata, failure, recoveryProfiles (named presets):
smoke: lifecycle+configuration / fastextended: +resilience / +moderateUsage:
Speed labels validated against CI execution times from PR #271.
Profile registry supports priority-based registration. Downstream
distributions can add their own profiles (or override upstream ones)
by adding a Go file with
init()callingRegisterProfile()withPriority > 0.Fixes #269
Assisted-by: 🤖 claude-opus-4-6@default
Summary by CodeRabbit
New Features
Tests