Multi-provider rebalancing: AWS-scope gate, scope AWS-locked helpers, non-AWS eval cases (v0.6.4)#123
Conversation
…l cases (v0.6.4) StorageOps serves all S3-compatible object storage, not only AWS. A self-review found the recent deterministic helpers leaning AWS-specific (IAM/notification validators) while the multi-provider scaffolding stayed intact. This corrects the lean and adds a guardrail. - New gate scripts/provider_scope_check.py (wired into ci.yml): flags any helper that hardcodes AWS-only identifiers (amazonaws.com / arn:aws:iam) without being provider-parameterised (--provider) and without an explicit "AWS-specific" scope declaration. Turns silent AWS-lock into a CI failure. Unit-tested. - Scoped the 3 AWS-locked helpers honestly: notification_config_analyzer.py, notification_target_policy_validator.py, cross_account_access_validator.py now declare AWS-specific in their docstrings, emit "model": "aws" in every result, and point at provider-differences (OSS RAM / COS CAM / BOS differ). SKILL.md "Run when" lines note the AWS model. Honest scoping rather than faking unverifiable BOS/OSS/COS support. - Rebalanced the corpus with non-AWS cases so the gate stops over-fitting AWS phrasing: bos-multipart-etag-leading-dash (BOS leading-dash multipart ETag vs corruption -> data-consistency) and oss-sigv4-region-required (OSS SigV4 signing-region mismatch -> s3-protocol-compatibility), each with a baseline. Validation: all gates green incl. the new provider-scope gate; 211 pytest + 21 extension tests; 30/30 baselines 100% PASS; routing-corpus recalls both non-AWS cases. Version 0.6.3 -> 0.6.4. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QhkyBLeDtBTmvTdVBqgCAV
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ad78c39359
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| provided. Check your key and signing method. | ||
|
|
||
| Client debug (redacted): | ||
| Authorization: AWS4-HMAC-SHA256 Credential=****/20260621/us-east-1/s3/aws4_request, ... |
There was a problem hiding this comment.
Redact the Authorization sample for validation
In the new golden case, this line still matches SECRET_PATTERNS in golden_case_validator.py (Authorization ... AWS4-HMAC-SHA256 \S+). I ran python skills/storageops-eval-golden-cases/scripts/golden_case_validator.py skills/storageops-eval-golden-cases/cases, and it fails on this file before the baseline eval step in .github/workflows/ci.yml, so the added case will make CI red until the header is further redacted/renamed or includes an allowed example marker.
Useful? React with 👍 / 👎.
The oss-sigv4-region-required input contained a literal `Authorization: AWS4-HMAC-SHA256 ...` line which the golden_case_validator secret scanner flags even with the credential masked. Replaced it with a credential-free SigV4 scope description that keeps the region evidence (us-east-1 vs cn-hangzhou). Case still routes to s3-protocol-compatibility and its baseline still scores PASS. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QhkyBLeDtBTmvTdVBqgCAV
StorageOps serves all S3-compatible object storage (AWS S3, MinIO, Baidu BOS, Alibaba OSS, Tencent COS, GCS), not only AWS. A self-review of v0.6.0–v0.6.3 found the recent deterministic helpers leaning AWS-specific (the IAM/notification validators) even though the multi-provider scaffolding (
detect_domainprovider signatures,provider-quirks/, provider-differences references, the 5-provider ETag matrix) stayed intact. This release corrects the lean and adds a guardrail so it can't grow back silently.1. New deterministic gate —
provider_scope_check.py(wired into CI)Flags any helper that hardcodes AWS-only identifiers (
amazonaws.com/arn:aws:iam) without being provider-parameterised (--provider) and without an explicitAWS-specificscope declaration. Turns silent AWS-lock into a CI failure — the same lever used for orphan references in v0.6.3. The genuinely multi-provider helpers (--provideron multipart-ETag, throttle-tuning, access-log) are exempt by construction. Unit-tested (4 cases).2. Scoped the three AWS-locked helpers honestly
notification_config_analyzer.py,notification_target_policy_validator.py, andcross_account_access_validator.pynow:AWS-specificin their docstrings,"model": "aws"in every result, andTheir SKILL.md "Run when" lines note the AWS model too. The honest move is to scope them, not to fake BOS/OSS/COS support we cannot verify — consistent with the project's "mark unverified, don't universalize AWS" stance.
3. Rebalanced the eval corpus with non-AWS cases
So the regression gate stops over-fitting AWS phrasing:
bos-multipart-etag-leading-dash— Baidu BOS leading-dash multipart ETag mistaken for corruption → data-consistencyoss-sigv4-region-required— Alibaba OSS SigV4 signing-region mismatch → s3-protocol-compatibilityEach with a baseline; both verified to route correctly via
detect_domain(routing-corpus top-2).Validation
All gates green incl. the new provider-scope gate; 211 pytest + 21 extension tests; 30/30 baselines 100% PASS; routing-corpus recalls both non-AWS cases. Version 0.6.3 → 0.6.4.
🤖 Generated with Claude Code
Generated by Claude Code