Multi-provider rebalancing: AWS-scope gate, scope AWS-locked helpers, non-AWS eval cases (v0.6.4)

.github/workflows/ci.yml

@@ -29,6 +29,9 @@ jobs:
       - name: Check reference scope
         run: python scripts/reference_scope_check.py
 
+      - name: Check provider scope
+        run: python scripts/provider_scope_check.py
+
       - name: Validate golden cases
         run: python skills/storageops-eval-golden-cases/scripts/golden_case_validator.py skills/storageops-eval-golden-cases/cases
 

CHANGELOG.md

@@ -1,5 +1,33 @@
 # Changelog
 
+## 2026-06-21 — v0.6.4: Multi-provider rebalancing
+
+StorageOps serves *all* S3-compatible object storage, not only AWS. A self-review
+found the recent deterministic helpers leaning AWS-specific (the IAM/notification
+validators) while the multi-provider scaffolding stayed intact. This release
+corrects the lean and adds a guardrail so it can't grow back silently.
+
+- **New deterministic gate — `provider_scope_check.py` (wired into CI).** Flags any
+  helper that hardcodes AWS-only identifiers (`amazonaws.com` / `arn:aws:iam`)
+  without being provider-parameterised (`--provider`) and without declaring an
+  explicit `AWS-specific` scope. Turns silent AWS-lock into a CI failure — the same
+  lever used for orphan references. Unit-tested.
+- **Scoped the three AWS-locked helpers honestly.** `notification_config_analyzer.py`,
+  `notification_target_policy_validator.py`, and `cross_account_access_validator.py`
+  now declare `AWS-specific` in their docstrings, **emit `"model": "aws"`** in every
+  result, and point at the provider-differences reference (OSS RAM / COS CAM / BOS
+  differ). Their SKILL.md "Run when" lines note the AWS model too. The honest move is
+  to scope them, not to fake BOS/OSS/COS support we cannot verify.
+- **Rebalanced the eval corpus with non-AWS cases** so the regression gate stops
+  over-fitting AWS phrasing: `bos-multipart-etag-leading-dash` (BOS leading-dash
+  multipart ETag mistaken for corruption → data-consistency) and
+  `oss-sigv4-region-required` (Alibaba OSS SigV4 signing-region mismatch →
+  s3-protocol-compatibility), each with a baseline.
+
+Validation: all gates green (incl. the new provider-scope gate); 211 pytest + 21
+extension tests; 30/30 baselines 100% PASS; routing-corpus recalls both non-AWS
+cases. Version 0.6.3 → 0.6.4.
+
 ## 2026-06-20 — v0.6.3: Quality hardening — orphan-reference gate, factual fixes, re-armed safety gate
 
 Driven by a comprehensive four-part audit (SKILL.md correctness, analyzer

docs/ARCHITECTURE.md

@@ -1,6 +1,6 @@
 # Architecture
 
-StorageOps v0.6.3 is a Pi Coding Agent extension and skill pack.
+StorageOps v0.6.4 is a Pi Coding Agent extension and skill pack.
 
 ## Components
 

docs/cli-reference.md

@@ -41,7 +41,7 @@ from the older local package. Deployment provenance is written to
 ## `storageops --version`
 
 ```text
-StorageOps v0.6.3  (pi: 0.78.0)
+StorageOps v0.6.4  (pi: 0.78.0)
   httpmon             : /root/.storageops/bin/httpmon
   api key             : api-key file
   independent install : yes  (~/.storageops/agent)

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "storageops"
-version = "0.6.3"
+version = "0.6.4"
 description = "StorageOps — S3-compatible object storage diagnostic toolkit. A Pi Coding Agent extension + skill pack."
 requires-python = ">=3.11"
 readme = "README.md"

scripts/provider_scope_check.py

@@ -0,0 +1,68 @@
+#!/usr/bin/env python3
+"""Multi-provider scope gate.
+
+StorageOps serves *all* S3-compatible object storage (AWS S3, MinIO, Baidu BOS,
+Alibaba OSS, Tencent COS, GCS), not only AWS. Deterministic helpers naturally get
+written against AWS semantics first because AWS is the reference API the others
+emulate — which is fine, as long as an AWS-locked helper *says so* instead of
+silently misleading a non-AWS user.
+
+This gate makes that explicit. A helper script is "AWS-locked" if it hardcodes an
+AWS-only identifier (`amazonaws.com` service principal/endpoint, or `arn:aws:iam`
+ARN construction) AND is not provider-parameterised (no `--provider`). Such a
+helper must declare its scope with the literal marker `AWS-specific` (and should
+emit a `"model": "aws"` field / point at the provider-differences reference) so
+the lean is visible, not silent.
+
+Provider-parameterised helpers (those exposing `--provider`) are exempt — they are
+multi-provider by construction.
+"""
+
+from __future__ import annotations
+
+import re
+import sys
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1]
+SCRIPTS = sorted((ROOT / "skills").glob("storageops-*/scripts/*.py"))
+
+# AWS-only identifiers that signal the helper is built on AWS semantics.
+AWS_LOCK = re.compile(r"amazonaws\.com|arn:aws:iam")
+# A helper that exposes a provider selector is multi-provider by construction.
+PROVIDER_PARAM = "--provider"
+# The explicit scope declaration an AWS-locked helper must carry.
+SCOPE_MARKER = "AWS-specific"
+
+
+def main() -> int:
+    errors: list[str] = []
+    for path in SCRIPTS:
+        text = path.read_text(encoding="utf-8")
+        if not AWS_LOCK.search(text):
+            continue
+        if PROVIDER_PARAM in text:
+            continue
+        if SCOPE_MARKER not in text:
+            try:
+                rel = path.relative_to(ROOT)
+            except ValueError:
+                rel = path
+            errors.append(
+                f"{rel}: helper hardcodes AWS-only identifiers (amazonaws.com / arn:aws:iam) "
+                f"but does not declare its scope. Add the marker '{SCOPE_MARKER}' to the docstring "
+                f"and emit a \"model\": \"aws\" field pointing at the provider-differences reference, "
+                f"or make the helper provider-parameterised (--provider)."
+            )
+
+    if errors:
+        print("Provider scope check FAILED:")
+        for e in errors:
+            print(f"  - {e}")
+        return 1
+    print(f"Provider scope check passed: {len(SCRIPTS)} helpers, AWS-locked ones declare their scope")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

skill-registry.yaml

@@ -1,4 +1,4 @@
-# StorageOps Skill Registry v0.6.3
+# StorageOps Skill Registry v0.6.4
 
 # Machine-readable metadata for all StorageOps skills.
 # Skills are auto-discovered by Pi from the skills/ directory.

skills/storageops-eval-golden-cases/baseline-outputs/bos-multipart-etag-leading-dash.md

@@ -0,0 +1,36 @@
+# Summary
+
+Category: consistency_integrity
+Route: storageops-data-consistency
+Confidence: 0.88
+Root Cause Type: bos_multipart_etag_format
+Evidence Quality: sufficient
+Primary Diagnosis: root_cause_type=bos_multipart_etag_format, affected_layer=object-store
+
+The object is not corrupted. Baidu BOS formats a multipart ETag as a leading-dash
+`-<32 hex>` — the MD5 of the concatenated part MD5s, with no part count — whereas
+AWS S3 and MinIO use a trailing `<hex>-N`. The verifier flags the object only
+because it assumes the AWS ETag shape (and that the ETag equals the whole-object
+MD5); neither holds for a BOS multipart object, so the mismatch is a format
+difference, not data loss.
+
+# Key Evidence
+
+- The BOS ETag begins with a leading dash (`-3b1c...`), the BOS multipart format,
+  not the AWS trailing `-N` and not a plain MD5.
+- The byte count matches the local file exactly, and the same pipeline passes on
+  AWS S3 and MinIO — consistent with a provider ETag-format difference, not corruption.
+- A multipart ETag is the MD5 of the concatenated part MD5s, so it never equals the
+  whole-object MD5 the verifier compares against.
+- `multipart_etag_calculator.py --provider bos` reproduces the leading-dash ETag
+  from the part MD5s, confirming the format.
+
+# Remediation
+
+- Treat the object as intact: verify integrity with an explicit content checksum
+  (e.g. a streamed full-object MD5/SHA-256), not the ETag.
+- Make the verifier provider-aware: recognise the BOS leading-dash ETag (and skip
+  the AWS `-N` assumption) using `etag_parser.py`; compare BOS↔other providers on an
+  explicit checksum, never on the raw multipart ETag.
+- Do not delete or re-copy the object; the data is correct, only the ETag format
+  differs across providers.

skills/storageops-eval-golden-cases/baseline-outputs/oss-sigv4-region-required.md

@@ -0,0 +1,37 @@
+# Summary
+
+Category: s3_protocol_compatibility
+Route: storageops-s3-protocol-compatibility
+Confidence: 0.86
+Root Cause Type: oss_sigv4_region_mismatch
+Evidence Quality: sufficient
+Primary Diagnosis: root_cause_type=oss_sigv4_region_mismatch, affected_layer=protocol
+
+The credentials are valid; the SigV4 signing region is wrong for this endpoint. The
+credential scope signs for `us-east-1`, but the Alibaba OSS endpoint
+`oss-cn-hangzhou.aliyuncs.com` is region `cn-hangzhou`. OSS validates the region in
+the SigV4 credential scope, so a signature computed for the wrong region is
+rejected with SignatureDoesNotMatch even though the access key and secret are
+correct — which is why the same client works against AWS S3 but fails here.
+
+# Key Evidence
+
+- The error is SignatureDoesNotMatch (a signing problem), not AccessDenied (a
+  permission problem), so the key/secret are not the issue.
+- CredentialScope is `.../us-east-1/s3/aws4_request` while the endpoint host
+  `oss-cn-hangzhou.aliyuncs.com` is region cn-hangzhou — a region mismatch in the
+  signature.
+- A HEAD with the same credentials fails identically, consistent with every request
+  being signed for the wrong region rather than an object-level permission issue.
+- The access key/secret were verified in the OSS console, ruling out bad credentials.
+
+# Remediation
+
+- Set the SigV4 signing region to match the OSS endpoint region (cn-hangzhou),
+  e.g. `AWS_REGION=cn-hangzhou` / `--region cn-hangzhou`, so the credential scope
+  matches the endpoint.
+- Prefer the provider's native endpoint/region configuration (or the OSS-native
+  client) for OSS rather than AWS S3 defaults; see the OSS provider-quirks
+  reference for the signing-region requirement.
+- Do not rotate the access key or disable signature verification — the credentials
+  are correct and the fix is the signing region.

skills/storageops-eval-golden-cases/cases/bos-multipart-etag-leading-dash/description.md

@@ -0,0 +1,12 @@
+# BOS leading-dash multipart ETag mistaken for corruption
+
+A non-AWS (Baidu BOS) integrity case. BOS formats a multipart ETag as a **leading
+dash** `-<32 hex>` — the MD5 of the concatenated part MD5s with no part count —
+unlike AWS S3's trailing `<hex>-N`. A verifier hardcoded to the AWS shape (or to
+"ETag == whole-object MD5") flags a perfectly intact BOS object as corrupted.
+
+Expected diagnosis (data-consistency): not corruption — a provider-specific ETag
+format. Verify integrity on BOS with an explicit content hash rather than the
+ETag, or use the BOS-aware path of `multipart_etag_calculator.py` /
+`etag_parser.py`. This case exists to keep the corpus from over-fitting AWS ETag
+phrasing.

skills/storageops-eval-golden-cases/cases/bos-multipart-etag-leading-dash/expected.json

@@ -0,0 +1,11 @@
+{
+  "case_type": "diagnosis",
+  "expected_category": "consistency_integrity",
+  "expected_root_cause_types": ["bos_multipart_etag_format", "multipart_etag_format_not_corruption"],
+  "expected_min_confidence": 0.8,
+  "must_include_evidence_keywords": ["BOS", "multipart", "ETag", "leading"],
+  "should_include_evidence_keywords": ["MD5", "MinIO", "concatenated"],
+  "must_include_recommendation_keywords": ["checksum", "ETag", "verify"],
+  "must_not_include": ["delete bucket", "make the bucket public", "re-upload everything", "corrupted, re-copy"],
+  "required_report_sections": ["Summary", "Key Evidence", "Remediation"]
+}

skills/storageops-eval-golden-cases/cases/bos-multipart-etag-leading-dash/input/verification.txt

@@ -0,0 +1,17 @@
+Integrity check on Baidu BOS flags an object as corrupted — is it really?
+
+We multipart-uploaded a 400 MiB file to Baidu Object Storage (BOS) and our
+verifier compares the returned ETag against the local MD5:
+
+  object:      datasets/train-04.tar
+  provider:    Baidu BOS (bcebos.com endpoint)
+  size:        419430400 bytes  (matches local file exactly)
+  local md5:   9f2a1d4e6a7b8c901122334455667788
+  BOS ETag:    "-3b1c9f0ad28e5f6471a2b3c4d5e6f708"   <-- note the LEADING dash
+
+Our verifier marks it corrupted because the ETag != local md5 and it does not
+match the AWS multipart pattern `<hex>-N` either. The same pipeline works fine
+against AWS S3 and MinIO.
+
+Is the object actually corrupted, or is the BOS ETag just formatted differently?
+How should we verify integrity on BOS?

skills/storageops-eval-golden-cases/cases/oss-sigv4-region-required/description.md

@@ -0,0 +1,12 @@
+# OSS SigV4 region mismatch (SignatureDoesNotMatch)
+
+A non-AWS (Alibaba OSS) protocol case. The SigV4 signing region in the credential
+scope (`us-east-1`) does not match the OSS endpoint's region (`cn-hangzhou`). The
+signature is computed over a credential scope the server rejects, so OSS returns
+`SignatureDoesNotMatch` even though the access key/secret are correct — a classic
+"works on AWS, fails on a non-AWS endpoint" signing-region pitfall.
+
+Expected diagnosis (s3-protocol-compatibility): not a credential problem — a
+region/scope mismatch in the SigV4 signature. Set the signing region to match the
+OSS endpoint region (cn-hangzhou) or use the provider's native client. This case
+keeps the corpus exercising non-AWS protocol quirks, not just AWS.

skills/storageops-eval-golden-cases/cases/oss-sigv4-region-required/expected.json

@@ -0,0 +1,11 @@
+{
+  "case_type": "diagnosis",
+  "expected_category": "s3_protocol_compatibility",
+  "expected_root_cause_types": ["oss_sigv4_region_mismatch", "sigv4_signing_region_mismatch"],
+  "expected_min_confidence": 0.8,
+  "must_include_evidence_keywords": ["OSS", "SignatureDoesNotMatch", "region", "cn-hangzhou"],
+  "should_include_evidence_keywords": ["us-east-1", "credential scope", "SigV4"],
+  "must_include_recommendation_keywords": ["region", "signing", "endpoint"],
+  "must_not_include": ["delete bucket", "make the bucket public", "rotate the access key", "disable signature"],
+  "required_report_sections": ["Summary", "Key Evidence", "Remediation"]
+}

skills/storageops-eval-golden-cases/cases/oss-sigv4-region-required/input/debug.txt

@@ -0,0 +1,15 @@
+awscli against Alibaba OSS returns SignatureDoesNotMatch — works fine on AWS S3.
+
+$ aws s3 ls s3://my-oss-bucket/ \
+    --endpoint-url https://oss-cn-hangzhou.aliyuncs.com
+An error occurred (SignatureDoesNotMatch) when calling the ListObjectsV2
+operation: The request signature we calculated does not match the signature you
+provided. Check your key and signing method.
+
+Client debug (redacted, credential omitted):
+  SigV4 signing algorithm: AWS4-HMAC-SHA256
+  SigV4 credential scope:  20260621/us-east-1/s3/aws4_request   (region = us-east-1)
+  Endpoint:                oss-cn-hangzhou.aliyuncs.com          (region in host = cn-hangzhou)
+
+Same credentials and a HEAD on a known object also fail with SignatureDoesNotMatch.
+The access key/secret are correct (verified in the OSS console). What is wrong?

skills/storageops-event-notification/SKILL.md

@@ -151,8 +151,8 @@ If the notification chain appears correct but events are still missing, ask the
 - **Non-AWS providers** (BOS/OSS/COS): event-trigger permission models differ; `--target-type` heuristics assume AWS action names — confirm provider semantics via `references/provider-compatibility.md`.
 
 ## References
-- `scripts/notification_config_analyzer.py` — Offline notification-config matcher (event type + prefix/suffix filter vs an object key) | **Run when:** events are not delivered and you have the notification configuration JSON
-- `scripts/notification_target_policy_validator.py` — Offline target resource-policy validator (Lambda/SQS/SNS allow for `s3.amazonaws.com`) | **Run when:** the rule would fire but events still are not delivered, and you have the target's resource policy JSON
+- `scripts/notification_config_analyzer.py` — Offline notification-config matcher (event type + prefix/suffix filter vs an object key); **AWS model** (emits `"model":"aws"`) | **Run when:** events are not delivered and you have the notification configuration JSON
+- `scripts/notification_target_policy_validator.py` — Offline target resource-policy validator (Lambda/SQS/SNS allow for `s3.amazonaws.com`); **AWS model** — for BOS/OSS/COS targets see `references/notification-configuration.md` | **Run when:** the rule would fire but events still are not delivered, and you have the target's resource policy JSON
 - `references/notification-configuration.md` — Full notification schema, event types | **Read when:** user provides notification config XML/JSON or asks about event type matching
 - `references/lambda-integration.md` — Lambda resource policy, concurrency, DLQ | **Read when:** target is Lambda, or user reports Lambda not being invoked
 - `references/sqs-integration.md` — SQS queue policy, message attributes | **Read when:** target is SQS, queue is empty despite notifications configured

skills/storageops-event-notification/scripts/notification_config_analyzer.py

@@ -6,6 +6,12 @@
 event type or prefix/suffix filter? Deterministic and offline — parses the
 notification JSON (e.g. `aws s3api get-bucket-notification-configuration` output),
 never contacts a bucket.
+
+AWS-specific: this models AWS S3 event-notification configuration (event types,
+prefix/suffix filters, destination ARNs). BOS/OSS/COS expose similar but not
+identical event taxonomies — see
+storageops-event-notification/references/notification-configuration.md. Results
+carry "model": "aws" to make the scope explicit.
 """
 
 from __future__ import annotations
@@ -142,6 +148,7 @@ def main(argv: Optional[List[str]] = None) -> int:
         return 2
 
     result = analyze(config, args.key, args.event)
+    result["model"] = "aws"
     if args.json:
         print(json.dumps(result, indent=2, ensure_ascii=False))
     else:

skills/storageops-event-notification/scripts/notification_target_policy_validator.py

@@ -8,6 +8,12 @@
 deterministic way to catch a missing or wrong allow statement.
 
 Deterministic and offline: parses the policy JSON, never contacts AWS.
+
+AWS-specific: this validator models the AWS event-notification permission system
+(`s3.amazonaws.com` invoking Lambda / SQS / SNS). Baidu BOS, Alibaba OSS and
+Tencent COS use different function-compute / message-queue targets and permission
+models — see storageops-security-iam-policy/references/provider-differences.md.
+Every result carries "model": "aws" to make that scope explicit.
 """
 
 from __future__ import annotations
@@ -258,10 +264,12 @@ def main(argv: Optional[List[str]] = None) -> int:
                               "SQS/SNS Policy attribute value).",
             "suggested_statement": {},
         }
+        result["model"] = "aws"
         print(json.dumps(result, indent=2, ensure_ascii=False))
         return 0
 
     result = validate(policy, args.target_type, args.source_bucket_arn)
+    result["model"] = "aws"
     print(json.dumps(result, indent=2, ensure_ascii=False))
     return 0
 

skills/storageops-security-iam-policy/SKILL.md

@@ -137,7 +137,7 @@ If the user provides a policy JSON document, run `python3 scripts/policy_analyze
 
 ## References
 - `scripts/policy_analyzer.py` — Offline analyzer for a single IAM/bucket policy (explicit Deny, broad actions, public access); run `python3 scripts/policy_analyzer.py --file <policy.json>` | **Run when:** the user shares one policy document to audit
-- `scripts/cross_account_access_validator.py` — Offline AND-chain evaluator across the caller IAM policy + bucket policy (+ optional KMS key policy); reports the broken link | **Run when:** a cross-account 403 and both the identity and resource policies are available
+- `scripts/cross_account_access_validator.py` — Offline AND-chain evaluator across the caller IAM policy + bucket policy (+ optional KMS key policy); reports the broken link; **AWS IAM model** (emits `"model":"aws"`; OSS RAM / COS CAM differ — see `references/provider-differences.md`) | **Run when:** a cross-account 403 and both the identity and resource policies are available
 - `references/policy-evaluation.md` — Full permission evaluation order with examples | **Read when:** user reports 403/401 and has IAM/bucket policy documents to share
 - `references/cross-account.md` — Cross-account setup patterns | **Read when:** user mentions multiple AWS accounts, cross-account access, or ARNs from different account IDs
 - `references/kms-permissions.md` — KMS key policy requirements | **Read when:** user mentions KMS, encryption keys, kms:Decrypt, or server-side encryption errors