Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,31 @@
# Changelog

## 2026-06-21 — v0.6.7: Routing robustness

Closes the routing gaps flagged in the v0.6.2 audit and deferred since. Changes are
to `detect_domain` only, guarded by the routing-corpus (every golden case must still
recall its skill top-2) and the negative-corpus (false positives must not route).

- **Constrained the bare `cost` signature.** `[/cost/i]` matched "cost center", "at
all costs", "cost of downtime" and misrouted unrelated text to lifecycle-cost. It
now requires a storage-cost neighbour token (storage/request/egress/transition/
tier/bill/lifecycle/retrieval/IA/Glacier/Archive/GB/TB) — mirroring the already-
guarded `sync`/`transfer` patterns. Two negative-corpus cases lock it.
- **Added a `transport` signal to network-endpoint-access**: `RequestTimeout`,
`connection reset` / `reset by peer`, `broken pipe`, `ECONNRESET`, `EPIPE`,
`unexpected EOF` — common mid-transfer failures that previously matched no
signature and so did not route.
- **New golden case** `network-connection-reset-upload` (large multipart PUTs reset
by a new firewall/NAT middlebox; small PUTs and downloads fine → network transport,
not auth), with a baseline — exercises and locks the new `transport` signal.

Deliberately not changed: `NoSuchKey`/`NoSuchBucket` were left unsignalled — they are
genuinely ambiguous (consistency vs CLI vs protocol) and a hard signal would risk
misrouting; that is better handled by triage context than a bare regex.

Validation: all gates green; 219 pytest + 21 extension tests (incl. the 2 new
negative-corpus cases); 32/32 baselines 100% PASS. Version 0.6.6 → 0.6.7.

## 2026-06-21 — v0.6.6: Correctness hardening (helper bug-fix release)

A focused bug-fix release. A correctness audit of the deterministic helpers (the
Expand Down
2 changes: 1 addition & 1 deletion docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Architecture

StorageOps v0.6.6 is a Pi Coding Agent extension and skill pack.
StorageOps v0.6.7 is a Pi Coding Agent extension and skill pack.

## Components

Expand Down
2 changes: 1 addition & 1 deletion docs/cli-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ from the older local package. Deployment provenance is written to
## `storageops --version`

```text
StorageOps v0.6.6 (pi: 0.78.0)
StorageOps v0.6.7 (pi: 0.78.0)
httpmon : /root/.storageops/bin/httpmon
api key : api-key file
independent install : yes (~/.storageops/agent)
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "storageops"
version = "0.6.6"
version = "0.6.7"
description = "StorageOps — S3-compatible object storage diagnostic toolkit. A Pi Coding Agent extension + skill pack."
requires-python = ">=3.11"
readme = "README.md"
Expand Down
2 changes: 1 addition & 1 deletion skill-registry.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# StorageOps Skill Registry v0.6.6
# StorageOps Skill Registry v0.6.7

# Machine-readable metadata for all StorageOps skills.
# Skills are auto-discovered by Pi from the skills/ directory.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Summary

Category: network_endpoint_access
Route: storageops-network-endpoint-access
Confidence: 0.85
Root Cause Type: middlebox_transport_reset
Evidence Quality: sufficient
Primary Diagnosis: root_cause_type=middlebox_transport_reset, affected_layer=network

Large uploads are being reset by something in the network path, not by S3 auth or
storage. The pattern — large multipart PUTs failing mid-transfer with "connection
reset by peer" / "broken pipe" while small PUTs and all downloads succeed, starting
right after a new firewall/NAT appliance was inserted — points to a middlebox
transport limit: an idle/connection timeout, MTU/MSS mismatch, or NAT
connection-tracking limit cutting the long-lived large-upload connections.

# Key Evidence

- Errors are "Connection reset by peer" (ECONNRESET) and "broken pipe" (EPIPE)
mid-transfer — transport resets, not 4xx auth responses.
- Only large/multipart uploads fail; small uploads and all downloads succeed, so the
break correlates with connection duration/size, not credentials or the bucket.
- The failures began when a new firewall/NAT appliance entered the path, and the
same client works from a different network segment — isolating the cause to the
path, not the endpoint or the application.

# Remediation

- Check the middlebox for an idle/connection **timeout** and a NAT
connection-tracking limit that would drop long-lived uploads; raise the timeout or
exempt the storage endpoint on the **firewall**.
- Test for an **MTU**/MSS mismatch (PMTUD black hole): try a lower MTU / enable
MSS clamping on the appliance, or test with a smaller multipart chunk size to see
if resets stop.
- Confirm with a path capture (reset packets / where the RST originates) and by
retrying from a segment that bypasses the appliance. Do not rotate credentials or
change the bucket policy — auth is not the cause.
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Connection-reset / broken-pipe on large uploads (transport)

A network/transport case. Large multipart PUTs are reset mid-transfer
("Connection reset by peer" / "broken pipe") while small PUTs and all downloads
succeed, right after a new firewall/NAT appliance entered the path — the classic
signature of an idle-timeout / MTU / connection-tracking limit on a middlebox, not
an auth or storage problem.

Expected routing: storageops-network-endpoint-access (transport signal). Expected
diagnosis: a path/middlebox transport issue (idle timeout, MTU/MSS clamping, or
NAT conntrack limit) cutting long-lived large-upload connections. Confirms the
v0.6.7 "transport" routing signal (connection reset / broken pipe / RequestTimeout)
routes correctly.
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"case_type": "diagnosis",
"expected_category": "network_endpoint_access",
"expected_root_cause_types": ["middlebox_transport_reset", "connection_reset_path_issue"],
"expected_min_confidence": 0.8,
"must_include_evidence_keywords": ["connection reset", "broken pipe", "large"],
"should_include_evidence_keywords": ["firewall", "small", "downloads"],
"must_include_recommendation_keywords": ["MTU", "timeout", "firewall"],
"must_not_include": ["delete bucket", "make the bucket public", "rotate the access key", "disable TLS"],
"required_report_sections": ["Summary", "Key Evidence", "Remediation"]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
Large uploads intermittently fail with "connection reset by peer" / broken pipe.

Symptoms:
- PUT of large objects (>500 MB, multipart) fails partway through ~30% of the time.
- Client errors alternate between "Connection reset by peer (ECONNRESET)" and
"broken pipe (EPIPE)" mid-transfer.
- Small objects (<5 MB) upload reliably; downloads of any size are fine.
- It started after the network team put a new firewall/NAT appliance in the path.

$ aws s3 cp ./big.tar s3://prod-bucket/big.tar
upload failed: ./big.tar to s3://prod-bucket/big.tar
("Connection reset by peer",)

Endpoint: s3.us-east-1.amazonaws.com (works from a different network segment)
The credentials and bucket policy are unchanged and small PUTs succeed, so this is
not an auth problem. Why do large uploads get reset, and how do we confirm it?
3 changes: 2 additions & 1 deletion storageops_cli/extensions/storageops.ts
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,7 @@ const DOMAIN_SIGNATURES: Record<string, Array<[RegExp, string]>> = {
[/连接(?:失败|超时|被拒)|证书|解析失败|无法访问/i, "network_cjk"],
[/VPC|endpoint|ENDPOINT/i, "endpoint"],
[/host\s*unreachable|no\s*route/i, "route"],
[/RequestTimeout|connection\s*reset|reset\s*by\s*peer|broken\s*pipe|\bECONNRESET\b|\bEPIPE\b|unexpected\s*EOF/i, "transport"],

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require context before treating EOF as transport

When a tool/config parsing error contains unexpected EOF, this bare transport alternative makes network rank ahead of the CLI/SDK skill because both domains score once and network is earlier in DOMAIN_SIGNATURES; for example detectDomain("rclone config failed: unexpected EOF while parsing config file") recommends storageops-network-endpoint-access before storageops-cli-sdk-diagnosis. This sends non-network EOF failures toward endpoint checks unless the pattern is constrained to HTTP/upload/connection context.

Useful? React with 👍 / 👎.

],
"storageops-cli-sdk-diagnosis": [
[/rclone/i, "rclone"],
Expand All @@ -195,7 +196,7 @@ const DOMAIN_SIGNATURES: Record<string, Array<[RegExp, string]>> = {
"storageops-lifecycle-cost": [
[/lifecycle/i, "lifecycle"],
[/Standard_IA|Glacier|Deep_Archive/i, "storage_class"],
[/cost|费用|计费|账单/i, "cost"],
[/\bcost\b.*\b(storage|request|egress|transition|tier|bill|lifecycle|retrieval|IA|Glacier|Archive|GB|TB)\b|\b(storage|request|egress|transition|tier|bill|lifecycle|retrieval|IA|Glacier|Archive|GB|TB)\b.*\bcost\b|费用|计费|账单/i, "cost"],

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep plural storage-cost mentions routable

The new guard only accepts the exact word cost, so common billing reports using plurals no longer match even with the intended storage-cost neighbor present; detectDomain("S3 storage costs increased this month"), "request costs for the bucket increased", and "egress costs are high" now produce no lifecycle-cost hit. This regresses ordinary cost-diagnosis wording that the previous substring pattern caught; allow costs? in the guarded form while still excluding idioms like at all costs.

Useful? React with 👍 / 👎.

[/transition|\bexpir(?:e|es|ed|ation|ing)?\b/i, "transition"],
[/objects.*small|small.*objects/i, "small_objects"],
],
Expand Down
10 changes: 10 additions & 0 deletions tests/extension/routing-negative-corpus.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,16 @@ const NEGATIVE_CASES: NegativeCase[] = [
text: "The archive object failed during upload.",
forbidden: [["storageops-bigdata-pipeline", "bigdata_engine"]],
},
{
name: "cost center is not lifecycle storage cost",
text: "Please tag the upload to the marketing cost center for the quarterly report.",
forbidden: [["storageops-lifecycle-cost", "cost"]],
},
{
name: "at all costs is not lifecycle storage cost",
text: "We must keep the service available at all costs during the migration window.",
forbidden: [["storageops-lifecycle-cost", "cost"]],
},
];

test("routing negative corpus blocks known bare-substring false positives", () => {
Expand Down
Loading