S3Lab

S3Lab is an offline S3 compatibility and protocol debugging lab for engineers testing local S3-shaped workflows. It provides a local endpoint, request tracing, snapshot and recovery checks, and a Docker-based compatibility harness for building evidence against real clients without using a cloud account.

S3Lab is experimental. Treat only the workflows listed in the checked-in compatibility report as verified. This project is not production object storage, a hosted service, or a broad S3 replacement.

Current Local Surface

The current endpoint supports both path-style and virtual-host-style localhost workflows:

• bucket create, list, head, and delete
• object put, head, get, list, and delete
• basic multipart upload and abort flows
• static local SigV4 credentials: s3lab / s3lab-secret
• presigned object GET and PUT requests
• filesystem-backed storage with WAL recovery, snapshots, restore, and reset
• virtual-host bucket/object routing for configured suffixes (default suffix: localhost)
• deterministic unsupported-subresource error payloads with explicit Resource values for bucket and object paths
• read-only local inspector UI
• baseline/candidate compare across captured exchanges
• structured server logs, access logs, and redacted request traces
• deterministic failure injection for V1 latency and injected 500 / 503 S3 XML errors
• Docker-first compatibility harness for reproducible client evidence

Run Locally

cargo run -- serve

Default local services:

S3 endpoint:  http://127.0.0.1:9000
Inspector UI: http://127.0.0.1:9001

Transport modes

Local default is loopback-only HTTP:

cargo run -- serve

HTTPS can be enabled in two ways:

• --tls-auto to generate and persist a local self-signed certificate under ./s3lab-data/.
• --tls-cert <cert.pem> --tls-key <key.pem> to use explicit PEM material.

cargo run -- serve --tls-auto --port 9000
cargo run -- serve --tls-cert ./server.crt --tls-key ./server.key

When HTTPS is active:

• Endpoint URLs use https://.
• ALPN advertises h2 first, with HTTP/1.1 fallback.
• The startup line shows h2 for HTTPS and h1 otherwise.

Example HTTPS startup output:

S3 endpoint:  https://127.0.0.1:9000 (h1)
Inspector UI: https://127.0.0.1:9001 (h1)
Data dir:     ./s3lab-data
TLS cert:     ./s3lab-data/s3lab-tls-cert.pem
TLS notice:   self-signed certificate generated; import it into client trust stores before HTTPS use

Remote hosts are still disallowed by default and must be explicitly enabled:

cargo run -- serve --allow-remote-host --tls-auto

See docs/SECURITY.md for production-like local/remote usage guidance.

Snapshot and reset commands:

cargo run -- snapshot save baseline
cargo run -- snapshot list
cargo run -- snapshot restore baseline
cargo run -- reset

Compare captured traffic:

cargo run -- compare \
  --record-dir ./recording \
  --baseline http://127.0.0.1:9000 \
  --candidate http://127.0.0.1:9001 \
  --report-dir ./compare-report

The compare command replays the same recording against both targets, reports deterministic status/header/body differences, and writes compare.json plus compare.txt when --report-dir is provided.

Snapshot saves are published only after the copied snapshot state validates, so failed saves do not expose a partial named snapshot.

Failure injection rules are local JSON config and are applied after route resolution and auth acceptance, before storage mutation:

cargo run -- serve --failure-config ./failures.json --seed 42

See Failure Injection for the V1 schema, deterministic seed behavior, and retry-testing examples.

Common development checks:

cargo fmt --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test

Public Compatibility Matrix

This matrix summarizes the latest checked-in report. Pass means every case in that suite passed for that client. Warn means required positive and negative cases passed, but at least one warning remains. Partial means some cases passed and some failed. Fail means the suite records failures for the current known behavior.

Client	Version recorded	Smoke	SigV4	Object correctness	Multipart	Trace	Persistence
AWS CLI	aws-cli/2.9.19	Pass	Warn	Partial	Pass	Pass	Pass
boto3	1.34.116	Pass	Warn	Partial	Pass	Pass	Pass
Go SDK v2	aws-sdk-go-v2-s3-v1.54.3	Pass	Warn	Partial	Pass	Pass	Pass
Java SDK v2	aws-sdk-java-v2-2.25.60	Pass	Warn	Partial	Pass	Pass	Pass
JavaScript SDK v3	aws-sdk-js-v3-3.583.0	Pass	Warn	Partial	Pass	Pass	Pass
rclone	v1.60.1-DEV	Pass	Warn	Partial	Pass	Pass	Pass
Rust SDK	aws-sdk-s3-1.34.0	Pass	Warn	Partial	Pass	Pass	Pass
s5cmd	v0.0.0-dev	Pass	Warn	Partial	Pass	Pass	Pass

The smoke suites verify the local workflow of creating a bucket, putting an object, reading object metadata and content, listing objects, deleting the object, and deleting the bucket. AWS CLI also records a separate get_object_payload smoke case.

The SigV4 suites verify valid auth, invalid secret rejection, presigned GET, and presigned PUT. The wrong_region case is currently recorded as warn.

The object-correctness suites pass key-shape and payload cases, including spaces, slashes, plus signs, percent encoding, Unicode, repeated overwrite, empty objects, binary payloads, 1 MiB payloads, and the current configurable large-object case. Current failures are metadata-related: Cache-Control and Content-Disposition are unsupported, and rclone/s5cmd cannot reliably express some metadata-control checks used by the harness.

The multipart suites pass basic multipart upload and abort workflows for every listed client. Additional S3Lab protocol probes pass duplicate-part, invalid-upload-id, missing-part, and wrong-ETag error cases.

The trace suites pass trace completeness and deterministic replay checks at the harness level. S3Lab does not yet expose a public replay API.

The persistence suites pass restart persistence, forced-kill recovery, and snapshot restore workflows for every listed client.

The application workflow harness includes DVC dataset push/pull and pipeline reproduction checks, plus Restic and Kopia backup repository evidence against the local S3Lab endpoint. Treat those workflows as verified only when they appear in the generated compatibility summary for the Docker run you are using.

Known Limitations

The latest report records these current limitations:

Area	Status	Current behavior
Routing	Partial	Path-style localhost routing and virtual-host-style routing are both covered (default virtual-host suffix is localhost).
Auth	Unsupported	Compatibility smoke tests use static local credentials only.
Auth	Unsupported	Unsigned requests remain accepted for local debugging; strict authentication mode is not implemented.
Auth	Unsupported	Presigned IAM-style temporary session token requests are not supported.
SigV4	Partial	AWS streaming payload markers are accepted as partial payload validation; full streaming chunk-signature validation is not complete.
Object metadata	Partial	Content-Type, Content-Length, and user metadata are verified; Cache-Control and Content-Disposition PUT headers are unsupported.
Object size	Partial	The configurable large-object single-PUT case is capped at 8 MiB.
Replay	Partial	Harness-level deterministic replay evidence exists for one fixed local sequence; there is no public replay API yet.
Failure injection	Partial	Config-file V1 supports deterministic latency and injected 500 InternalError / 503 ServiceUnavailable S3 XML responses for matched local requests.
ACLs	Unsupported	Public-read object ACLs are rejected; only x-amz-acl: private is accepted as a local client compatibility no-op.
Bucket lifecycle	Unsupported	Bucket lifecycle APIs are not implemented.
Bucket policy	Unsupported	Bucket policy APIs are not implemented.
Bucket versioning	Unsupported	Bucket versioning APIs are not implemented; S3Lab stores single-version local object state.
Object Lock	Unsupported	Object Lock headers are rejected because retention and legal hold behavior is not implemented.
Server-side encryption	Unsupported	Server-side encryption request headers are rejected; S3Lab stores local debugging objects without S3-managed encryption semantics.

Unsupported probes are reported explicitly so the project does not imply support by omission.

Project Principles

• offline by default
• no telemetry by default
• no cloud account required for the primary workflow
• deterministic traces, snapshots, and tests where practical
• compatibility claims backed by reproducible evidence
• unsupported or partial behavior documented clearly
• CI-friendly and cross-platform development

License

Licensed under the Apache License, Version 2.0. See LICENSE.