diff --git a/.gitignore b/.gitignore index 4ccc5cc..ea529a2 100644 --- a/.gitignore +++ b/.gitignore @@ -23,3 +23,4 @@ wheels/ .venv uv.lock plan.md +site/ diff --git a/.readthedocs.yaml b/.readthedocs.yaml new file mode 100644 index 0000000..ccc1ee6 --- /dev/null +++ b/.readthedocs.yaml @@ -0,0 +1,13 @@ +version: 2 + +build: + os: "ubuntu-22.04" + tools: + python: "3.12" + +python: + install: + - requirements: docs/requirements.txt + +mkdocs: + configuration: mkdocs.yml diff --git a/CLAUDE.md b/CLAUDE.md index c0e4fcb..a7a0375 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,12 +8,11 @@ Guidance for AI agents (Claude Code, etc.) working in this repository. **Where to find what:** -- [`docs/engineering.md`](docs/engineering.md) — the distilled design reference: invariants and *why*, the five protocol seams, exception contract, module layout, testing patterns, optional-extras pattern, remaining roadmap. Read this before adding any new module or extension point. -- [`docs/deferred-work.md`](docs/deferred-work.md) — review-surfaced items that are real but not actionable now. -- [`docs/superpowers/specs/`](docs/superpowers/specs/) and [`docs/superpowers/plans/`](docs/superpowers/plans/) — per-feature design specs and implementation plans (active work). -- [`docs/archive/`](docs/archive/) — historical bmad-era planning bundle (PRD, architecture, epics, product briefs, per-story specs for 1-1 through 1-5). Consult only for original rationale or specific FR/NFR citations. +- [`docs/dev/engineering.md`](docs/dev/engineering.md) — the distilled design reference: invariants and *why*, the five protocol seams, exception contract, module layout, testing patterns, optional-extras pattern, remaining roadmap. Read this before adding any new module or extension point. +- [`planning/deferred-work.md`](planning/deferred-work.md) — review-surfaced items that are real but not actionable now. +- [`planning/specs/`](planning/specs/) and [`planning/plans/`](planning/plans/) — per-feature design specs and implementation plans (active work). -**Per-feature workflow:** brainstorming → spec in `docs/superpowers/specs/` → writing-plans → plan in `docs/superpowers/plans/` → executing-plans (or subagent-driven-development) → requesting-code-review → finishing-a-development-branch. Topic slugs are kebab-case descriptions (`msgspec-decoder-adapter`), not story IDs. +**Per-feature workflow:** brainstorming → spec in `planning/specs/` → writing-plans → plan in `planning/plans/` → executing-plans (or subagent-driven-development) → requesting-code-review → finishing-a-development-branch. Topic slugs are kebab-case descriptions (`msgspec-decoder-adapter`), not story IDs. ## Commands @@ -99,5 +98,5 @@ Five documented internal boundaries. AI agents must respect them — never cross ## When in doubt -- Check [`docs/engineering.md`](docs/engineering.md) before adding a new module or extension point; `docs/archive/architecture.md` has the deeper historical rationale if needed. +- Check [`docs/dev/engineering.md`](docs/dev/engineering.md) before adding a new module or extension point. - Surface ambiguity as a documentation gap rather than improvising. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f3c5b55..0bfbb37 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,48 +1,6 @@ -# Contributing to httpware +# Contributing -Thank you for your interest in contributing. `httpware` is an open-source resilience-first async HTTP client framework for Python, maintained under the [`modern-python`](https://github.com/modern-python) org. +The contributing guide is published as part of the project documentation: +**https://httpware.readthedocs.io/en/latest/dev/contributing/** -## Quick start - -```bash -git clone https://github.com/modern-python/httpware.git -cd httpware -just install # uv lock --upgrade && uv sync --all-extras --frozen --group lint -just lint # ruff format + ruff check + ty check -just test # pytest with coverage -``` - -## Development workflow - -1. **Open an issue first** for non-trivial changes — design discussion catches issues earlier than code review. -2. **Branch from `main`**, use a descriptive name (`feat/retry-budget-jitter`, `fix/transport-cancel-leak`). -3. **Run `just lint` and `just test`** locally before pushing. CI will reject changes that fail either. -4. **Add tests** for any code change. Property-based tests (via Hypothesis) are required for concurrency-sensitive code (retry budget, bulkhead, retry interleaving). -5. **Open a pull request** against `main`. PR titles use conventional-commits style (`feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`). - -## Code style - -- `ruff format` enforces formatting; do not hand-format. -- Type-check with `ty` (Astral). Use `# ty: ignore[]` for suppressions, not `# type: ignore`. -- Do NOT use `from __future__ import annotations`. Python 3.11+ is the floor. -- Module docstrings are required; per-method docstrings only when types alone are insufficient. - -See `docs/concepts/middleware.md` and `docs/recipes/custom-middleware.md` (once published) for architecture conventions. - -## Architecture invariants - -These are enforced by CI grep gates. Do not break them in pull requests: - -- No `import httpx2` outside `src/httpware/transports/httpx2.py`. -- No `httpx2._*` (private API) usage anywhere in the library. -- No `from __future__ import annotations`. -- No `print()` calls. -- No `logging.basicConfig()` or bare `logging.getLogger()`. - -## Code of Conduct - -By participating in this project, you agree to abide by its Code of Conduct. Be excellent to one another. - -## License - -By contributing, you agree that your contributions will be licensed under the project's MIT license. +Source: [`docs/dev/contributing.md`](docs/dev/contributing.md). diff --git a/README.md b/README.md index 344f6ac..2efeb66 100644 --- a/README.md +++ b/README.md @@ -7,9 +7,9 @@ **Async HTTP client framework for Python.** -`httpware` is a typed, async HTTP client library built on `httpx2` with a protocol-based seam so the transport is swappable. Middleware composes via an onion model. Pydantic and msgspec response decoding ship out of the box. `RecordedTransport` replaces respx for transport-level tests. +`httpware` is a typed, async HTTP client library with a protocol-based seam so the transport is swappable (`httpx2` ships as the default). Middleware composes via an onion model. Pydantic and msgspec response decoding ship out of the box. `RecordedTransport` replaces `respx` for transport-level tests. -> **Status:** Pre-1.0 (0.1.0 alpha). Public API is subject to change between minor releases until v1.0. Resilience middleware (retry / timeout / bulkhead), streaming, and observability are not yet shipped — track progress on GitHub. +> **Status:** Pre-1.0 (0.1.0 alpha). Public API is subject to change between minor releases until v1.0. Resilience middleware (retry / timeout / bulkhead), streaming, and observability are not yet shipped. ## Install @@ -23,7 +23,7 @@ Optional extras: pip install httpware[msgspec] # MsgspecDecoder ``` -(`otel`, `niquests`, and `all` extras are declared but their integrations have not shipped yet.) +(`otel`, `niquests`, and `all` extras are declared; integrations have not shipped yet.) ## Quickstart @@ -43,20 +43,12 @@ async def main() -> None: print(user.name) ``` -## What ships in 0.1.0 +## 📚 [Documentation](https://httpware.readthedocs.io) -- **`AsyncClient`** — eight HTTP method shortcuts (`get`, `post`, `put`, `patch`, `delete`, `head`, `options`, `request`) with typed `response_model` overloads; per-call overrides for `headers`, `params`, `cookies`, `timeout`, `json`, `content`; httpx-style `base_url` join; `with_options(...)` returns a view sharing the same transport. -- **Transport-agnostic seam.** `httpx2` is confined to `httpware.transports.httpx2.Httpx2Transport`. Implement the `Transport` protocol to swap backends. -- **Middleware foundation.** `Middleware` protocol, `Next` type alias, and phase decorators (`@before_request`, `@after_response`, `@on_error`). The chain is composed at `AsyncClient` construction; consumers don't compose chains themselves. -- **Pluggable response decoding.** `PydanticDecoder` (default) with cached `TypeAdapter`; `MsgspecDecoder` via `httpware[msgspec]`. -- **`RecordedTransport`** — built-in test double with a route table, observed-request list, and `aclose_calls` counter. -- **Status-keyed exception hierarchy** — `StatusError`, 4xx / 5xx subclasses, plain typed fields (`status: int`, `body: bytes`, `headers`, `json`, `request_method`, `request_url`). Pickleable; userinfo redacted in `__repr__`. -- **No `httpx2` exception types** leak through `httpware`. The transport seam maps them to `httpware` exceptions. +## 📦 [PyPI](https://pypi.org/project/httpware) + +## 📝 [License](./LICENSE) ## Part of `modern-python` Browse the full list of templates and libraries in [`modern-python`](https://github.com/modern-python) — see the org profile for the categorized index. - -## License - -MIT — see [LICENSE](./LICENSE). diff --git a/docs/archive/README.md b/docs/archive/README.md deleted file mode 100644 index 67f209a..0000000 --- a/docs/archive/README.md +++ /dev/null @@ -1,17 +0,0 @@ -# Archive - -This directory contains the bmad-era planning artifacts for `httpware`: - -- `prd.md` — 47 functional and 25 non-functional requirements. -- `architecture.md` — twelve architectural decisions, the five protocol seams, full module layout. -- `epics.md` — six epics with 32 stories. -- `product-brief-httpware.md` and `product-brief-httpware-distillate.md` — executive brief and detail pack from the predecessor `community-of-python/base-client` scoping exercise. -- `stories/` — per-story specs (1-1 through 1-5) and the retired `sprint-status.yaml`. - -These files are **historical reference, not authoritative**. The load-bearing decisions were distilled into [`../engineering.md`](../engineering.md) on 2026-05-31 when the project switched workflows from bmad to superpowers. Consult these archived files only when you need: - -- Original rationale behind a decision (e.g., "why did we choose `httpx2` over `aiohttp`?"). -- The specific FR/NFR numbers that a future spec wants to cite (e.g., `archive/prd.md#NFR-12`). -- The Given/When/Then acceptance criteria from a completed story. - -For everything else — invariants, seams, module layout, conventions, the remaining roadmap — read `../engineering.md` and `../../CLAUDE.md`. diff --git a/docs/archive/architecture.md b/docs/archive/architecture.md deleted file mode 100644 index 72bdddf..0000000 --- a/docs/archive/architecture.md +++ /dev/null @@ -1,1034 +0,0 @@ ---- -stepsCompleted: - - step-01-init - - step-02-context - - step-03-starter - - step-04-decisions - - step-05-patterns - - step-06-structure - - step-07-validation - - step-08-complete -status: complete -completedAt: 2026-05-11 -inputDocuments: - - docs/prd.md - - docs/product-brief-httpware.md - - docs/product-brief-httpware-distillate.md -workflowType: architecture -project_name: httpware -user_name: Artur Shiriev -date: 2026-05-11 -updated: 2026-05-12 -update_note: "Reflects the pydantic/httpx2 fork (2026-05-11); transport, dependencies, CI greps switched from encode/httpx 0.28 to pydantic/httpx2 2.0.0b1." ---- - -# Architecture Decision Document - -_This document builds collaboratively through step-by-step discovery. Sections are appended as we work through each architectural decision together._ - -## Project Context Analysis - -### Requirements Overview - -**Functional Requirements:** -47 FRs organized into 9 capability areas: Client Construction & Lifecycle (FR1–6), Request & Response (FR7–11), Transport Layer (FR12–16), Middleware System (FR17–22), Resilience (FR23–30), Validation & Typed Responses (FR31–35), Error Handling (FR36–40), Testing Support (FR41–43), Observability (FR44–47). Each FR is implementation-agnostic and testable. - -Architectural implication: each capability area maps to a discrete internal module, with clearly defined protocols at the boundaries (`Transport`, `ResponseDecoder`, `Middleware`). The public API surface is ~25 symbols — narrow on purpose to support v1.x stability commitments (NFR18). - -**Non-Functional Requirements driving architecture:** - -| NFR | Architectural pressure | -|---|---| -| NFR1 (≤15% framework overhead) | Shallow middleware chain; minimal per-request allocations; default-on middlewares must be efficient | -| NFR2 (cached `TypeAdapter`) | Module-level cache keyed by `response_model` — explicit memoization layer | -| NFR3 (`validate_json` single pass) | Decoder receives raw `content: bytes`, not already-parsed `dict` | -| NFR4 (no blocking calls in hot path) | Static check or test on the framework's call-graph; pure async/await throughout | -| NFR12 (RetryBudget concurrency-safe under 10k Hypothesis trials) | Either single `asyncio.Lock` or lock-free token-bucket with monotonic timestamps; design choice is load-bearing | -| NFR14 (event-loop-bound client) | Client construction binds to the active loop; transport lazy-creation must be careful about loop affinity | -| NFR15 (CancelledError never swallowed) | Every middleware must re-raise `CancelledError` without transformation; built-in middlewares CI-tested for this | -| NFR16 (streaming pool return on any exception) | Streaming context manager must use `try/finally` and guarantee transport-pool release | -| NFR17 (`ty` type check; py.typed) | Generic-aware method overloads for `response_model: type[T] | None`; protocol types must be `@runtime_checkable` or carefully designed | -| NFR18 (no v1.x breaking changes) | Internal/private modules clearly demarcated (`httpware._internal`); deprecation infra in place | -| NFR19 (OTel semconv conformance) | Observability middleware emits structured attributes per OTel HTTP-client spec, CI-validated | - -**Scale & Complexity:** - -- Project complexity: **medium** — low domain/regulatory load, moderate concurrency-correctness load, novel for Python ecosystem -- Primary domain: **Python async library / framework** (no service, no UI, no persistence) -- Estimated architectural components: **~10 internal modules** organized by capability area -- Core LOC estimate: 1500-2000 baseline, 4000-6000 realistic ceiling -- Public API surface: ~25 symbols -- Distribution: PyPI; pure-Python wheel; build backend `uv-build`; install extras for msgspec, otel, niquests - -### Technical Constraints & Dependencies - -**Hard constraints (from PRD):** - -- Python 3.11+ floor (`asyncio.TaskGroup`, `except*` syntax required) -- Async-only public API; no sync facade in v1.0 -- Backend HTTP client is `httpx2 >=2.0.0, <3.0` for v1.0 (Pydantic Services stewardship line; same API as `encode/httpx` 0.28); **no httpx2 private-API usage** (`httpx2._client`, `httpx2._types`) enforced by CI grep -- `pydantic >=2.0, <3.0` for default decoder -- Pure-Python; no compiled extensions, no platform-specific wheels -- All public types `py.typed`; `ty` (Astral) passes in CI -- Default `Limits`: `max_connections=100, max_keepalive=20, keepalive_expiry=5.0` -- Default `Timeout`: `connect=5, read=30, write=30, pool=5` (split, not single-value) -- OpenTelemetry semantic-convention conformance (HTTP-client spec) - -**Optional dependencies (install extras):** - -- `httpware[msgspec]` → `msgspec >=0.18` -- `httpware[otel]` → `opentelemetry-api`, `opentelemetry-sdk` -- `httpware[niquests]` → `niquests` (post-v1.0, Growth phase) - -**Backwards-compatibility commitment:** no breaking changes within v1.x; deprecation warnings emitted one minor version before removal. - -### Cross-Cutting Concerns Identified - -These concerns touch every internal module and must be designed for explicitly rather than added later: - -1. **Type-safety with generics.** Every request method must carry the `response_model: type[T] | None` `TypeVar` through the call chain so that `response_model=User` yields a return type of `User`, and `response_model=None` yields a `Response` wrapper. Overload signatures or `TypeIs` machinery required. -2. **Async event-loop binding.** A client instance must operate correctly within its creating loop and produce documented undefined behavior outside it. Lifecycle: construction is loop-agnostic, first I/O binds the transport to the active loop. -3. **Cancellation correctness.** `asyncio.CancelledError` is propagated unchanged through every middleware in the framework; failure-classification logic in `Retry`, `RetryBudget`, and the future circuit-breaker plug-in explicitly excludes it. Tests verify each middleware does not swallow or transform it. -4. **Immutability.** `Request` and `Response` are frozen dataclasses. Mutation methods (`req.with_header(...)`, `req.with_url(...)`) return new instances. Prevents middleware action-at-a-distance bugs and is required for safe retry rebuild. -5. **Secret redaction.** A single configurable redaction hook is invoked everywhere headers or bodies leave the framework: logs, OTel spans, exception `repr()`, debug output. Default redacted-header allowlist applies; users can extend. -6. **Observability hooks.** Lifecycle events (request start/complete, retry attempted, budget exhausted, timeout, error) are emitted from canonical points in the middleware chain. Hook signatures are stable across the v1.x line. -7. **Packaging discipline.** Optional extras must not be imported at the top-level of the base install; lazy imports gated by `ImportError` with helpful messages. -8. **Testability as a first-class concern.** `RecordedTransport` is part of `httpware` (not a separate `httpware-testing` package); consumers can ship tests against `RecordedTransport` without dev-dependency overhead. - -## Starter Template Evaluation - -### Primary Technology Domain - -Python async library (pip-installable, PyPI-distributed). No project-template tradition equivalent to web/mobile scaffolds; convention is to start from a minimal `pyproject.toml` and add infrastructure incrementally. - -### Starter Options Considered - -| Option | Pros | Cons | Verdict | -|---|---|---|---| -| `uv init --lib` | Minimal, uses `uv_build` (matches PRD), src layout, no template baggage | Lacks org conventions; manual port needed | **Selected** | -| Fork `modern-python/modern-di` | Inherits org conventions (Justfile, GHA workflow, ruff config, ty config, release flow) | Heavier; manual rename pass needed | Used as reference | -| `copier` template (org-owned) | Reusable across future `modern-python` libs | None exists today; not worth building now | Deferred | -| `cookiecutter-pypackage` | Mature | Pulls conventions that don't match `modern-python` house style | Rejected | - -### Selected Starter: `uv init --lib httpware` - -**Rationale:** - -- `uv_build` is already committed in the PRD as the PEP 517 build backend -- `uv init --lib` produces the minimum-viable scaffold (src layout, `__init__.py`, py.typed marker, `pyproject.toml`) with no opinionated extras -- Org conventions are copied from `modern-python/modern-di`, keeping the repo shape consistent with the org's house style - -**Initialization Command:** - -```bash -uv init --lib httpware -cd httpware -# Copy org conventions from modern-python/modern-di: -# Justfile, .github/workflows/, [tool.ruff] config, ty config, release flow -# Add: py.typed marker, SECURITY.md, CONTRIBUTING.md, LICENSE -``` - -**Architectural decisions provided by starter:** - -- **Language & runtime:** Python 3.11+, async-only; no compiled extensions -- **Build backend:** `uv_build` (matching `base-client` and `modern-di`; PEP 517 compliant via `[build-system] requires = ["uv_build"]`) -- **Project layout:** `src/httpware/` (src layout — keeps test code from accidentally importing local source). Note: `modern-di` itself uses a flat layout (`modern_di/` at repo root with `[tool.uv.build-backend] module-name = "modern_di"`); `uv init --lib` defaults to src layout, which is the safer choice. -- **Type marker:** `py.typed` ships in the package -- **Package manager / lock file:** `uv` (lockfile `uv.lock` committed, matching base-client and modern-di) - -**Decisions NOT provided by starter (to be made in subsequent steps):** - -- Internal module layout (`client.py`, `request.py`, `response.py`, `errors.py`, `middleware/`, `transports/`, `decoders/`, `_internal/`) -- Default Limits / Timeout values (committed by PRD) -- Middleware interface shape -- Transport protocol surface -- Response decoder protocol surface -- Exception hierarchy - -**Tooling to copy from `modern-python/modern-di` (org-convention reference):** - -- `Justfile` with `install`, `test`, `lint`, `format`, `release` recipes -- `.github/workflows/` configuration (Python 3.11–3.14 matrix; ruff, ty, pytest) -- `[tool.ruff]` config — `select = ["ALL"]`, `line-length = 120`, `target-version = "py311"` (raised from modern-di's `py310`), `fix = true`, `unsafe-fixes = true`, ignore set: `D1`, `S101`, `TCH`, `FBT`, `D203`, `D213`, `COM812`, `ISC001` -- Type checker: **`ty`** (Astral) — matches `modern-di` and the org's house preference. NOT mypy/pyright. -- `[tool.pytest.ini_options]` — `asyncio_mode = "auto"`, `asyncio_default_fixture_loop_scope = "function"`, `--cov=.` enabled -- Dev dep group: `pytest`, `pytest-cov`, `pytest-asyncio`, `pytest-repeat`, `pytest-benchmark` -- Lint dep group: `ruff`, `ty`, `eof-fixer`, `typing-extensions` -- Release flow (tag-triggered PyPI publish via Trusted Publishers, Sigstore attestation per NFR9) - -**Note:** Project initialization using this approach should be the first implementation story. - -## Core Architectural Decisions - -### Decision Priority Analysis - -**Critical decisions (block implementation):** -1. Decision 1 (Request/Response data types) — every other module depends on these primitives -2. Decision 2 (Transport protocol shape) + Decision 3 (exception mapping at the httpx2 seam) -3. Decision 8 (ResponseDecoder protocol) — enables typed responses end-to-end -4. Decision 4 (Middleware execution model) — enables the resilience layer - -**Important decisions (shape architecture significantly):** -- Decision 5 (RetryBudget data structure — load-bearing for NFR12) -- Decision 6 (Retry middleware: pure-Python, no tenacity dep) -- Decision 7 (Bulkhead via `asyncio.Semaphore` per host) -- Decision 9 (AsyncClient + `with_options` pool-sharing) -- Decision 10 (Streaming response model — separate `StreamResponse` type) -- Decision 11 (Observability two-layer architecture) -- Decision 12 (Redactor at every emission point) - -**Deferred decisions (post-MVP):** -- NiquestsTransport implementation details (Growth phase — same `Transport` protocol) -- Circuit-breaker middleware (Growth phase — plugs into the named extension slot) -- Sync API parallel class hierarchy (deferred; possibly never) -- OpenAPI codegen integration (Vision phase) - -### Data Architecture (library-internal types) - -**Decision 1 — Request/Response data types:** `dataclasses.dataclass(frozen=True, slots=True)`. - -- Immutable; mutation via `dataclasses.replace`-backed `with_*` methods returning new instances -- `slots=True` cuts per-instance memory and prevents attribute typos -- Avoids pulling `pydantic.BaseModel` into the hot path; `Request`/`Response` are pure stdlib -- `Response.content: bytes` is the primitive; `.json()` and `.text` are lazy properties -- `Limits` and `Timeout` config types are also frozen dataclasses - -Trade-off: rejected `attrs` (extra dep, not justified) and `pydantic.BaseModel` for primitives (NFR1 overhead budget). - -### Transport Protocol & Seam - -**Decision 2 — Transport protocol shape:** - -```python -@runtime_checkable -class Transport(Protocol): - async def __call__(self, request: Request) -> Response: ... - def stream(self, request: Request) -> AbstractAsyncContextManager[StreamResponse]: ... - async def aclose(self) -> None: ... -``` - -- Async `__call__` for unary requests; explicit `stream` returning an async context manager (FR11, NFR16) -- `aclose()` invoked by the client's `__aexit__` -- `@runtime_checkable` enables isinstance checks; cost is acceptable for a protocol with few methods - -**Decision 3 — Exception mapping at the seam.** The `httpx2 → httpware` mapping lives entirely inside `httpware/transports/httpx2.py`. No other module imports httpx2 exception types. - -| httpx2 exception | httpware exception | -|---|---| -| `ConnectError`, `NetworkError`, `ProxyError`, `UnsupportedProtocol`, `ProtocolError`, `RemoteProtocolError`, `LocalProtocolError`, `DecodingError`, `TooManyRedirects`, `InvalidURL` | `TransportError` | -| `ConnectTimeout`, `ReadTimeout`, `WriteTimeout`, `PoolTimeout` | `TimeoutError` | -| HTTP 4xx response | `BadRequestError` / `UnauthorizedError` / ... / `RateLimitedError` (per status) | -| HTTP 5xx response | `InternalServerError` / `ServiceUnavailableError` / `ServerStatusError` (default) | - -Status-to-exception mapping is a module-level dict keyed by `int`; unknown 4xx → `ClientStatusError`, unknown 5xx → `ServerStatusError`. - -### Middleware Execution Model - -**Decision 4 — Recursive async-callable onion with explicit `Next` type alias:** - -```python -Next = Callable[[Request], Awaitable[Response]] - -class Middleware(Protocol): - async def __call__(self, request: Request, next: Next) -> Response: ... -``` - -- Composed at client construction by folding the middleware list into a single coroutine; the bottom of the chain calls `transport.__call__` -- Short-circuit supported (middleware may not call `next`, returning a synthesized `Response`) — FR21 -- Phase-shortcut decorators (`@before_request`, `@after_response`, `@on_error`) wrap user functions into a `Middleware` adapter class -- Built-in middlewares (`Retry`, `RetryBudget`, `Bulkhead`, `Timeout`, `Observability`) ship as classes implementing this protocol - -Trade-off: rejected iterator-based (ASGI-style) middleware as unnecessary complexity for the linear no-branching case; rejected functional composition because it can't model after-response or short-circuit cases cleanly. - -### Resilience Implementation - -**Decision 5 — RetryBudget data structure:** Token bucket with `asyncio.Lock` + monotonic clock. - -- Lock held only during the read-modify-write of token count (microseconds; trivial vs network latency) -- Token refill driven by `time.monotonic()` deltas; no background task or timer -- State: `tokens_remaining: float`, `last_refill_at: float`, `ratio: float`, `min_per_sec: float`, `ttl: float` -- Property-based invariants verified via Hypothesis (NFR12, ≥10,000 trials): token count never negative, refill rate honors `min_per_sec` floor and `ratio` cap, concurrent acquires never double-spend -- Public state-inspection API: `budget.tokens_remaining`, `budget.in_use_ratio` (FR46) - -Trade-off: lock-free CAS rejected (Python lacks the primitives; would need third-party atomics). `asyncio.Semaphore` rejected (doesn't model refill rate). - -**Decision 6 — Retry middleware:** Pure-Python (no `tenacity` dependency for v1.0). - -- `RetryPolicy(max_attempts, base_delay, max_delay, retryable_statuses, retryable_exceptions, idempotent_methods, respect_retry_after)` -- Default: max 3 attempts; full-jitter exponential backoff (`delay = random.uniform(0, base * 2 ** (attempt - 1))`, capped at `max_delay=8s`) -- `Retry-After` (seconds or HTTP-date) takes precedence over computed backoff -- Only retries idempotent methods (GET/HEAD/PUT/DELETE) by default; POST/PATCH require explicit opt-in (FR24) - -Trade-off: `tenacity` rejected to avoid extra dependency and keep the retry logic transparent; ~100 LOC of loop code is the cost. - -**Decision 7 — Bulkhead implementation:** `asyncio.Semaphore` keyed per-host. - -- Semaphore registry stored in a `weakref.WeakValueDictionary` keyed by `key(request)` (default: `request.url.host`) -- Saturation behavior configurable: `queue` (default) or `fail_fast` (raises `BulkheadFullError(TransportError)`) -- Weakrefs allow transient hosts to garbage-collect - -### Validation & Decoding - -**Decision 8 — ResponseDecoder protocol:** - -```python -class ResponseDecoder(Protocol): - def decode(self, content: bytes, model: type[T]) -> T: ... -``` - -- Operates on raw `bytes` (NFR3 — single parse pass) -- Pydantic adapter (default): `TypeAdapter(model).validate_json(content)` with `@functools.lru_cache(maxsize=None)` on `TypeAdapter` construction keyed by `model` (NFR2) -- Msgspec adapter (`httpware[msgspec]`): `msgspec.json.decode(content, type=model)` -- Custom decoders supplied via constructor; missing-extra → `ImportError` with install hint - -### Configuration & Lifecycle - -**Decision 9 — AsyncClient internals and `with_options`:** - -- `AsyncClient` holds a single immutable `ClientConfig` frozen dataclass + a `Transport` instance -- `with_options(**overrides)` returns a new `AsyncClient` sharing the same `transport` (and connection pool) with `dataclasses.replace`-updated `config` -- Transport reference-counted via private `_ref_count` on the transport; outermost `__aexit__` calls `transport.aclose()` only when count returns to its initial value -- Auth normalization at construction: `_normalize_auth(value)` returns a `Middleware` regardless of input shape (str → static-bearer middleware; callable → token-provider middleware; Middleware → identity). FR5 union internalized. -- `AsyncClient.from_url(base_url, **kwargs)` classmethod factory builds a sensibly-configured client (FR2) - -### Streaming Response Model - -**Decision 10 — Separate `StreamResponse` type:** - -```python -@dataclass(frozen=True, slots=True) -class StreamResponse: - status: int - headers: Mapping[str, str] - url: str - _stream: AsyncIterator[bytes] # private - _release: Callable[[], Awaitable[None]] # private - - async def iter_bytes(self, chunk_size: int = 8192) -> AsyncIterator[bytes]: ... - async def iter_text(self, chunk_size: int = 8192) -> AsyncIterator[str]: ... - async def iter_lines(self) -> AsyncIterator[str]: ... -``` - -- `client.stream(...)` is `@asynccontextmanager` that always calls `_release` on exit, including on `CancelledError` (NFR15, NFR16) -- Separate type from `Response` — prevents accidental `.content` access on streaming responses (which would force a buffer read) - -### Observability Architecture - -**Decision 11 — Two-layer observability:** - -- **Layer 1 (free, always-on):** Lifecycle event callbacks registered on the client. Hook signatures: `on_request_start(req)`, `on_request_complete(req, resp)`, `on_retry_attempt(req, attempt, delay)`, `on_retry_budget_exhausted(req)`, `on_timeout(req, phase)`, `on_exception(req, exc)`. Called by built-in middleware at canonical points. Zero non-stdlib deps. -- **Layer 2 (opt-in via `httpware[otel]`):** `OpenTelemetryMiddleware` translates Layer-1 events into OTel spans and metrics conforming to HTTP-client semantic conventions (NFR19). Imported only when extras installed. - -No global logging configuration. Library uses `logging.getLogger("httpware")` only inside the optional observability middleware; emits nothing in unconfigured installs (NFR47). - -### Secret Redaction - -**Decision 12 — `Redactor` at every emission point:** - -```python -@dataclass(frozen=True) -class Redactor: - headers: frozenset[str] = frozenset({ - "authorization", "cookie", "set-cookie", - "x-api-key", "x-auth-token", "proxy-authorization", - }) - redact_bodies: bool = True - - def redact_headers(self, headers: Mapping[str, str]) -> Mapping[str, str]: ... - def redact_body(self, body: bytes | None) -> bytes | None: ... -``` - -- All `__repr__` methods on `Request`/`Response`/exception types pass through the client's redactor (NFR7, NFR8) -- OTel middleware redacts before emitting span attributes -- Default is on; users override via `AsyncClient(redactor=Redactor(headers=frozenset({...})))` - -### Decision Impact Analysis - -**Implementation sequence (load-bearing order):** - -1. `Request`, `Response`, `Limits`, `Timeout` data types (Decision 1) -2. `Transport` protocol (Decision 2) + exception hierarchy (Decision 3 — though only mapping table requires httpx2; exception classes themselves come first) -3. `Httpx2Transport` adapter implementing `Transport` -4. `ResponseDecoder` protocol + pydantic adapter (Decision 8) -5. `Middleware` protocol + `Next` type + composition logic (Decision 4) -6. `Retry`, `RetryBudget`, `Bulkhead`, `Timeout` middlewares (Decisions 5–7) -7. `AsyncClient` wiring (Decision 9) -8. `StreamResponse` + `client.stream()` (Decision 10) -9. Observability hooks layer (Decision 11, Layer 1) -10. `Redactor` integration across emission points (Decision 12) -11. OTel middleware in `transports/_otel.py` (Decision 11, Layer 2) — extras-gated -12. `RecordedTransport` test double - -**Cross-component dependencies:** - -- Middlewares depend on `Request`/`Response`/exception types but not on `Transport` -- `AsyncClient` depends on `Transport` + `Middleware` + `ResponseDecoder` + `Redactor` -- `OpenTelemetryMiddleware` depends on Layer-1 hooks AND `opentelemetry-api` (extras) -- `MsgspecDecoder` depends on `msgspec` (extras) -- Nothing in the core library imports `httpx2` outside `transports/httpx2.py` - -## Implementation Patterns & Consistency Rules - -### Pattern Categories Defined - -**Critical conflict points for a Python library:** - -1. Naming (modules, classes, functions, private symbols) -2. Structure (where tests live; where private code lives; how subpackages are organized) -3. Type-hint style (future annotations, generics, protocol vs ABC) -4. Async naming (a-prefix or no) -5. Exception construction format -6. Logging conventions (logger name, level discipline) -7. Optional-extra import pattern -8. Public API export discipline (`__all__` location) -9. Test file conventions (naming, layout, fixture scope) -10. Docstring style - -### Naming Patterns - -**Modules** — `snake_case`. Match `modern-di` style: `client.py`, `request.py`, `response.py`, `errors.py`, `transports/httpx2.py`, `decoders/pydantic.py`, `_internal/lock_pool.py`. No `httpware_client.py` (redundant prefix); no `Client.py` (PascalCase forbidden for module names). - -**Classes** — `PascalCase`. Examples: `AsyncClient`, `Request`, `Response`, `StreamResponse`, `RecordedTransport`, `Httpx2Transport`, `RetryBudget`, `PydanticDecoder`, `BadRequestError`. No `HTTPClient` (acronym capitalization avoided — `Http` is two letters by Python convention; matches httpx/httpx2 style: `httpx2.AsyncClient` not `HTTPClient`). - -**Functions and methods** — `snake_case`. Examples: `with_options`, `from_url`, `iter_bytes`, `iter_lines`, `aclose`, `normalize_auth`. Verbs preferred over nouns for actions. - -**Variables** — `snake_case`. Constants `UPPER_SNAKE_CASE`. Type variables `T`, `M`, single-letter PascalCase. - -**Private symbols** — `_leading_underscore` for module-private symbols (functions/classes not in `__all__`). `_internal/` subpackage for cross-module private code that needs to be importable across modules but is not part of the public API. Double-underscore name-mangling NOT used. - -**Test naming** — `test_.py` mirroring the module under test: `test_client.py`, `test_middleware_retry.py`, `test_transports_httpx2.py`. Test functions `test_` (`test_get_returns_typed_response`, `test_retry_honors_retry_after`). Fixture names `` (`fake_transport`, `client`, `recorded`). - -### Structure Patterns - -**Layout — src/-style:** - -``` -src/httpware/ - __init__.py # public re-exports + __all__ - client.py # AsyncClient - request.py # Request + with_* - response.py # Response, StreamResponse - errors.py # exception hierarchy - config.py # Limits, Timeout, ClientConfig, Redactor - middleware/ - __init__.py # Middleware, Next, before_request, after_response, on_error - retry.py # Retry - retry_budget.py # RetryBudget - bulkhead.py # Bulkhead - timeout.py # Timeout (middleware) - observability.py # lifecycle hooks (Layer 1) - _otel.py # OpenTelemetryMiddleware (Layer 2, extras-gated) - transports/ - __init__.py # Transport protocol - httpx2.py # Httpx2Transport + exception mapping - recorded.py # RecordedTransport - niquests.py # (Growth phase) - decoders/ - __init__.py # ResponseDecoder protocol - pydantic.py # PydanticDecoder + TypeAdapter cache - msgspec.py # MsgspecDecoder (extras-gated) - _internal/ - chain.py # middleware composition - clock.py # monotonic-time helpers used by RetryBudget - types.py # internal type aliases not part of public API - py.typed # zero-byte marker file - -tests/ - test_client.py - test_request.py - test_middleware_retry.py - test_middleware_retry_budget.py # property-based tests live here - test_transports_httpx2.py - test_transports_recorded.py - test_decoders_pydantic.py - test_errors.py - test_streaming.py - test_observability.py - conftest.py - -examples/ - quickstart.py - service_client.py - custom_middleware.py - streaming.py -``` - -**Rules:** - -- Tests live in top-level `tests/`, NOT co-located. `pyproject.toml`'s `[tool.pytest.ini_options] pythonpath = ["src"]` finds the package. -- `examples/` is shipped in the repo but excluded from the wheel. -- No `utils.py` catch-all; helpers go in `_internal/` named for what they help with. -- No `lib/`, `common/`, `core/` dumping grounds. - -### Type-Hint Style - -- **No `from __future__ import annotations`.** Python 3.11+ floor means PEP 604 union syntax (`A | B`) is native; no need for future-annotations stringification. Matches `modern-di`. -- **PEP 604 union syntax** preferred over `typing.Union`: `int | None` not `Optional[int]`. -- **`list[T]`, `dict[K, V]`, `tuple[T, ...]`** (PEP 585 generics) instead of `typing.List`, etc. -- **Type aliases** declared with `type X = ...` (PEP 695) where supported; fallback to `X: TypeAlias = ...` only if needed for `ty` compatibility. -- **Generics** via `class Foo[T]:` (PEP 695) syntax (Python 3.12+) where possible; for 3.11 compat use explicit `TypeVar` with `class Foo(Generic[T]):`. Confirm `ty`'s preference here at first implementation; if `ty` prefers PEP 695, raise floor to 3.12. -- **Protocols** declared with `Protocol`, `@runtime_checkable` only when isinstance checks are actually needed (`Transport`, `Middleware`, `ResponseDecoder`). -- **Suppression comments** are `# ty: ignore[]` (per user global pref), NOT `# type: ignore` or `# mypy: ignore`. -- **Imports** at module top level. NO TYPE_CHECKING guards unless avoiding a runtime circular import. - -### Async Naming - -- **No `a` prefix on async methods** unless the symbol already exists with a sync counterpart. Match httpx2's convention (same as httpx): `client.get(...)` not `client.aget(...)`. `aclose()` is the sole exception — used to disambiguate from any potential future sync `close()` and to match httpx2's pattern. -- **Context managers:** `__aenter__` / `__aexit__` always implemented as a pair. `@asynccontextmanager` for inline factories. -- **Async generators:** `async def iter_*` returning `AsyncIterator[T]`. - -### Exception Construction - -**All `httpware` exceptions are constructed with keyword arguments only**, no positional: - -```python -raise NotFoundError( - status=404, - body=resp.content, - headers=resp.headers, - json=resp.json() if resp.is_json else None, - request_method=req.method, - request_url=str(req.url), -) -``` - -**Mandatory fields on every status exception:** `status: int`, `body: bytes`, `headers: Mapping[str, str]`, `json: Any | None`, `request_method: str`, `request_url: str`. - -**`__repr__` format:** `""` — never includes body or headers in the default repr (NFR8). The `Redactor` is invoked on demand if a user inspects `e.headers` / `e.body`. - -**No bare `Exception` raises.** Every internal raise is one of the public exception types or a private `_internal` subclass. - -### Logging Conventions - -- **One library logger:** `logging.getLogger("httpware")`. Submodule loggers acquired as `logging.getLogger(f"httpware.{__name__.split('.')[-1]}")` — used inside transports and observability middleware only. -- **No log emission in the hot path of unconfigured installs** (NFR47). Logging is invoked only inside the optional observability middleware. -- **Level discipline:** DEBUG only. The library never emits INFO/WARNING/ERROR. The observability middleware may, but only if the user opts in by installing `httpware[otel]` and configuring a handler. -- **No `print()` anywhere.** Lint-enforced. -- **Structured logging via OTel attributes** — when the OTel middleware logs, it does so via span events with structured attributes per HTTP-client semconv (NFR19), not as free-form strings. - -### Optional-Extra Import Pattern - -Inside any module that uses an optional dependency: - -```python -# decoders/msgspec.py -try: - import msgspec -except ImportError as e: - raise ImportError( - "MsgspecDecoder requires the 'msgspec' extra. " - "Install with: pip install httpware[msgspec]" - ) from e -``` - -**Rules:** - -- Optional deps are imported at the **top of the optional module**, not lazily inside functions -- Top-level `httpware/__init__.py` never imports optional modules; users explicitly import `httpware.decoders.msgspec` or `httpware.middleware._otel` -- `try/except ImportError` raises a helpful message with the install command -- No `if importlib.util.find_spec(...)` runtime checks in the hot path - -### Public API Export Discipline - -- **Single source of truth:** `httpware/__init__.py` defines `__all__` listing every public symbol. -- **Re-export via explicit imports:** `from httpware.client import AsyncClient`, not `from httpware.client import *`. -- **Private modules use `_` prefix** and are NOT importable as `httpware._internal.foo` per documentation; downstream consumers who import private modules accept that they may break. -- **API surface tests:** a CI test asserts that `set(httpware.__all__) == EXPECTED_SET` to catch accidental additions or removals. Changes to the set require a changelog entry. - -### Docstring Style - -- **Module docstring:** one short line describing the module's purpose. Optional second paragraph for detail. -- **Class docstring:** one short line; followed by usage example only if non-obvious. -- **Public method docstring:** required. PEP 257 short-summary style. Args/Returns sections only when types alone are insufficient (rare — type hints carry most of the load). -- **Private function/method docstring:** optional; missing-docstring rule (`D1`) is ignored per ruff config. -- **No `# noqa`** comments without a rule code. No `# type: ignore` (use `# ty: ignore[rule]` per user global pref). - -### Test Conventions - -- **`pytest-asyncio` mode:** `asyncio_mode = "auto"` — async test functions don't need `@pytest.mark.asyncio`. -- **Fixture scope:** `function` by default (per `modern-di`'s `asyncio_default_fixture_loop_scope = "function"`). -- **Property-based tests** (Hypothesis) live in `test__props.py` (e.g., `test_middleware_retry_budget_props.py`) — separates fast unit tests from slow property tests. -- **Test fixtures** are defined in `conftest.py` if shared across files; otherwise inline. -- **No `unittest.TestCase`** subclasses; pytest function-style tests only. -- **Mocking:** `RecordedTransport` for network mocking. `unittest.mock.MagicMock` allowed for internal collaborators. No `respx` in `httpware`'s own tests (eat your own dogfood — though respx is acceptable in cross-compatibility tests). - -### Enforcement Guidelines - -**All AI agents MUST:** - -- Import via absolute paths only inside `httpware/`; relative imports only within the same subpackage (`from .pydantic import PydanticDecoder` inside `decoders/`) -- Place all new public symbols in `httpware/__init__.py`'s `__all__` AND add a changelog entry -- Add a property-based test for any new concurrency-touching code (retry-budget extensions, new resilience middleware) -- Run `ruff format`, `ruff check`, `ty`, `pytest` locally before pushing -- Reject `from __future__ import annotations`; reject `# type: ignore`; reject `print()` -- Map every new transport-specific exception to a public `httpware` exception in the transport adapter — never propagate - -**Pattern enforcement (CI):** - -- `ruff check` with the modern-di ignore set (D1, S101, TCH, FBT, D203, D213, COM812, ISC001) -- `ty` (Astral) on `src/httpware/` and on `examples/` -- `pytest --cov=httpware` with ≥90% threshold (NFR23) -- API-surface snapshot test (catches accidental public-symbol drift) -- `grep -r 'import httpx2\|from httpx2' src/httpware/` returns matches only inside `transports/httpx2.py` (Success Criteria → Technical Success) -- `grep -r 'httpx2\._' src/httpware/` returns zero matches - -### Pattern Examples - -**Good — exception construction with keyword args:** - -```python -status_to_exc = {404: NotFoundError, 429: RateLimitedError, ...} -exc_class = status_to_exc.get(resp.status_code, ClientStatusError if resp.status_code < 500 else ServerStatusError) -raise exc_class( - status=resp.status_code, - body=await resp.aread(), - headers=dict(resp.headers), - json=_try_json(resp), - request_method=req.method, - request_url=str(req.url), -) -``` - -**Anti-pattern — positional construction, leaking transport types:** - -```python -# WRONG: positional args, leaks httpx2.Response -raise NotFoundError(resp) -``` - -**Good — middleware with explicit `Next`:** - -```python -class TracingMiddleware: - async def __call__(self, req: Request, next: Next) -> Response: - with tracer.start_as_current_span(f"{req.method} {req.url.path}"): - return await next(req) -``` - -**Anti-pattern — middleware that swallows `CancelledError`:** - -```python -# WRONG: swallows CancelledError; breaks NFR15 -async def __call__(self, req: Request, next: Next) -> Response: - try: - return await next(req) - except Exception: # bare-Exception catches CancelledError on 3.11+ - return Response(status=599, ...) -``` - -The correct pattern uses `except StatusError` or a specific exception class, never bare `Exception`. - -## Project Structure & Boundaries - -### Complete Project Directory Structure - -See *Implementation Patterns → Structure Patterns* above for the full `src/httpware/` tree. Summarized: - -``` -src/httpware/ -├── __init__.py # public exports + __all__ -├── client.py # AsyncClient -├── request.py # Request + with_* -├── response.py # Response, StreamResponse -├── errors.py # status-keyed exception hierarchy -├── config.py # Limits, Timeout, ClientConfig, Redactor -├── middleware/ -│ ├── __init__.py # Middleware, Next, decorators -│ ├── retry.py -│ ├── retry_budget.py -│ ├── bulkhead.py -│ ├── timeout.py -│ ├── observability.py # Layer 1 (lifecycle hooks) -│ └── _otel.py # Layer 2 (extras-gated) -├── transports/ -│ ├── __init__.py # Transport protocol -│ ├── httpx2.py # Httpx2Transport + exception mapping -│ ├── recorded.py # RecordedTransport -│ └── niquests.py # Growth phase -├── decoders/ -│ ├── __init__.py # ResponseDecoder protocol -│ ├── pydantic.py # cached TypeAdapter -│ └── msgspec.py # extras-gated -├── _internal/ -│ ├── chain.py # middleware composition -│ ├── clock.py # monotonic-time helpers -│ └── types.py # internal type aliases -└── py.typed -tests/ # tests at repo root, not co-located -examples/ # excluded from wheel -docs/ # mkdocs source -``` - -**Repo-root configuration:** - -``` -httpware/ -├── README.md -├── LICENSE # MIT -├── SECURITY.md # CVE disclosure channel, 90-day window -├── CONTRIBUTING.md -├── CHANGELOG.md # Keep a Changelog format -├── pyproject.toml # [project], [build-system], [tool.ruff], [tool.pytest.ini_options] -├── uv.lock # committed -├── Justfile # install/test/lint/format/release recipes -├── mkdocs.yml # docs config -├── context7.json # context7 docs index (matches modern-di) -├── CLAUDE.md # AI-agent guidance for downstream consumers / contributors -├── .github/ -│ ├── workflows/ -│ │ ├── ci.yml # ruff, ty, pytest on push/PR -│ │ ├── publish.yml # tag-triggered PyPI publish (Trusted Publishers) -│ │ └── property-tests.yml # nightly Hypothesis run with high trial count -│ └── ISSUE_TEMPLATE/ -├── .gitignore -└── .readthedocs.yaml # if hosting on RTD; otherwise GitHub Pages -``` - -### Architectural Boundaries - -There are no service boundaries in `httpware` (it's a library, not a service). The relevant boundaries are between **internal protocol seams** — each is a point where one module talks to another only through a documented protocol, and the protocol is the only thing that can change without ripple effects. - -**Seam 1 — `Middleware ↔ Transport`** (innermost boundary): - -- Definition: the bottom of the middleware chain calls `transport.__call__(request) -> response`. Nothing else in the framework touches the transport. -- Crosses: `_internal/chain.py` composes the chain; `client.py` provides the transport. -- Stability: Transport protocol is part of the v1.x public-API contract (NFR18). Adding a new transport (e.g. NiquestsTransport) must not require chain changes. - -**Seam 2 — `AsyncClient ↔ Middleware`**: - -- Definition: client construction folds the middleware list into a single coroutine via `_internal/chain.compose(middlewares, transport) -> Next`. Calling `client.get(...)` invokes that coroutine with a fresh `Request`. -- Crosses: `client.py` builds the chain; `middleware/__init__.py` provides primitives. -- Stability: `Middleware` and `Next` types are part of the v1.x public contract. Adding a new built-in middleware does not change the protocol. - -**Seam 3 — `AsyncClient ↔ ResponseDecoder`**: - -- Definition: after the transport returns a `Response`, the client invokes `decoder.decode(response.content, response_model)` if `response_model` is provided. -- Crosses: `client.py` calls the decoder; `decoders/*` provides implementations. -- Stability: `ResponseDecoder` protocol is part of the v1.x public contract. - -**Seam 4 — `Httpx2Transport ↔ httpx2`** (external dependency boundary): - -- Definition: the only module that imports `httpx2` is `transports/httpx2.py`. It adapts `httpx2.AsyncClient` and `httpx2.Request`/`httpx2.Response`/`httpx2.Timeout`/`httpx2.Limits` to httpware's types, and maps every httpx2 exception to a httpware exception. -- Crosses: nothing else in `httpware/` imports httpx2. -- Enforcement: CI grep test (Technical Success in PRD). -- Stability: this is where httpx2-version-specific code lives. An httpx2 GA release (or eventual 3.0) touches one file. - -**Seam 5 — `httpware ↔ optional extras`** (external dependency boundary): - -- Definition: optional modules (`decoders/msgspec.py`, `middleware/_otel.py`, future `transports/niquests.py`) are the only places that import their respective extras. Base install does not import any of them. -- Enforcement: import-time test that imports `httpware` with no extras installed and verifies it doesn't fail. - -### Requirements to Structure Mapping - -Mapping the 47 FRs to their primary implementation module(s): - -| FR group | Capability area | Primary module(s) | -|---|---|---| -| FR1–FR6 | Client Construction & Lifecycle | `client.py`, `config.py` | -| FR7–FR11 | Request & Response | `client.py`, `request.py`, `response.py` | -| FR12–FR16 | Transport Layer | `transports/__init__.py` (protocol), `transports/httpx2.py` (default impl wrapping `pydantic/httpx2`) | -| FR17–FR22 | Middleware System | `middleware/__init__.py` (protocol + decorators), `_internal/chain.py` (composition) | -| FR23–FR25 | Retry | `middleware/retry.py` | -| FR26–FR27 | RetryBudget | `middleware/retry_budget.py` | -| FR28 | Bulkhead | `middleware/bulkhead.py` | -| FR29 | Timeout (per-attempt) | `middleware/timeout.py` | -| FR30 | Circuit-breaker extension slot | (no implementation in v1.0; documented in `middleware/__init__.py` chain ordering) | -| FR31–FR35 | Validation & Typed Responses | `decoders/__init__.py` (protocol), `decoders/pydantic.py` (default), `decoders/msgspec.py` (extras), `client.py` (response_model wiring) | -| FR36–FR40 | Error Handling | `errors.py` (hierarchy), `transports/httpx2.py` (mapping at seam) | -| FR41–FR43 | Testing Support | `transports/recorded.py` | -| FR44 | Lifecycle hooks | `middleware/observability.py` (Layer 1) | -| FR45 | OpenTelemetry instrumentation | `middleware/_otel.py` (Layer 2, extras-gated) | -| FR46 | RetryBudget state inspection | `middleware/retry_budget.py` (public `tokens_remaining`, `in_use_ratio`) | -| FR47 | No global logging | enforced via convention + CI grep for `logging.basicConfig` | - -**Cross-cutting concerns to modules:** - -| Concern | Where implemented | -|---|---| -| Type-safety with generics | `client.py` (method overloads for `response_model: type[T] \| None`) | -| Event-loop binding | `transports/httpx2.py` (lazy `httpx2.AsyncClient` creation on first `__call__`) | -| Cancellation correctness | every `middleware/*.py` (CI test verifies `CancelledError` propagation) | -| Immutability | `request.py`, `response.py` (frozen dataclasses + `with_*` methods) | -| Secret redaction | `config.py` (`Redactor` class) + every emission point (`__repr__`, OTel, logs) | -| Observability hooks | `middleware/observability.py` (Layer 1 emission) | -| Packaging discipline | top-level `__init__.py` only imports from non-extras modules | -| Testability | `transports/recorded.py` (`RecordedTransport`) | - -### Integration Points - -**Internal communication (within the library):** - -- Request flow: `AsyncClient.get(url, ...)` → builds `Request` → invokes composed middleware chain → bottom calls `Transport.__call__(request)` → returns `Response` → if `response_model` is set, calls `ResponseDecoder.decode(response.content, model)` → returns to user. -- Configuration flow: `AsyncClient(**kwargs)` → builds `ClientConfig` → builds default `Httpx2Transport` if not supplied → composes middlewares with chain ending at transport. -- Lifecycle: `async with AsyncClient(...) as client:` → `__aenter__` increments `transport._ref_count` → `__aexit__` decrements; if zero, calls `transport.aclose()`. - -**External integrations (what users plug in):** - -- Custom `Transport` (FR13) — any class satisfying the `Transport` protocol can replace `Httpx2Transport`. Plug-in point: `AsyncClient(transport=...)`. -- Custom `Middleware` (FR17) — any callable matching `(req, next) -> response`. Plug-in point: `AsyncClient(middleware=[...])`. -- Custom `ResponseDecoder` (FR34) — any class with a `decode(content, model)` method. Plug-in point: `AsyncClient(decoder=...)`. -- Observability backends — register Layer 1 hooks at client construction OR install `httpware[otel]` for Layer 2 integration with OTel exporters. -- Circuit-breaker plug-in (FR30) — third-party middleware plugs into the documented extension slot in the chain ordering. Reference implementation (post-MVP) wraps `purgatory`. - -**Data flow diagram:** - -``` -User code - │ - │ await client.get("/users/1", response_model=User) - ▼ -AsyncClient.request() - │ builds Request (frozen dataclass) - ▼ -[Observability outer wrap] - │ emits on_request_start - ▼ -[RetryBudget] - │ acquires token (or shortcuts to TransportError if exhausted) - ▼ -[Retry] - │ loop start; emits on_retry_attempt if attempt > 1 - ▼ -[Extension slot — circuit breaker plugs in here when shipped] - │ - ▼ -[Bulkhead] - │ acquires per-host semaphore - ▼ -[Timeout] - │ starts per-attempt deadline - ▼ -Transport.__call__(request) - │ Httpx2Transport → httpx2.AsyncClient → network - │ exception mapping at the seam - ▼ -Response (frozen dataclass) - │ propagates back up the chain - ▼ -[Retry decides: retry-able? loop again. Else return.] - ▼ -[Observability emits on_request_complete] - ▼ -AsyncClient.request() returns Response - │ if response_model is set: decoder.decode(response.content, model) - ▼ -User code receives typed T -``` - -### File Organization Patterns - -**Configuration files** — all live at the repo root (`pyproject.toml`, `Justfile`, `mkdocs.yml`, `.gitignore`, `LICENSE`, `SECURITY.md`, `CONTRIBUTING.md`, `CHANGELOG.md`, `CLAUDE.md`, `context7.json`, `.readthedocs.yaml`, `uv.lock`). No nested `config/` directory. - -**Source code** — under `src/httpware/`. Capability-aligned modules. Tests in `tests/`, not co-located. Examples in `examples/` (repo only, not in wheel — controlled via `[tool.uv.build-backend] module-name = "httpware"`). - -**Test organization** — flat under `tests/`, one file per `httpware/` module. Property-based tests in `_props.py` suffix files. `conftest.py` for shared fixtures. - -**Docs organization** — under `docs/`: - -``` -docs/ -├── index.md # README content for the docs site -├── quickstart.md -├── migration-from-base-client.md # release blocker -├── concepts/ -│ ├── middleware.md -│ ├── transports.md -│ ├── decoders.md -│ ├── retries-and-budget.md -│ └── exceptions.md -├── recipes/ -│ ├── custom-middleware.md -│ ├── authentication.md -│ ├── observability.md -│ └── testing.md -└── api/ # auto-generated via mkdocstrings -``` - -### Development Workflow Integration - -**Development server structure** — n/a (library, no server). Local development is `uv sync && pytest`. - -**Build process** — `uv build` produces wheel + sdist. `uv-build` PEP 517 backend; no setup.py, no build script. CI artifacts uploaded to GitHub Releases on tag. - -**Deployment structure** — release flow (matching `modern-di`): - -1. Update `CHANGELOG.md` for the release notes -2. Bump version in `pyproject.toml` -3. Tag with `vX.Y.Z` -4. GitHub Actions `publish.yml` triggers on tag, builds, uploads to PyPI via Trusted Publishers, attaches Sigstore attestation (NFR9) -5. Read the Docs build (if hosting there) triggers from main + tags - -## Architecture Validation Results - -### Coherence Validation ✅ - -**Decision Compatibility:** - -All 12 core architectural decisions reinforce rather than contradict each other. - -- Decision 1 (frozen dataclasses) supports Decision 4 (immutable middleware request) and Decision 6 (safe retry rebuild) without conflict -- Decision 2 (Transport protocol) is the foundation for Decisions 3 (exception mapping seam), 9 (lifecycle), and 10 (streaming) -- Decision 4 (onion middleware) is the substrate on which Decisions 5, 6, 7, 11, 12 all live -- Decision 8 (ResponseDecoder protocol) mirrors Decision 2 (Transport protocol) — same anti-leakage pattern applied to validation -- Decision 11 (two-layer observability) sits cleanly on Decision 4's middleware substrate; Layer 1 is just a built-in middleware - -No contradictions found across the 12 decisions or 47 FRs. - -**Pattern Consistency:** - -The patterns codify the implementation conventions for the decisions: - -- Async naming, module naming, and exception construction patterns are consistent with the data-type and transport decisions -- The CI grep enforcement (`httpx2` imports outside `transports/httpx2.py`) directly verifies Seam 4 from the structure section -- Test conventions (no `respx`, `RecordedTransport` only) match Decision 1 (own the abstractions) -- Logger naming (`logging.getLogger("httpware")`) matches the no-global-logging policy (NFR47, Decision 11) - -**Structure Alignment:** - -The 10-module layout supports the architecture: - -- One module per protocol seam (`transports/`, `middleware/`, `decoders/`) -- `_internal/` houses cross-module helpers (chain composition, clock, types) without exposing them -- Tests mirror modules 1:1; property-based tests are isolated by file naming convention -- Optional extras live in dedicated modules (`middleware/_otel.py`, `decoders/msgspec.py`, future `transports/niquests.py`) — never imported from the top-level - -### Requirements Coverage Validation ✅ - -**Functional Requirements coverage:** All 47 FRs map to specific modules (see *Requirements to Structure Mapping* in step 6). Capability-area-to-module is 1-to-1 or 1-to-few; no FR is orphaned and no FR is implemented across more than 2 modules. - -**Non-Functional Requirements coverage:** All 25 NFRs are addressable by the documented architecture. Spot-check: - -| NFR | Architectural support | -|---|---| -| NFR1 (≤15% overhead) | Frozen dataclasses, cached TypeAdapter, no `from __future__ import annotations`, shallow chain composition | -| NFR2 (cached TypeAdapter) | `decoders/pydantic.py` module-level `@functools.lru_cache` | -| NFR3 (validate_json single pass) | `ResponseDecoder.decode(content: bytes, model)` signature | -| NFR4 (no blocking calls) | Async-only throughout; CI grep for `requests`, `time.sleep` | -| NFR12 (RetryBudget concurrency) | Decision 5: `asyncio.Lock` + monotonic clock; property-based tests | -| NFR14 (event-loop binding) | Decision 9: lazy transport creation on first I/O | -| NFR15 (CancelledError) | Decision 4 pattern + per-middleware test | -| NFR16 (streaming pool return) | Decision 10: `StreamResponse._release` always called via `@asynccontextmanager` `finally` | -| NFR17 (`ty` type check) | Patterns: full annotations, PEP 604/585, `py.typed` marker | -| NFR18 (v1.x stability) | Patterns: `__all__` snapshot test, `_internal/` for private code | -| NFR19 (OTel semconv) | Decision 11: Layer 2 `_otel.py` middleware; CI conformance check | - -No NFR lacks architectural support. - -### Implementation Readiness Validation ✅ - -**Decision completeness:** All 12 decisions specify enough to begin implementation. Each includes the data structure, the public interface shape, and the trade-off considered. - -**Structure completeness:** Full directory tree specified to the file level. Module responsibilities explicit. Test file naming and location explicit. Configuration files at the repo root enumerated. - -**Pattern completeness:** Naming, structure, type hints, async, exceptions, logging, extras, exports, tests, docstrings — all 10 conflict categories addressed with concrete rules and pattern/anti-pattern examples. - -### Gap Analysis Results - -Honest list of gaps. None are blocking implementation; all are resolvable in early stories. - -**Critical gaps:** None. - -**Important gaps (resolve in early implementation):** - -1. **PEP 695 generic syntax vs older `Generic[T]`** — patterns prefer PEP 695 (`class Foo[T]:`, `type X = ...`) but PRD floor is Python 3.11; PEP 695 needs 3.12+. Decision: use older `Generic[T]` / `TypeVar` syntax on 3.11; revisit when raising the floor (3.10 EOL is Oct 2026; 3.11 EOL is Oct 2027). Document this in the implementation story for module skeletons. -2. **`@runtime_checkable` cost analysis** — applied to `Transport`, `Middleware`, `ResponseDecoder`. Need to verify no measurable per-request overhead from `isinstance` checks in the hot path. Mitigation: only do `isinstance` checks at client construction, not per-request. -3. **Auth string-coercion middleware shape** — `auth=str` is normalized to a static-bearer middleware, but the exact wire format (`Authorization: Bearer ` vs another scheme) needs spec. Defer to implementation: probably `Bearer` as default, with `AuthMiddleware(scheme=...)` for non-bearer schemes. -4. **Property-based test scenarios** — count target (≥10,000 trials) is set; specific Hypothesis strategies for RetryBudget invariants need authoring. Spec lives in the test-implementation story. -5. **OTel attribute emission list** — semantic-convention conformance is committed (NFR19, NFR45), but the exact attribute set per span/metric isn't enumerated here. Defer to the OTel middleware implementation story; reference will be opentelemetry-specification HTTP-client-semconv at the time of implementation. - -**Nice-to-have gaps (deferrable indefinitely):** - -6. **Module-content-hash test** — patterns mention a "public-symbol drift" test via `__all__` snapshot. The exact comparison mechanism (frozenset equality vs string snapshot) is an implementation detail. -7. **Pydantic v3 migration plan** — NFR20 commits to documenting a migration plan when v3 ships. Pydantic v3 timeline unknown; plan can be written reactively. -8. **`httpware[all]` meta-extra** — mentioned in PRD but not load-bearing; can be added in pyproject.toml at any time. -9. **Documentation content** — `docs/` structure specified; actual prose content is a v1.0 release-blocker but architecturally trivial. -10. **Sustainability / maintainer governance** — explicitly deferred by maintainer in PRD. Worth re-raising before v1.0 cut, not architecturally blocking. - -### Validation Issues Addressed - -None of the gaps above block proceeding to implementation. Items 1–5 are sequenced into early implementation stories; items 6–10 are tracked but not gating. - -### Architecture Completeness Checklist - -**Requirements Analysis** - -- [x] Project context thoroughly analyzed -- [x] Scale and complexity assessed -- [x] Technical constraints identified -- [x] Cross-cutting concerns mapped - -**Architectural Decisions** - -- [x] Critical decisions documented with versions (12 decisions, all with rationale and trade-offs) -- [x] Technology stack fully specified (Python 3.11+, httpx2 2.0.0+, pydantic v2, msgspec, OTel, uv_build, ruff, ty) -- [x] Integration patterns defined (5 seams) -- [x] Performance considerations addressed (NFR1–NFR5 mapped to decisions) - -**Implementation Patterns** - -- [x] Naming conventions established -- [x] Structure patterns defined -- [x] Communication patterns specified (middleware onion + hooks) -- [x] Process patterns documented (exception construction, logging, extras-import, test conventions) - -**Project Structure** - -- [x] Complete directory structure defined (file-level) -- [x] Component boundaries established (5 seams) -- [x] Integration points mapped (FR-to-module table + data-flow diagram) -- [x] Requirements to structure mapping complete (all 47 FRs mapped) - -### Architecture Readiness Assessment - -**Overall Status:** READY FOR IMPLEMENTATION - -All 16 checklist items confirmed. No Critical Gaps. 5 Important Gaps documented and queued for early-implementation resolution. - -**Confidence Level:** high - -The architecture is concrete enough that an AI agent (or a small team of humans) can begin implementation without further architectural decisions. The Important Gaps are well-bounded: each has a documented mitigation or a clear stage at which to resolve it. - -**Key Strengths:** - -- **Five protocol seams** make every load-bearing extension point first-class and documented. Future changes (NiquestsTransport, third-party circuit breaker, OpenAPI codegen) drop into known slots. -- **No httpx2 leakage** — enforced by CI grep, designed at every level (no public type references httpx2; mapping happens at one file). -- **Resilience composition order is named** — the "extension slot" is a documented contract, not a convention. Third-party middleware authors have a reliable target. -- **Test ergonomics built-in** — `RecordedTransport` is a first-class API, not a separate package. Consumer tests have a clear, low-overhead pattern. -- **Conventions match the org** — copying from `modern-di` keeps every `modern-python` library readable to a single set of muscle-memory conventions (ruff config, `ty`, `uv_build`, layout). - -**Areas for Future Enhancement:** - -- Reference circuit-breaker middleware (Growth phase; wraps `purgatory`) -- NiquestsTransport (Growth phase; second backend proves the abstraction) -- LLM-gateway preset (Vision phase; concrete answer for AI-service consumers) -- Sustainability / governance section in PRD (deferred per maintainer; revisit before v1.0 cut) -- Property-based test strategies for RetryBudget (early implementation story) -- Migration to PEP 695 generics when Python 3.11 floor is raised - -### Implementation Handoff - -**AI Agent Guidelines:** - -- Follow all 12 architectural decisions exactly as documented -- Apply the 10 implementation-pattern categories consistently across every module -- Respect the 5 protocol seams — never import across them except through the documented protocol -- Refer to this document for all architectural questions; if a question isn't answered here, surface it as a documentation gap rather than improvising -- Run `ruff check`, `ty`, `pytest` before pushing every change -- Add a property-based test for any code touching concurrency primitives -- Map every transport-specific exception to a public `httpware` exception at the transport seam - -**First Implementation Priority:** - -```bash -uv init --lib httpware -cd httpware -# Copy org conventions from modern-python/modern-di: -# Justfile, .github/workflows/, [tool.ruff], ty config, [tool.pytest.ini_options] -# Add: py.typed marker, SECURITY.md, CONTRIBUTING.md, LICENSE (MIT), CHANGELOG.md, CLAUDE.md -# Configure pyproject.toml with: -# - deps: httpx2>=2.0.0,<3.0, pydantic>=2.0,<3.0 -# - extras: msgspec, otel, niquests (placeholder) -# - [tool.uv.build-backend] module-name = "httpware" -``` - -The first implementation story is the project scaffold itself. Subsequent stories implement in the load-bearing order documented in Decision 13 (Decision Impact Analysis). diff --git a/docs/archive/epics.md b/docs/archive/epics.md deleted file mode 100644 index 2a3bf1a..0000000 --- a/docs/archive/epics.md +++ /dev/null @@ -1,823 +0,0 @@ ---- -stepsCompleted: - - step-01-validate-prerequisites - - step-02-design-epics - - step-03-create-stories - - step-04-final-validation -status: complete -inputDocuments: - - docs/prd.md - - docs/architecture.md - - docs/product-brief-httpware.md - - docs/product-brief-httpware-distillate.md -project_name: httpware -updated: 2026-05-12 -update_note: "Reflects the pydantic/httpx2 fork (2026-05-11); transport adapter, dependencies, and CI grep stories switched from encode/httpx 0.28 to pydantic/httpx2 2.0.0b1." ---- - -# httpware - Epic Breakdown - -## Overview - -This document provides the complete epic and story breakdown for `httpware`, decomposing the requirements from the PRD and Architecture into implementable stories. The library has no UI; no UX Design document applies. - -## Requirements Inventory - -### Functional Requirements - -**Client Construction & Lifecycle** -- FR1: Consumer can construct `AsyncClient` with optional `base_url`, default headers, default query params, timeout, limits, auth, transport, decoder, and middleware list. -- FR2: Consumer can construct `AsyncClient` via `AsyncClient.from_url(base_url, ...)` for one-line default configuration. -- FR3: Consumer can use the client as an async context manager (`async with`), closing the transport on exit. -- FR4: Consumer can derive a new client with overridden defaults via `client.with_options(**overrides)` sharing transport/pool. -- FR5: Consumer can pass auth as static string, sync callable, async callable, or custom `Middleware`. -- FR6: Consumer can configure connection limits (max_connections, max_keepalive, expiry) and timeouts (split or single). - -**Request & Response** -- FR7: Consumer can issue GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS via dedicated methods plus `client.request(method, url, ...)`. -- FR8: Consumer can override per-request headers, query, cookies, timeout; provide body via `json=`, `data=`, `files=`, or `content=`. -- FR9: Consumer receives `httpware.Response` exposing `status`, `headers`, `content`, `text`, `json()`, `url`, `elapsed` — no transport types. -- FR10: Consumer can request a typed response by passing `response_model=T`, receiving a value of type `T`. -- FR11: Consumer can stream via `async with client.stream(...) as resp` with `iter_bytes`, `iter_text`, `iter_lines`. - -**Transport Layer** -- FR12: Framework defines a `Transport` Protocol any HTTP-client backend must satisfy. -- FR13: Consumer can supply a custom `Transport` at client construction. -- FR14: Framework ships default `Httpx2Transport` adapting `httpx2.AsyncClient`. -- FR15: Swapping `Transport` requires no consumer code changes beyond construction. -- FR16: Framework's public exports do not include underlying HTTP client's types; `httpx2.*` is not re-exported. - -**Middleware System** -- FR17: Consumer can supply ordered list of `Middleware` instances at client construction. -- FR18: Consumer can implement `Middleware` via async callable matching `(req, next) -> Response`. -- FR19: Consumer can author middleware via `@before_request`, `@after_response`, `@on_error` decorators. -- FR20: Framework documents stable middleware execution order (`Observability → RetryBudget → Retry → [extension slot] → Bulkhead → Timeout → Transport`) with named extension slot. -- FR21: Consumer can short-circuit the middleware chain by not calling `next` and returning a synthesized `Response`. -- FR22: `Request` objects are immutable; mutation via `req.with_header(...)` etc. returns new instance. - -**Resilience** -- FR23: Framework retries failed requests per configurable policy (attempts, backoff, retryable statuses, retryable exceptions). -- FR24: Framework retries only idempotent methods (GET/HEAD/PUT/DELETE) by default; POST/PATCH opt-in. -- FR25: Full-jitter exponential backoff between retries; honor `Retry-After` header. -- FR26: Framework enforces retry budget (token-bucket admission control) capping retries/sec; rejected retries surface original error. -- FR27: Consumer can configure or disable retry budget at construction. -- FR28: Framework enforces per-host bulkhead (concurrency cap) with queue or fail-fast. -- FR29: Framework enforces per-attempt timeout; timed-out attempts raise `TimeoutError` and are retry-eligible. -- FR30: Consumer can plug a circuit-breaker (or other resilience primitive) into the documented extension slot without library changes; no built-in CB in v1.0. - -**Validation & Typed Responses** -- FR31: Framework defines a `ResponseDecoder` Protocol adapting raw response bytes to a typed model. -- FR32: Framework ships default pydantic-based `ResponseDecoder` with cached `TypeAdapter` and single-pass JSON validation. -- FR33: Framework ships msgspec-based `ResponseDecoder` via `httpware[msgspec]` extra. -- FR34: Consumer can supply custom `ResponseDecoder` at construction. -- FR35: Consumer can decode into pydantic models, dataclasses, TypedDict, `list[T]`, `dict[K,V]`, primitives. - -**Error Handling** -- FR36: Framework raises `httpware`-owned exceptions only; no transport-specific exceptions surface to consumers. -- FR37: Framework provides status-keyed exception hierarchy: BadRequest, Unauthorized, Forbidden, NotFound, Conflict, UnprocessableEntity, RateLimited, InternalServerError, ServiceUnavailable; base classes ClientStatusError (4xx), ServerStatusError (5xx), StatusError. -- FR38: Framework provides `TransportError` for connection/network failures and `TimeoutError` for client-side timeouts. -- FR39: Every exception exposes plain-typed fields: `status: int`, `body: bytes`, `headers: Mapping`, `json: Any | None`, `request_method: str`, `request_url: str`. -- FR40: Framework excludes `asyncio.CancelledError` from automatic retry and resilience-middleware failure accounting. - -**Testing Support** -- FR41: Framework ships `RecordedTransport` accepting `(method, url_pattern) → Response | Exception` mapping; exposes `.calls`. -- FR42: Consumer can construct client with `transport=RecordedTransport({...})` to drive tests without network. -- FR43: `RecordedTransport` supports both response and exception side-effects; calls inspectable for method, URL, headers, body. - -**Observability** -- FR44: Framework emits lifecycle hooks: request start, request complete, retry attempted, retry budget exhausted, per-attempt timeout, exception raised. -- FR45: Framework ships OpenTelemetry instrumentation middleware via `httpware[otel]` extra, conforming to OTel HTTP-client semconv. -- FR46: Consumer can inspect retry-budget runtime state (tokens remaining, in-use ratio) for `/healthz` integration. -- FR47: Framework does not configure global logging or emit logs in hot path unless observability middleware is explicitly installed. - -### NonFunctional Requirements - -**Performance** -- NFR1: Per-request framework overhead ≤15% over raw `httpx2.AsyncClient` + manual pydantic at 100 RPS on 5KB JSON payloads. Benchmark published with each release. -- NFR2: `TypeAdapter` instances cached per `response_model`; zero per-request construction after warm-up. -- NFR3: Default `ResponseDecoder` uses `validate_json(content)`, single parse pass. -- NFR4: No synchronous I/O, blocking calls, or GIL-heavy work on framework hot path. -- NFR5: Cold-start (first import + first request) ≤200ms on Python 3.11 developer-class machine. - -**Security** -- NFR6: TLS verification enabled by default; opt-out via explicit `verify=False`. -- NFR7: Configurable secret-redaction hook invoked on every header/body emission. Default redacted-header allowlist: Authorization, Cookie, Set-Cookie, X-Api-Key, X-Auth-Token, Proxy-Authorization. -- NFR8: No request/response body emitted to logs or spans by default. -- NFR9: Releases via PyPI Trusted Publishers + Sigstore attestation; SBOM attached to each GitHub Release. -- NFR10: SECURITY.md documents disclosure channel with 90-day private-disclosure window. - -**Concurrency & Throughput** -- NFR11: Single `AsyncClient` supports concurrent requests up to `max_connections` without framework-introduced lock contention beyond transport's requirements. -- NFR12: `RetryBudget` token accounting concurrency-safe under ≥10,000 Hypothesis trials with no race conditions or invariant violations. -- NFR13: Middleware execution is per-request and stateless by default; shared state is consumer responsibility. -- NFR14: `AsyncClient` bound to creating event loop; cross-loop sharing is documented undefined behavior. - -**Reliability & Correctness** -- NFR15: `asyncio.CancelledError` is never swallowed, transformed, or counted as failure by any built-in middleware. -- NFR16: Streaming-response context managers guarantee underlying connection returns to pool on any exception including `CancelledError`. -- NFR17: All public types pass `ty` (Astral) type checking on Python 3.11+; `py.typed` marker ships. -- NFR18: No breaking changes within v1.x; deprecations carry one-minor-version `DeprecationWarning` before removal. - -**Integration** -- NFR19: OpenTelemetry instrumentation conforms to OTel HTTP-client semantic conventions; CI-validated. -- NFR20: Compatible with pydantic v2 (`>=2.0, <3.0`) and msgspec (`>=0.18`). -- NFR21: Imports cleanly alongside FastAPI, Starlette, Litestar; smoke-tested in CI. -- NFR22: PEP 621 `pyproject.toml`; install/build succeed under `pip`, `uv`, `poetry`, `pdm` using `uv_build` PEP 517 backend. - -**Maintainability & Quality** -- NFR23: ≥90% line coverage on `httpware/` core modules (transports/decoders excluded), enforced in CI. -- NFR24: Property-based tests (Hypothesis) cover concurrency-sensitive primitives (RetryBudget, Bulkhead, retry interleaving, request immutability) with ≥10,000 trials per CI run. -- NFR25: CI runs on every push/PR: ruff lint, ty type check on `httpware/` and reference consumer, pytest with coverage, property suite, real-endpoint smoke test. - -### Additional Requirements - -From Architecture document — technical requirements that shape epic and story design: - -**Project scaffold (Epic 1 Story 1 — starter template):** -- Initialize with `uv init --lib httpware`; build backend is `uv_build` (PEP 517) -- Copy org conventions from `modern-python/modern-di`: `Justfile`, `.github/workflows/`, `[tool.ruff]` config, `ty` lint dep, `[tool.pytest.ini_options]`, release flow -- Add: `py.typed` marker, `SECURITY.md`, `CONTRIBUTING.md`, `LICENSE` (MIT), `CHANGELOG.md`, `CLAUDE.md`, `context7.json` -- `pyproject.toml` declares: `httpx2>=2.0.0,<3.0`, `pydantic>=2.0,<3.0`; extras `[msgspec]`, `[otel]`, `[niquests]` -- Layout: `src/httpware/` with capability-aligned modules (`client.py`, `request.py`, `response.py`, `errors.py`, `config.py`, `middleware/`, `transports/`, `decoders/`, `_internal/`) -- Lint group: `ruff`, `ty`, `eof-fixer`, `typing-extensions` -- Dev group: `pytest`, `pytest-cov`, `pytest-asyncio`, `pytest-repeat`, `pytest-benchmark`, `hypothesis` - -**Protocol seams (load-bearing — must implement to support all FRs):** -- Seam 1: `Middleware ↔ Transport` (chain bottom calls `transport.__call__`) -- Seam 2: `AsyncClient ↔ Middleware` (chain composed at construction) -- Seam 3: `AsyncClient ↔ ResponseDecoder` (called when `response_model` provided) -- Seam 4: `Httpx2Transport ↔ httpx2` (only module importing httpx2; exception mapping table) -- Seam 5: `httpware ↔ optional extras` (extras imported only inside their modules) - -**Architectural decisions to apply across all stories (12 numbered decisions):** -1. Request/Response as `dataclasses.dataclass(frozen=True, slots=True)` -2. `Transport` protocol with async `__call__`, `stream` context manager, `aclose` -3. httpx2→httpware exception mapping table in `transports/httpx2.py` only -4. Recursive async-callable onion middleware with explicit `Next` type alias -5. RetryBudget = `asyncio.Lock` + monotonic-clock token bucket -6. Pure-Python retry (no `tenacity` dependency) -7. `asyncio.Semaphore`-based bulkhead with weakref-keyed per-host registry -8. `ResponseDecoder` protocol with cached `TypeAdapter` pydantic adapter -9. `AsyncClient` holds immutable `ClientConfig`; `with_options()` returns new client sharing transport -10. Separate `StreamResponse` type with `_release` callable; `@asynccontextmanager` ensures release -11. Two-layer observability: Layer 1 hooks always-on, Layer 2 OTel middleware extras-gated -12. `Redactor` class registered on client, invoked at every emission point - -**Implementation patterns (10 categories) — must be enforced for all code in all stories:** -- Module/class/function/variable naming (Python conventions + project-specific) -- Project structure (src/ layout, tests/ at root, examples/ excluded from wheel) -- Type-hint style (no `from __future__ import annotations`, PEP 604/585, `# ty: ignore` not `# type: ignore`) -- Async naming (no `a` prefix except `aclose`) -- Exception construction (keyword-only, plain fields, no transport types) -- Logging (single `logging.getLogger("httpware")`, DEBUG only, no global config) -- Optional-extra imports (top-of-module try/except ImportError with install hint) -- Public API exports (`__all__` in `__init__.py`, API-snapshot CI test) -- Test conventions (`pytest-asyncio` auto mode, `RecordedTransport` only, property-based tests in `_props.py`) -- Docstring style (PEP 257; class/method required, missing-docstring `D1` ignored) - -**Migration deliverable (release blocker for Epic that includes v1.0 cut):** -- Migration guide from `base-client` to `httpware` with per-symbol replacement table, before/after code blocks, side-by-side example, and known gotchas - -### UX Design Requirements - -Not applicable. `httpware` is a developer library with no user interface. No UX Design document exists for this project. - -### FR Coverage Map - -| FR | Epic | Note | -|---|---|---| -| FR1 | 1 | `AsyncClient(**kwargs)` | -| FR2 | 1 | `AsyncClient.from_url(...)` | -| FR3 | 1 | `async with AsyncClient(...) as c:` | -| FR4 | 1 | `client.with_options(**overrides)` | -| FR5 | 2 | auth coercion `str \| Callable \| Middleware` | -| FR6 | 1 | `Limits`, `Timeout` config | -| FR7 | 1 | HTTP methods + `request()` | -| FR8 | 1 | per-request overrides + body forms | -| FR9 | 1 | `Response` with plain fields | -| FR10 | 1 | `response_model=T` | -| FR11 | 4 | streaming | -| FR12 | 1 | `Transport` protocol | -| FR13 | 1 | custom `Transport` at construction | -| FR14 | 1 | `Httpx2Transport` default | -| FR15 | 1 | transport swap with zero consumer changes | -| FR16 | 1 (enforced by Story 6.4 CI gate) | no httpx2 in public exports | -| FR17 | 2 | middleware list at construction | -| FR18 | 2 | `Middleware` protocol with `(req, next)` | -| FR19 | 2 | phase-shortcut decorators | -| FR20 | 2 (extension slot doc in 3.6, observability ordering in 5.1) | chain ordering | -| FR21 | 2 | short-circuit middleware | -| FR22 | 2 | `Request` immutability helpers | -| FR23 | 3 | Retry policy | -| FR24 | 3 | idempotent-method default | -| FR25 | 3 | full-jitter + Retry-After | -| FR26 | 3 | RetryBudget token bucket | -| FR27 | 3 | configure/disable budget | -| FR28 | 3 | Bulkhead semaphore | -| FR29 | 3 | per-attempt Timeout | -| FR30 | 3 | extension-slot documentation | -| FR31 | 1 | `ResponseDecoder` protocol | -| FR32 | 1 | pydantic decoder | -| FR33 | 1 | msgspec decoder (extras-gated) | -| FR34 | 1 | custom decoder via constructor | -| FR35 | 1 | decode targets (pydantic, dataclasses, etc.) | -| FR36 | 1 | exception ownership | -| FR37 | 1 | status-keyed exception hierarchy | -| FR38 | 1 | `TransportError`, `TimeoutError` | -| FR39 | 1 | plain exception fields | -| FR40 | 1 (`CancelledError` discipline reinforced in every later epic) | `CancelledError` excluded | -| FR41 | 1 | `RecordedTransport` | -| FR42 | 1 | construct client with `RecordedTransport` | -| FR43 | 1 | call inspection / side-effects | -| FR44 | 5 | lifecycle hook callbacks | -| FR45 | 5 | OTel middleware | -| FR46 | 3 | RetryBudget state inspection | -| FR47 | 5 | no global logging | - -## Epic List - -### Epic 1: Make typed HTTP requests with sensible defaults -A developer installs `httpware`, writes `await client.get(url, response_model=User)`, gets a typed result, handles errors with status-keyed exceptions, and tests it via `RecordedTransport`. The library is useful as-is — independently shippable as a v0.1.0. -**FRs covered:** FR1, FR2, FR3, FR4, FR6, FR7, FR8, FR9, FR10, FR12, FR13, FR14, FR15, FR16, FR31, FR32, FR33, FR34, FR35, FR36, FR37, FR38, FR39, FR40, FR41, FR42, FR43 - -### Epic 2: Compose request-handling logic via middleware -A developer writes custom middleware (signing, correlation IDs, tracing) and composes it into their client. The framework's extensibility is real and ergonomic. -**FRs covered:** FR5, FR17, FR18, FR19, FR20, FR21, FR22 - -### Epic 3: Survive upstream failures with composable resilience -A developer's client survives 429s, retries idempotent methods, doesn't retry-storm via the budget, caps per-host concurrency, and times out per-attempt. Production-ready resilience without bolting on `tenacity` or a buggy circuit-breaker. -**FRs covered:** FR23, FR24, FR25, FR26, FR27, FR28, FR29, FR30, FR46 - -### Epic 4: Stream responses without buffering -A developer streams large downloads or SSE/LLM responses without loading them into memory; pool returns are guaranteed on any exception including `CancelledError`. -**FRs covered:** FR11 - -### Epic 5: Observe and instrument the client -A developer instruments their client with lifecycle hooks, integrates with OpenTelemetry exporters, and ships with sensible secret redaction. Operations team can observe retry storms, budget exhaustion, and breaker rejections from a real Grafana board. -**FRs covered:** FR44, FR45, FR47 - -### Epic 6: Ship v1.0 -A `base-client` consumer reads the migration guide, swaps imports, and ships on `httpware` within a few hours. The library is publicly trustable — signed releases, SBOM, security disclosure channel, public benchmarks. -**FRs covered:** (none directly; ships the deliverables required by Success Criteria, NFR9, NFR10, NFR18, NFR23–25, and the migration-guide release blocker) - ---- - -## Epic 1: Make typed HTTP requests with sensible defaults - -A developer installs `httpware`, writes `await client.get(url, response_model=User)`, gets a typed result, handles errors with status-keyed exceptions, and tests it via `RecordedTransport`. Independently shippable as v0.1.0. - -### Story 1.1: Project scaffold and tooling - -As a `httpware` maintainer, -I want a fully-configured project skeleton with the org's conventions, -So that subsequent stories can implement library code without fighting tooling. - -**Acceptance Criteria:** - -**Given** a fresh checkout of a new GitHub repo at `modern-python/httpware` -**When** I run `uv init --lib httpware` followed by the org-convention port from `modern-python/modern-di` -**Then** the repo has `src/httpware/__init__.py`, `src/httpware/py.typed`, `pyproject.toml` declaring `httpx2>=2.0.0,<3.0` and `pydantic>=2.0,<3.0` as dependencies -**And** extras `[msgspec]`, `[otel]`, `[niquests]`, `[all]` are declared -**And** dev/lint dep groups match `modern-di` (pytest, pytest-cov, pytest-asyncio, pytest-repeat, pytest-benchmark, hypothesis; ruff, ty, eof-fixer, typing-extensions) -**And** `[tool.ruff]`, `[tool.pytest.ini_options]` match `modern-di` with `target-version = "py311"` -**And** root files exist: `Justfile`, `LICENSE` (MIT), `SECURITY.md`, `CONTRIBUTING.md`, `CHANGELOG.md`, `CLAUDE.md`, `context7.json`, `.gitignore` -**And** `.github/workflows/ci.yml` runs `ruff check`, `ty`, `pytest --cov` on Python 3.11–3.14 -**And** `uv build` produces a wheel and `pip install dist/*.whl` succeeds in a clean venv - -### Story 1.2: Core data types - -As a library author, -I want immutable `Request`, `Response`, `Limits`, `Timeout`, and `ClientConfig` types, -So that every other module has stable primitives to build on. - -**Acceptance Criteria:** - -**Given** the scaffold from Story 1.1 -**When** I implement `src/httpware/request.py`, `src/httpware/response.py`, `src/httpware/config.py` -**Then** `Request` is a `@dataclass(frozen=True, slots=True)` with fields `method: str`, `url: str`, `headers: Mapping[str, str]`, `params: Mapping[str, str]`, `cookies: Mapping[str, str]`, `body: bytes | None`, `extensions: Mapping[str, Any]` -**And** `Request` has methods `with_header(name, value) -> Request`, `with_url(url) -> Request`, `with_body(body) -> Request`, `with_query(params) -> Request`, each returning a new instance via `dataclasses.replace` -**And** `Response` is a `@dataclass(frozen=True, slots=True)` with fields `status: int`, `headers: Mapping[str, str]`, `content: bytes`, `url: str`, `elapsed: float` -**And** `Response.text` and `Response.json()` are computed accessors (not stored) -**And** `Limits`, `Timeout`, `ClientConfig` are frozen dataclasses with the defaults specified in the architecture (`Timeout(connect=5, read=30, write=30, pool=5)`, `Limits(max_connections=100, max_keepalive_connections=20, keepalive_expiry=5.0)`) -**And** `ty` passes; tests cover `with_*` immutability and that two `Request` instances with identical fields compare equal - -### Story 1.3: Exception hierarchy with plain fields - -As a consumer developer, -I want a status-keyed exception hierarchy with plain typed fields, -So that I can catch `NotFoundError` etc. without importing httpx2 and without inspecting transport types. - -**Acceptance Criteria:** - -**Given** `Request` and `Response` types from Story 1.2 -**When** I implement `src/httpware/errors.py` -**Then** the module defines `ClientError`, `TransportError`, `TimeoutError`, `StatusError(ClientError)`, `ClientStatusError(StatusError)`, `ServerStatusError(StatusError)`, and the leaf classes `BadRequestError`, `UnauthorizedError`, `ForbiddenError`, `NotFoundError`, `ConflictError`, `UnprocessableEntityError`, `RateLimitedError`, `InternalServerError`, `ServiceUnavailableError` -**And** every status exception's `__init__` takes only keyword arguments: `status: int`, `body: bytes`, `headers: Mapping[str, str]`, `json: Any | None`, `request_method: str`, `request_url: str` -**And** a module-level dict `STATUS_TO_EXCEPTION: Mapping[int, type[StatusError]]` maps the canonical status codes to their leaf exceptions -**And** unknown 4xx falls back to `ClientStatusError`; unknown 5xx falls back to `ServerStatusError` -**And** `__repr__` format is `""` and never includes body or headers -**And** `__all__` lists every exception; `ty` passes - -### Story 1.4: Transport protocol and Httpx2Transport adapter - -As a library author, -I want a `Transport` protocol and a default `Httpx2Transport` implementation, -So that the entire library talks to one abstraction and httpx2 is confined to a single file. - -**Acceptance Criteria:** - -**Given** the data types and exception hierarchy from Stories 1.2 and 1.3 -**When** I implement `src/httpware/transports/__init__.py` and `src/httpware/transports/httpx2.py` -**Then** `Transport` is a `@runtime_checkable Protocol` with three methods: `async def __call__(self, request: Request) -> Response`, `def stream(self, request: Request) -> AbstractAsyncContextManager[StreamResponse]` (signature only — implementation deferred to Epic 4 with `NotImplementedError`), `async def aclose(self) -> None` -**And** `Httpx2Transport` accepts a `httpx2.AsyncClient` (or constructs a default one from `Limits` / `Timeout`) and implements `__call__` by translating `Request` → `httpx2.Request`, awaiting `client.send`, and translating back to `Response` -**And** `Httpx2Transport.__call__` maps every `httpx2.HTTPError` subclass to a `httpware` exception per the architecture's mapping table, and never lets an `httpx2` exception escape -**And** `Httpx2Transport.__call__` raises one of `BadRequestError`/.../`ServiceUnavailableError`/`ClientStatusError`/`ServerStatusError` for any non-2xx response -**And** `grep -r 'import httpx2\|from httpx2' src/httpware/` returns matches only inside `transports/httpx2.py` -**And** tests cover: success path, each mapped exception class for representative httpx2 exceptions, status-code mapping for 200/400/401/403/404/409/422/429/500/503 - -### Story 1.5: ResponseDecoder protocol and pydantic adapter - -As a consumer developer, -I want to decode response bodies into pydantic models in a single parse pass, -So that `response_model=User` returns a typed `User` with minimal overhead. - -**Acceptance Criteria:** - -**Given** the `Response` type from Story 1.2 -**When** I implement `src/httpware/decoders/__init__.py` and `src/httpware/decoders/pydantic.py` -**Then** `ResponseDecoder` is a `@runtime_checkable Protocol` with method `def decode(self, content: bytes, model: type[T]) -> T` -**And** `PydanticDecoder.decode(content, model)` calls `_get_adapter(model).validate_json(content)` where `_get_adapter` is `@functools.lru_cache(maxsize=None)`-decorated and returns `pydantic.TypeAdapter(model)` -**And** unit tests verify: decoding into a pydantic `BaseModel`, into a `dataclass`, into `list[User]`, into `dict[str, User]`, into a primitive `int` -**And** a benchmark test confirms ≥2× faster than `pydantic.TypeAdapter(model).validate_python(json.loads(content))` on a 5KB JSON payload (NFR3) -**And** a test verifies the cache: 1000 calls to `decode(content, User)` construct exactly one `TypeAdapter` (NFR2) - -### Story 1.6: msgspec decoder via extras - -As a consumer developer with high-throughput needs, -I want to plug in a msgspec decoder via `pip install httpware[msgspec]`, -So that I get faster validation than pydantic with the same `response_model=` API. - -**Acceptance Criteria:** - -**Given** the `ResponseDecoder` protocol from Story 1.5 -**When** I implement `src/httpware/decoders/msgspec.py` -**Then** the module imports `msgspec` at the top and on `ImportError` raises with message `"MsgspecDecoder requires the 'msgspec' extra. Install with: pip install httpware[msgspec]"` -**And** `MsgspecDecoder.decode(content, model)` calls `msgspec.json.decode(content, type=model)` -**And** `pyproject.toml`'s `[project.optional-dependencies]` declares `msgspec = ["msgspec>=0.18"]` -**And** importing `httpware` (without `httpware[msgspec]` installed) does not import `msgspec` (verified by an import-time test) -**And** unit tests cover decoding into `msgspec.Struct` and into pydantic models (msgspec also handles those) - -### Story 1.7: AsyncClient with HTTP methods, response_model, with_options, lifecycle - -As a consumer developer, -I want a single `AsyncClient` class that I can construct, use, and close — issuing HTTP requests with optional typed responses, -So that I have the v0.1.0 entry point of the library. - -**Acceptance Criteria:** - -**Given** Stories 1.2–1.6 -**When** I implement `src/httpware/client.py` -**Then** `AsyncClient.__init__` accepts (keyword-only): `base_url: str | None`, `default_headers: Mapping[str, str] | None`, `default_query: Mapping[str, str] | None`, `timeout: Timeout | float | None`, `limits: Limits | None`, `transport: Transport | None`, `decoder: ResponseDecoder | None`, `middleware: list[Middleware] | None` (parameter present but ignored — wired in Epic 2) -**And** if `transport` is omitted, a default `Httpx2Transport` is constructed from `limits` and `timeout` -**And** if `decoder` is omitted, a `PydanticDecoder` is used -**And** `AsyncClient.from_url(base_url, **kwargs)` is a classmethod returning an `AsyncClient` -**And** the client implements `__aenter__` returning `self` and `__aexit__` calling `transport.aclose()` -**And** methods `get`, `post`, `put`, `patch`, `delete`, `head`, `options`, `request` all exist with overloads such that `response_model: type[T] | None = None` returns `T` when `T` is provided and `Response` when `None`; `ty` validates the overload against an example consumer -**And** every method accepts per-call overrides: `headers`, `params`, `cookies`, `timeout`, `json`, `data`, `files`, `content` -**And** `client.with_options(**overrides)` returns a new `AsyncClient` sharing the same `transport` instance -**And** integration tests issue a real GET to `httpbingo.org/json`, decode into a pydantic model, assert success -**And** unit tests verify `with_options` returns a different instance with the same transport reference - -### Story 1.8: RecordedTransport for testing - -As a consumer developer, -I want a built-in `RecordedTransport` test double, -So that I can write tests without `respx` and without mocking transport-level types. - -**Acceptance Criteria:** - -**Given** the `Transport` protocol from Story 1.4 and `Response` from Story 1.2 -**When** I implement `src/httpware/transports/recorded.py` -**Then** `RecordedTransport(routes: Mapping[tuple[str, str], Response | Exception])` constructs the transport with a route table keyed by `(method, url_pattern)` -**And** `await transport(request)` looks up `(request.method, request.url)` in routes; on match, returns the `Response` or raises the `Exception`; on no match, raises `RuntimeError(f"No route for {request.method} {request.url}")` -**And** every received `Request` is appended to `transport.calls: list[Request]` -**And** url-pattern matching supports exact match (v0.1.0 scope; regex/glob deferred) -**And** unit tests verify: response side-effect, exception side-effect, `.calls[0].method` and `.calls[0].url` inspection, raise on missing route -**And** a documentation example shows a 3-line pytest fixture wiring `RecordedTransport` to `AsyncClient` and asserting on `.calls` - ---- - -## Epic 2: Compose request-handling logic via middleware - -A developer writes custom middleware (signing, correlation IDs, tracing) and composes it into their client. - -### Story 2.1: Middleware protocol, Next type, and chain composition - -As a library author, -I want a `Middleware` protocol with explicit `Next` semantics and a chain composer, -So that built-in and user middleware live on the same axis and compose correctly. - -**Acceptance Criteria:** - -**Given** the `Request`, `Response`, `Transport` types from Epic 1 -**When** I implement `src/httpware/middleware/__init__.py` and `src/httpware/_internal/chain.py` -**Then** `Next` is exported as `type Next = Callable[[Request], Awaitable[Response]]` (PEP 695 if 3.12+, else `TypeAlias`) -**And** `Middleware` is a `@runtime_checkable Protocol` with `async def __call__(self, request: Request, next: Next) -> Response` -**And** `compose(middlewares: list[Middleware], transport: Transport) -> Next` returns a coroutine that, when called with a `Request`, invokes the outermost middleware, which receives a `Next` that calls the second middleware, …, with the bottom of the chain calling `transport.__call__` -**And** an empty middleware list composes to a `Next` that calls `transport.__call__` directly -**And** unit tests verify: ordering (outer-to-inner), short-circuit (a middleware that doesn't call `next` returns its synthesized `Response`), `CancelledError` propagation through every middleware (NFR15) - -### Story 2.2: Phase-shortcut decorators - -As a consumer developer, -I want `@before_request`, `@after_response`, `@on_error` decorators, -So that I can write simple lifecycle hooks without authoring a full `Middleware` class. - -**Acceptance Criteria:** - -**Given** the `Middleware` protocol from Story 2.1 -**When** I implement decorator helpers in `src/httpware/middleware/__init__.py` -**Then** `@before_request` wraps `async def f(req: Request) -> Request` into a `Middleware` that applies `f` then calls `await next(req)` -**And** `@after_response` wraps `async def f(req: Request, resp: Response) -> Response` into a `Middleware` that calls `await next(req)` then applies `f` -**And** `@on_error` wraps `async def f(req: Request, exc: BaseException) -> Response | None` into a `Middleware` that catches `Exception` (NOT `BaseException`, so `CancelledError` propagates), calls `f`; if `f` returns a `Response`, return it; if `f` returns `None`, re-raise; never catches `CancelledError` -**And** unit tests verify each phase shortcut composes correctly with other middleware and respects `CancelledError` - -### Story 2.3: Request immutability helpers - -As a consumer developer, -I want ergonomic `with_*` mutators on `Request`, -So that middleware can rewrite requests immutably. - -**Acceptance Criteria:** - -**Given** the `Request` type from Story 1.2 -**When** I extend `Request` with additional helpers -**Then** `with_header(name: str, value: str) -> Request` returns a new `Request` with the header added or replaced -**And** `with_headers(headers: Mapping[str, str]) -> Request` returns a new `Request` with the supplied headers merged in -**And** `with_url(url: str) -> Request`, `with_body(body: bytes | None) -> Request`, `with_query(params: Mapping[str, str]) -> Request` exist and behave consistently -**And** every `with_*` returns a new `Request`; the original is unchanged (verified by tests) -**And** the same helpers exist on `Response` where they make sense (`with_headers`, `with_status`) - -### Story 2.4: Auth coercion as middleware - -As a consumer developer, -I want to pass `auth=` as a string, callable, or full `Middleware`, -So that simple cases are one-liners and complex cases are still possible. - -**Acceptance Criteria:** - -**Given** the `Middleware` protocol from Story 2.1 -**When** I implement auth normalization in `src/httpware/middleware/__init__.py` (or `_internal/auth.py`) -**Then** `_normalize_auth(value: str | Callable[[], str | Awaitable[str]] | Middleware | None) -> Middleware | None` returns a `Middleware` -**And** `_normalize_auth("token")` returns a middleware that adds `Authorization: Bearer token` header to every request -**And** `_normalize_auth(lambda: "token")` returns a middleware that calls the callable per request and adds the header -**And** `_normalize_auth(my_middleware)` returns `my_middleware` unchanged -**And** `_normalize_auth(None)` returns `None` -**And** unit tests verify each branch and that the bearer-scheme middleware is the second-to-innermost (just outside transport) when auto-added - -### Story 2.5: Wire middleware into AsyncClient - -As a consumer developer, -I want my supplied `middleware=[...]` list to actually run when I issue requests, -So that the framework's extensibility is real. - -**Acceptance Criteria:** - -**Given** Stories 2.1–2.4 and the `AsyncClient` from Story 1.7 -**When** I update `AsyncClient.__init__` and the request-issuing methods -**Then** the constructor composes `middleware + ([_normalize_auth(auth)] if auth else [])` with `compose(...)` against the transport, storing the resulting `Next` callable on `self._dispatch` -**And** `client.get(url, ...)` builds a `Request` and awaits `self._dispatch(req)` instead of `self._transport(req)` directly -**And** an integration test passes `middleware=[trace_middleware, sign_middleware]` and verifies both ran in declared order -**And** `client.with_options(middleware=[...])` returns a new client with a recomposed chain (not sharing the cached `_dispatch` of the parent) - ---- - -## Epic 3: Survive upstream failures with composable resilience - -A developer's client survives 429s, retries idempotent methods, doesn't retry-storm via the budget, caps per-host concurrency, and times out per-attempt. - -### Story 3.1: Timeout middleware (per-attempt) - -As a consumer developer, -I want a per-attempt timeout that I can configure separately from total request time, -So that long retries don't compound into runaway operations. - -**Acceptance Criteria:** - -**Given** the `Middleware` protocol from Epic 2 -**When** I implement `src/httpware/middleware/timeout.py` -**Then** `Timeout(seconds: float)` is a middleware that wraps `await next(req)` in `asyncio.timeout(seconds)` -**And** on timeout, raises `httpware.TimeoutError(...)` with `request_method`, `request_url` populated -**And** `CancelledError` from outer cancellation propagates without being caught (distinguish via `asyncio.timeout` semantics — outer cancel cancels the whole task; inner timeout fires `TimeoutError`) -**And** unit tests verify: timeout fires when downstream sleeps longer than the limit; outer `task.cancel()` propagates without conversion to `TimeoutError` - -### Story 3.2: Retry middleware - -As a consumer developer, -I want my client to retry transient failures with full-jitter exponential backoff, -So that intermittent upstream errors don't surface to my code. - -**Acceptance Criteria:** - -**Given** the `Middleware` protocol and exception hierarchy -**When** I implement `src/httpware/middleware/retry.py` -**Then** `Retry(max_attempts: int = 3, base_delay: float = 0.5, max_delay: float = 8.0, retryable_statuses: frozenset[int] = frozenset({429, 500, 502, 503, 504}), retryable_exceptions: tuple[type[BaseException], ...] = (TransportError, TimeoutError), idempotent_methods: frozenset[str] = frozenset({"GET", "HEAD", "PUT", "DELETE"}), respect_retry_after: bool = True)` is the constructor -**And** the middleware retries only on idempotent methods OR if the request has an explicit `extensions["idempotent"] = True` marker -**And** backoff delay = `random.uniform(0, base_delay * 2 ** (attempt - 1))`, capped at `max_delay`; if response carries `Retry-After` (seconds or HTTP-date) AND `respect_retry_after`, use that instead -**And** `CancelledError` short-circuits the retry loop (NFR15) -**And** unit tests cover: retry on 503, retry on `TransportError`, no retry on 4xx (except 429), no retry on POST without explicit marker, `Retry-After` honored, max-attempts respected, full-jitter distribution sampled - -### Story 3.3: RetryBudget data structure - -As a library author, -I want a concurrency-safe token-bucket data structure with monotonic-clock refill, -So that the retry budget primitive can be tested in isolation before being wrapped as middleware. - -**Acceptance Criteria:** - -**Given** stdlib `asyncio` and `time` only (no third-party concurrency libs) -**When** I implement `src/httpware/_internal/clock.py` (`monotonic()` wrapper) and `src/httpware/middleware/retry_budget.py`'s internal `_TokenBucket` class -**Then** `_TokenBucket(min_per_sec: float, ratio: float, ttl: float)` stores `tokens_remaining: float`, `last_refill_at: float`, an `asyncio.Lock` -**And** `await _TokenBucket.try_acquire(cost: float = 1.0) -> bool` refills based on monotonic-clock delta then attempts to deduct `cost`; returns True on success, False on insufficient tokens -**And** the lock is held only across the refill+deduct, never across an `await` to user code -**And** Hypothesis property tests (`test_middleware_retry_budget_props.py`) cover ≥10,000 trials with concurrent acquires from `asyncio.gather` and verify: tokens never go negative, refill rate honors `min_per_sec` floor and `ratio` cap, no double-spend -**And** the test runs in CI as part of the regular pytest suite (no special invocation) - -### Story 3.4: RetryBudget middleware integration - -As a consumer developer, -I want a retry budget wrapped into a middleware with state-inspection API, -So that I can cap retry traffic across my whole client and observe budget state from `/healthz`. - -**Acceptance Criteria:** - -**Given** the `_TokenBucket` from Story 3.3 -**When** I implement `RetryBudget` middleware in `src/httpware/middleware/retry_budget.py` -**Then** `RetryBudget(min_per_sec: float = 10.0, ratio: float = 0.2, ttl: float = 10.0)` is the constructor (Finagle defaults) -**And** the middleware tracks "is this attempt a retry?" via `request.extensions["retry_attempt"]` (set by the `Retry` middleware before re-issuing); if it is a retry, it must acquire a token from the bucket before calling `next`; if acquisition fails, it raises the original exception or returns the original response (whichever it received from upstream) -**And** the middleware exposes public read-only properties `tokens_remaining: float`, `in_use_ratio: float` for `/healthz`-style integration (FR46) -**And** unit tests verify: budget exhaustion under sustained-failure load short-circuits subsequent retries; non-retry attempts are never gated; `tokens_remaining` reflects actual state under concurrent load - -### Story 3.5: Bulkhead middleware - -As a consumer developer, -I want per-host concurrency caps, -So that one slow upstream can't saturate my client's connection pool. - -**Acceptance Criteria:** - -**Given** the `Middleware` protocol -**When** I implement `src/httpware/middleware/bulkhead.py` -**Then** `Bulkhead(max_concurrent: int, key: Callable[[Request], str] = lambda r: urlparse(r.url).hostname or "", on_full: Literal["queue", "fail_fast"] = "queue")` is the constructor -**And** the middleware maintains a `weakref.WeakValueDictionary[str, asyncio.Semaphore]` of per-key semaphores, lazily created -**And** with `on_full="queue"`, requests await semaphore acquisition; with `on_full="fail_fast"`, requests over the cap raise `BulkheadFullError(TransportError)` -**And** `CancelledError` during semaphore acquisition releases the slot (verified by test) -**And** unit tests verify: concurrency cap is enforced; per-host isolation (one slow host doesn't block another); fail-fast raises `BulkheadFullError` immediately - -### Story 3.6: Document the extension slot - -As a third-party middleware author, -I want a documented contract for plugging a circuit-breaker (or any resilience primitive) into the middleware chain, -So that I can ship a reusable `httpware-circuit-breaker` package in v1.x without library changes. - -**Acceptance Criteria:** - -**Given** Epic 2's middleware system and Stories 3.1–3.5 -**When** I update `src/httpware/middleware/__init__.py`'s docstring and `docs/concepts/middleware.md` -**Then** the documented chain ordering names the slot explicitly: `Observability → RetryBudget → Retry → [extension slot] → Bulkhead → Timeout → Transport` -**And** the docs explain when middleware should plug into the slot (per-attempt rejection / accounting; circuit-breaker semantics) vs. another position -**And** an example in `examples/circuit_breaker_with_purgatory.py` demonstrates wrapping `purgatory` as a middleware in the slot, showing it works without library changes (this example is post-MVP and may be a stub in v1.0; the docs section is the deliverable here) - ---- - -## Epic 4: Stream responses without buffering - -A developer streams large downloads or SSE/LLM responses without loading them into memory; pool returns are guaranteed on any exception. - -### Story 4.1: StreamResponse type - -As a consumer developer, -I want a `StreamResponse` distinct from `Response`, -So that I can't accidentally call `.content` on a streaming response and force a buffer. - -**Acceptance Criteria:** - -**Given** the `Response` type from Epic 1 -**When** I extend `src/httpware/response.py` -**Then** `StreamResponse` is a `@dataclass(frozen=True, slots=True)` with fields `status: int`, `headers: Mapping[str, str]`, `url: str`, and private fields `_stream: AsyncIterator[bytes]`, `_release: Callable[[], Awaitable[None]]` -**And** `iter_bytes(chunk_size: int = 8192)` returns an async iterator yielding `bytes` chunks from `_stream` -**And** `iter_text(chunk_size: int = 8192, encoding: str | None = None)` decodes incrementally; encoding inferred from `Content-Type` header if not supplied -**And** `iter_lines()` yields decoded lines split on `\n`, handling chunk boundaries -**And** `StreamResponse` does NOT have `.content`, `.text`, or `.json()` attributes (compile-time hint via `__slots__`) -**And** unit tests with a stub `_stream` verify each iterator produces the expected output - -### Story 4.2: Transport.stream implementation in Httpx2Transport - -As a library author, -I want `Httpx2Transport.stream` to actually open a streaming response and yield it via the `StreamResponse` type, -So that the streaming path through Transport works end-to-end. - -**Acceptance Criteria:** - -**Given** the `Transport` protocol from Story 1.4 (which already declares `stream`) and `StreamResponse` from Story 4.1 -**When** I implement `Httpx2Transport.stream` in `src/httpware/transports/httpx2.py` -**Then** `transport.stream(request)` is an `@asynccontextmanager` that calls `httpx2.AsyncClient.stream(...)`, wraps the resulting httpx2 response into a `StreamResponse` with `_stream = response.aiter_raw()` and `_release = response.aclose` -**And** the context manager calls `_release` in a `finally` block, including on `CancelledError` propagation (NFR16) -**And** httpx2 exception → httpware exception mapping is applied at this seam, same table as `__call__` -**And** `RecordedTransport.stream` is also implemented: yields a `StreamResponse` whose `_stream` iterates over a pre-supplied list of byte chunks -**And** integration tests verify: stream a 1MB response from `httpbingo.org/stream-bytes/1048576` without exceeding 100KB resident memory beyond baseline; stream is properly released when the consumer breaks out of the iteration early - -### Story 4.3: AsyncClient.stream context manager - -As a consumer developer, -I want `async with client.stream("GET", url) as resp:` semantics on the public client, -So that I can write SSE/LLM/large-download code idiomatically. - -**Acceptance Criteria:** - -**Given** Stories 4.1, 4.2 and the `AsyncClient` from Epic 1 -**When** I add `AsyncClient.stream` to `src/httpware/client.py` -**Then** `client.stream(method: str, url: str, **kwargs) -> AbstractAsyncContextManager[StreamResponse]` builds a `Request` and delegates to `transport.stream(request)`, ensuring the middleware chain is also applied (chain composition for streaming is identical to non-streaming, but the `Next` returns a `StreamResponse` instead of `Response`) -**And** if a middleware in the chain doesn't support streaming (returns `Response` instead of `StreamResponse`), the framework raises a clear `TypeError` at construction time, not at request time -**And** on consumer-raised exceptions inside the `async with`, `_release` is still called -**And** integration test with `RecordedTransport`: stream a multi-chunk response, raise an exception mid-iteration, verify `_release` was called - ---- - -## Epic 5: Observe and instrument the client - -A developer instruments their client with lifecycle hooks, integrates with OpenTelemetry exporters, and ships with sensible secret redaction. - -### Story 5.1: Layer 1 observability middleware (lifecycle hooks) - -As a consumer developer, -I want to register lifecycle callbacks for request start/complete/retry/timeout/error events, -So that I can wire my own logging or metrics without depending on OpenTelemetry. - -**Acceptance Criteria:** - -**Given** the `Middleware` protocol -**When** I implement `src/httpware/middleware/observability.py` -**Then** `Observability(on_request_start: Callable[[Request], None] | None = None, on_request_complete: Callable[[Request, Response], None] | None = None, on_retry_attempt: Callable[[Request, int, float], None] | None = None, on_retry_budget_exhausted: Callable[[Request], None] | None = None, on_timeout: Callable[[Request, str], None] | None = None, on_exception: Callable[[Request, BaseException], None] | None = None)` is the constructor -**And** the middleware is the **outermost** in the default chain composition order (per Decision 11) -**And** every hook is awaitable-aware: if a hook returns a coroutine, it is awaited; otherwise it's called synchronously -**And** if a hook raises, the exception is caught and logged via `logging.getLogger("httpware").debug(...)`, never propagated (an instrumentation bug must not break the request flow) -**And** unit tests verify each hook fires at the right point, async and sync hook variants both work, and a hook exception doesn't break the request - -### Story 5.2: Wire emission into resilience middlewares - -As an `Observability` middleware, -I want `Retry`, `RetryBudget`, `Bulkhead`, and `Timeout` middlewares to publish their internal events to me, -So that my hooks fire at the right moments. - -**Acceptance Criteria:** - -**Given** Story 5.1 and the resilience middlewares from Epic 3 -**When** I add an event-bus-style mechanism (single shared `EventEmitter` instance attached to `Observability`) -**Then** `Retry` calls `emitter.emit("retry_attempt", request, attempt, delay)` before each retry -**And** `RetryBudget` calls `emitter.emit("retry_budget_exhausted", request)` when token acquisition fails -**And** `Timeout` calls `emitter.emit("timeout", request, "read")` (or similar phase) on timeout firing -**And** `Bulkhead` calls `emitter.emit("bulkhead_full", request)` on `fail_fast` rejection -**And** `Observability` translates emitter events into the public hook callbacks (`on_retry_attempt` etc.) -**And** if no `Observability` middleware is in the chain, emitter calls are no-ops (zero overhead) -**And** unit tests verify each event flows through to the corresponding hook - -### Story 5.3: Redactor class and integration - -As a consumer developer, -I want secret-bearing headers redacted from every framework emission point by default, -So that `Authorization` tokens don't end up in my Grafana board or logs. - -**Acceptance Criteria:** - -**Given** the `Request`, `Response`, and exception types from Epic 1 -**When** I implement `Redactor` in `src/httpware/config.py` and integrate it -**Then** `Redactor(headers: frozenset[str] = DEFAULT_REDACTED_HEADERS, redact_bodies: bool = True)` is the constructor -**And** `DEFAULT_REDACTED_HEADERS = frozenset({"authorization", "cookie", "set-cookie", "x-api-key", "x-auth-token", "proxy-authorization"})` (case-insensitive matching) -**And** `Redactor.redact_headers(headers)` returns a new mapping with redacted values replaced by `""` -**And** `Redactor.redact_body(body)` returns `b""` if `redact_bodies` is True, else returns body unchanged -**And** the default `Redactor` is on the default `ClientConfig`; users can override via `AsyncClient(redactor=...)` -**And** `Request.__repr__`, `Response.__repr__`, every exception's `__repr__` invoke the client's `Redactor` (passed via `extensions["redactor"]` or a context var) before emission -**And** unit tests verify: default headers are redacted; custom redactor with extra headers works; bodies redacted by default; `redact_bodies=False` preserves body - -### Story 5.4: OpenTelemetry middleware - -As a consumer developer with OpenTelemetry infrastructure, -I want a drop-in OTel middleware that emits semantic-convention-conformant spans and metrics, -So that my Grafana dashboards "just work" without me writing translation code. - -**Acceptance Criteria:** - -**Given** Stories 5.1, 5.2, 5.3 and `pip install httpware[otel]` -**When** I implement `src/httpware/middleware/_otel.py` -**Then** the module imports `opentelemetry.trace`, `opentelemetry.metrics` at the top with the standard `try/except ImportError` install-hint pattern -**And** `OpenTelemetryMiddleware(tracer_provider=None, meter_provider=None)` constructor accepts optional OTel providers (defaults to global) -**And** every request creates a span named `"HTTP {method}"` with attributes per OTel HTTP-client semconv: `http.request.method`, `url.full`, `server.address`, `server.port`, and on response: `http.response.status_code` -**And** sensitive header values are redacted via the client's `Redactor` before becoming span attributes -**And** the middleware emits the histograms `http.client.request.duration` and counters per the semconv -**And** a CI test imports the OTel semconv schema package and asserts the middleware's emitted attribute names are a subset of the schema's HTTP-client attributes (NFR19) -**And** integration tests verify spans are created and exported via an in-memory test exporter - -### Story 5.5: Logging policy enforcement - -As a library maintainer, -I want a CI gate that prevents anyone from accidentally adding `print()` or top-level logging configuration, -So that the no-global-logging guarantee (NFR47) is enforced rather than convention-only. - -**Acceptance Criteria:** - -**Given** the `httpware` source tree -**When** I add a CI grep step -**Then** `grep -rn 'print(' src/httpware/` returns zero matches (or only matches inside `# noqa`-annotated docstrings, of which there should be none) -**And** `grep -rn 'logging.basicConfig\|logging.getLogger()' src/httpware/` returns zero matches (`logging.getLogger("httpware")` and its sub-loggers are allowed; the bare `getLogger()` form is not) -**And** `pytest -W error::Warning` runs the test suite with warnings as errors and passes — the framework must not emit any warnings during unconfigured use -**And** the CI step fails the build on violation - ---- - -## Epic 6: Ship v1.0 - -A `base-client` consumer reads the migration guide, swaps imports, and ships on `httpware` within a few hours. The library is publicly trustable. - -### Story 6.1: Migration guide from base-client - -As a `base-client` consumer, -I want a step-by-step migration guide with side-by-side examples, -So that I can move my service to `httpware` in a day, not a week. - -**Acceptance Criteria:** - -**Given** all prior epics complete and a working `httpware` v0.x -**When** I author `docs/migration-from-base-client.md` -**Then** the guide includes: a "Why migrate" section linking to the PRD and noting the `encode/httpx` → `pydantic/httpx2` transition that base-client consumers need to handle anyway; an at-a-glance per-symbol replacement table covering `httpx.AsyncClient`/`Request`/`Response`/`HTTPStatusError`/`codes.is_*`/`_client.USE_CLIENT_DEFAULT`/`Timeout`, plus `respx.mock`, `circuit_breaker_box.Retrier`, and `tenacity.retry` -**And** six step-by-step migration steps, each with a "before" and "after" code block: replace AsyncClient construction; update Response handling; replace error handling; replace respx with RecordedTransport; remove Retrier and tenacity decorators; verify OTel hookup -**And** a side-by-side reference appendix migrating one full example service from `base-client/examples/` to `httpware` -**And** a "Gotchas" section calling out: exception fields are plain types not `httpx.Response` (and not `httpx2.Response`); `auth=` union accepts string for static bearer tokens; `with_options` returns a new client (not mutates); the underlying transport changed from `encode/httpx` to `pydantic/httpx2` but this is invisible through the `httpware.*` types -**And** the migration guide is referenced from the `httpware` README as the recommended on-ramp - -### Story 6.2: Documentation site (mkdocs) - -As a potential adopter, -I want a hosted documentation site with quickstart, concepts, recipes, and API reference, -So that I can evaluate `httpware` and integrate it without reading the source. - -**Acceptance Criteria:** - -**Given** the source code of all prior epics -**When** I author the `docs/` tree and `mkdocs.yml` -**Then** the docs build with `mkdocs build` without warnings or broken links (CI-enforced) -**And** the structure matches the architecture's Documentation Organization section (`index.md`, `quickstart.md`, `migration-from-base-client.md`, `concepts/{middleware,transports,decoders,retries-and-budget,exceptions}.md`, `recipes/{custom-middleware,authentication,observability,testing}.md`, `api/` auto-generated via `mkdocstrings`) -**And** the site is hosted on Read the Docs (or GitHub Pages) and a build is triggered from `main` and tags -**And** the README links to the hosted docs URL -**And** every public symbol has at least a one-line docstring (verified by a CI test that imports `httpware` and asserts every name in `__all__` has a non-empty `__doc__`) - -### Story 6.3: Public benchmark suite - -As a potential adopter and as the maintainer, -I want a public benchmark comparing `httpware` to raw `httpx2 + tenacity`, -So that the "is this slow?" objection is closed pre-emptively and we catch perf regressions. - -**Acceptance Criteria:** - -**Given** all prior epics -**When** I implement `benchmarks/` and add a `just bench` recipe -**Then** the suite measures end-to-end request latency for: `httpware` with default config + `response_model=User`; raw `httpx2.AsyncClient.get(...)` + manual `pydantic.TypeAdapter(User).validate_json(...)` + `tenacity.retry`; baseline `httpx2.AsyncClient.get(...)` no validation no retry -**And** the workload is 1000 sequential requests against `httpbingo.org/json` returning a 5KB JSON response, plus a 100-RPS concurrent variant -**And** the benchmark output is a Markdown table appended to `benchmarks/RESULTS.md` with median, p95, p99 latency -**And** the CI job runs the benchmark on every release tag and posts the table as a comment on the release -**And** the per-request overhead delta is asserted to be ≤15% (NFR1); the build fails the release if exceeded - -### Story 6.4: CI enforcement gates - -As a library maintainer, -I want CI gates that enforce the architectural invariants automatically, -So that PRs that violate them never merge. - -**Acceptance Criteria:** - -**Given** the source tree and CI workflow from Story 1.1 -**When** I extend `.github/workflows/ci.yml` -**Then** a step runs `! grep -rE 'import httpx2|from httpx2' src/httpware/ tests/ examples/ | grep -v 'src/httpware/transports/httpx2.py'` and fails the build if it finds matches outside `transports/httpx2.py` -**And** a step runs `! grep -rE 'httpx2\._' src/httpware/` and fails on any private-API usage (NFR4 / FR16) -**And** an `__all__`-snapshot test asserts that `set(httpware.__all__)` equals a frozenset literal in `tests/test_api_surface.py`; changes require updating both files (catches accidental public-API additions per NFR18) -**And** the OTel-conformance test from Story 5.4 runs in CI -**And** the no-print / no-basicConfig grep from Story 5.5 runs in CI -**And** the property-test job from Story 3.3 runs as part of the standard pytest invocation -**And** all gates run on every push and PR; build fails on any violation - -### Story 6.5: Release flow with Trusted Publishers and Sigstore - -As a security-conscious consumer of `httpware`, -I want releases signed via PyPI Trusted Publishers with Sigstore attestation and an attached SBOM, -So that I can verify the artifact's provenance. - -**Acceptance Criteria:** - -**Given** a PyPI account configured with Trusted Publishers for `modern-python/httpware` -**When** I push a tag matching `vX.Y.Z` -**Then** `.github/workflows/publish.yml` triggers, builds the wheel and sdist via `uv build`, generates an SBOM (CycloneDX format) via `cyclonedx-py` or equivalent, and uploads to PyPI via Trusted Publishers (no PyPI API token in repo secrets) -**And** Sigstore attestation is automatically attached to the upload -**And** the SBOM is also uploaded as a release asset on the GitHub Release -**And** the release notes are auto-extracted from `CHANGELOG.md` for the matching version -**And** `SECURITY.md` documents the disclosure channel (`security@...` or GitHub Security Advisories) with a 90-day private-disclosure commitment (NFR10) -**And** a manual smoke test verifies: install from PyPI in a clean venv after the release, verify the Sigstore attestation with `cosign verify-attestation`, import `httpware`, run the quickstart example diff --git a/docs/archive/prd.md b/docs/archive/prd.md deleted file mode 100644 index bb31559..0000000 --- a/docs/archive/prd.md +++ /dev/null @@ -1,692 +0,0 @@ ---- -stepsCompleted: - - step-01-init - - step-02-discovery - - step-02b-vision - - step-02c-executive-summary - - step-03-success - - step-04-journeys - - step-05-domain-skipped - - step-06-innovation - - step-07-project-type - - step-08-scoping - - step-09-functional - - step-10-nonfunctional - - step-11-polish - - step-12-complete -status: complete -releaseMode: phased -inputDocuments: - - docs/product-brief-httpware.md - - docs/product-brief-httpware-distillate.md -workflowType: prd -project_name: httpware -classification: - projectType: developer_tool - domain: general - complexity: low - projectContext: greenfield ---- - -# Product Requirements Document - httpware - -**Author:** Artur Shiriev -**Date:** 2026-05-11 -**Updated:** 2026-05-12 — reflects the `pydantic/httpx2` fork; transport switched from `encode/httpx` 0.28 to `pydantic/httpx2` 2.0.0b1. - -## Executive Summary - -`httpware` is a Python async HTTP client framework for building resilient service clients. It supersedes `community-of-python/base-client` (to be deprecated) and ships under `github.com/modern-python`. - -The framework owns the abstraction layer above the underlying HTTP client (`httpx2` by default; niquests planned). Consumers never import the transport; swapping it is a constructor argument. Resilience primitives — retries, timeouts, bulkheads, and a Finagle-style retry budget — are composable middleware; circuit breakers have a stable extension point but are not implemented in v1.0. Tests use a `RecordedTransport` and assert against plain exception fields (`status`, `body`, `headers`, `json`), never against the underlying client's types. - -**Target users:** -- *Primary:* Backend Python teams in `modern-python` and partner orgs building async service-to-service clients (FastAPI-era backends). -- *Secondary:* Teams building LLM and AI-gateway clients — high-volume, high-failure HTTP workloads where middleware-composed resilience is hand-rolled today. -- *Tertiary:* Wider Python community building service clients on PyPI. - -**Problem solved:** Python has no canonical resilience-first HTTP framework. Existing wrappers (including `base-client`) leak transport types through their public APIs, making any transport migration — including the just-happened `encode/httpx` → `pydantic/httpx2` transition — a breaking change for every downstream consumer. Available resilience libraries are buggy or unmaintained (`circuit-breaker-box` has 5 verified critical bugs). - -**Why now:** The `encode/httpx` → `pydantic/httpx2` fork (2026-05-11) is a forcing function. `base-client` cannot be salvaged in place (it imports from `httpx._client` and `httpx._types` private modules), and every consumer needs an httpx-to-httpx2 path anyway. Rebuilding the wrapper as a transport-agnostic framework is the moment to also close the resilience-framework gap that's been open in Python (analogous to Polly on .NET, resilience4j on JVM) since async HTTP became standard. - -### What Makes This Special - -**Core insight:** The Python async-HTTP problem isn't which client to pick. It's that no framework owns the layer *above* the client. Owning that layer turns transport choice into a reversible decision and resilience into composable primitives — both problems collapse into one architectural move. - -**Differentiators:** -1. **Transport-agnostic public API.** `httpware.Request`, `httpware.Response`, `httpware.Transport`, `httpware.Middleware` are first-class types. The underlying client sits behind a small `Transport` protocol. No public symbol references `httpx2`. -2. **Onion middleware with phase shortcuts.** `Observability → RetryBudget → Retry → [extension slot] → Bulkhead → Timeout → Transport`. Built-in primitives and user middleware live on the same axis. Third-party circuit breakers plug into the extension slot without library changes. -3. **Retry budget by default.** Token-bucket admission control (Finagle defaults: 20% retry ratio, 10/sec floor, 10s TTL). The most effective single control against retry storms — absent from every popular Python HTTP library. -4. **Pluggable validation.** `response_model=` accepts any type via a `ResponseDecoder` protocol. Default pydantic adapter (cached `TypeAdapter`, `validate_json(content)`); msgspec adapter shipped. Same anti-leakage discipline applied to validation. -5. **`RecordedTransport` for tests.** Mocking is a 3-line fixture keyed on `(method, url) → Response`. No respx routes, no transport-level mocking, no httpx2-typed assertions. -6. **Stainless-pattern typed responses.** `await client.get("/users/1", response_model=User)` returns a typed `User`. `with_options(...)` returns a new client sharing the pool. Granular status-keyed exception hierarchy with plain fields. - -**Positioning:** `httpware` is to Python what Polly is to .NET and resilience4j is to the JVM — a canonical resilience-first HTTP framework. Transport-agnosticism is the proof point that lets it stand independent of any underlying client's governance trajectory. - -## Project Classification - -| Field | Value | -|---|---| -| **Project Type** | `developer_tool` — pip-installable Python library/framework | -| **Domain** | `general` — Python developer infrastructure (no regulated-industry constraints) | -| **Domain Complexity** | `low` — implementation complexity is moderate-to-high (async, resilience, transport abstraction) but domain/compliance load is minimal | -| **Project Context** | `greenfield` — new library; supersedes `community-of-python/base-client` (to be deprecated) | -| **Org** | `github.com/modern-python` | - -## Success Criteria - -### User Success - -`httpware` succeeds for its users (Python service-team developers) when: - -- **Time-to-functional-client.** A new resilient service client (base URL + 3 endpoint methods + tests + typed responses + retries + retry budget) takes ≤50 LOC of consumer code, including imports. -- **Test ergonomics.** A unit test that mocks an HTTP call requires ≤3 lines of fixture setup via `RecordedTransport`. Zero references to `httpx2.*` or `respx.*` in consumer test code that is not specifically integration-testing the transport. -- **Transport reversibility.** Switching from `Httpx2Transport` to `NiquestsTransport` (once shipped) is a one-line constructor change and requires zero changes to consumer code — verified by migrating one example project. -- **Error handling.** Consumer code catches `httpware.NotFoundError`, `httpware.RateLimitedError`, etc. with plain fields (`e.status: int`, `e.body: bytes`, `e.headers`, `e.json`). Zero `except httpx2.*` clauses in any migrated codebase. -- **Discoverability.** A consumer can answer "how do I add request signing?" by reading the middleware authoring guide and writing one class in <30 LOC. -- **Aha moment.** When a developer migrating from `base-client` deletes the per-test `respx.route(...).mock(...)` boilerplate and replaces it with a 3-line `RecordedTransport`, AND when they delete the bolt-on tenacity decorators because retry policy is now a constructor argument. - -### Business Success - -For an OSS library shipping under `modern-python`: - -- **3 months post-v1.0:** ≥1 production service inside `modern-python` (or partner org) migrated from `base-client` to `httpware` and stable for ≥30 days. -- **6 months post-v1.0:** All known `base-client` consumers migrated. `community-of-python/base-client` repo archived with a README pointer to `httpware`. -- **6 months post-v1.0:** ≥3 external PyPI projects (non-`modern-python`, non-`community-of-python`) declaring `httpware` as a dependency. -- **12 months post-v1.0:** Library cited in ≥2 external sources (blog post, conference talk, Awesome-Python list) as the recommended Python resilience-first HTTP framework. - -### Technical Success - -Hard, verifiable criteria: - -- `grep -r 'import httpx2\|from httpx2' httpware/ tests/ examples/` returns matches only inside `httpware/transports/httpx2.py` (the transport adapter module). -- `grep -r 'httpx2\._' httpware/` returns zero matches (no httpx2 private-API usage anywhere — this is the bar `base-client` currently fails with `encode/httpx`). -- Property-based test suite (Hypothesis) for `RetryBudget` admission control passes ≥10,000 trials covering concurrent failure scenarios with no race-condition failures or invariant violations. -- Performance budget: per-request overhead measured as the wall-clock delta of `client.get(url, response_model=User)` vs raw `httpx2.AsyncClient().get(url)` + manual `pydantic.TypeAdapter(User).validate_json(resp.content)` is ≤15% on typical 5KB JSON payloads at 100 RPS sustained. Benchmark published with each release. -- ≥90% line coverage on `httpware/` core (transports excluded, since transport adapters are largely passthrough). -- All public types `py.typed`; `ty` (Astral) passes on `httpware/` and on a reference consumer. -- Python 3.11+ supported (3.11, 3.12, 3.13, 3.14 when available) on CI. - -### Measurable Outcomes - -Tracked publicly in the repo: - -| Outcome | Metric | Target | -|---|---|---| -| Migration acceptance | `modern-python` services on `httpware` | 1 (3 mo), all known (6 mo) | -| External adoption | PyPI dependents (non-modern-python) | ≥3 (6 mo) | -| Performance honesty | Published benchmark vs raw httpx2+pydantic | Every release | -| Quality bar | Property-based test trials passing | ≥10,000 | -| API hygiene | httpx2 imports outside `transports/httpx2.py` | 0 | -| Test ergonomics | Lines of test fixture code in migrated consumers | ≤3 per test | - -## Product Scope - -### MVP — Minimum Viable Product - -Everything required for a `modern-python` team to migrate one production service off `base-client`: - -- **Async API.** `AsyncClient` with `get`/`post`/`put`/`patch`/`delete`/`head`/`options`/`request` methods. Async-only; no sync facade. -- **`Transport` protocol** + default **`Httpx2Transport`** wrapping `httpx2.AsyncClient` and adapting types at the seam. Sensible default `Timeout(connect=5, read=30, write=30, pool=5)` and `Limits(max_connections=100, max_keepalive_connections=20)`. -- **Single-call typed-response API.** `await client.get(url, response_model=T)` returns `T`. `response_model=None` returns a `Response` wrapper. -- **`ResponseDecoder` protocol** + pydantic adapter (cached `TypeAdapter` per model_type, `validate_json(content)`). -- **Middleware system.** Onion model with `Middleware`, `Next`, phase-shortcut helpers `@before_request`, `@after_response`, `@on_error`. Order: `Observability → RetryBudget → Retry → [extension slot] → Bulkhead → Timeout → Transport`. Extension slot is the documented contract for plug-in middleware (notably a future circuit breaker). -- **Built-in middleware (v1.0):** `Retry` (full-jitter exponential backoff, idempotent-method-only by default, Retry-After-aware), `RetryBudget` (Finagle defaults: 20% retry ratio, 10/sec floor, 10s TTL), `Bulkhead` (`asyncio.Semaphore` per key), `Timeout` (per-attempt), `Observability` (hooks + optional OpenTelemetry span integration). -- **Exception hierarchy** keyed by HTTP status with plain fields (`status`, `body`, `headers`, `json`, `request_method`, `request_url`). `TransportError`, `TimeoutError`, `BadRequestError`, `UnauthorizedError`, `ForbiddenError`, `NotFoundError`, `ConflictError`, `UnprocessableEntityError`, `RateLimitedError`, `InternalServerError`, `ServiceUnavailableError`. -- **`RecordedTransport`** test double accepting `{(method, url_pattern): Response | Exception}` and exposing `.calls` for assertions. -- **`with_options(...)`** returning a new client sharing the pool. -- **Streaming** via `async with client.stream(method, url, ...) as resp`, with `iter_bytes`, `iter_text`, `iter_lines`. -- **Auth abstraction** accepting `str | Callable[[], str | Awaitable[str]] | Middleware`. -- **Security defaults:** TLS verification on; configurable secret-redaction hook for log/span emission; CVE disclosure channel documented in `SECURITY.md`. -- **Docs:** README, migration guide from `base-client` (with side-by-side examples), middleware authoring guide, RecordedTransport cookbook. -- **Acceptance:** ≥1 production consumer in `modern-python` running on `httpware` for ≥30 days. - -### Growth Features (Post-MVP) - -Features that make `httpware` competitive beyond the migration use case: - -- **`NiquestsTransport`** — second backend that proves the abstraction. HTTP/2 + HTTP/3 by default. Triggers the "transport reversibility" success metric. -- **Reference circuit-breaker middleware.** Companion package or example wrapping `purgatory`. Validates the extension slot is real and usable. -- **msgspec `ResponseDecoder` adapter** — faster validation path for high-throughput services. -- **OpenTelemetry semantic-convention auto-instrumentation** — spans named per `http.client.request.*` conventions; metric names per `http.client.request.duration`, etc. -- **`client.gather(...)`** — concurrency helper using `TaskGroup` with semaphore-aware queueing, sized against `Limits.max_connections`. -- **FastAPI / Litestar integration recipes** — context-managed dependency injection for shared clients, request-scoped middleware (correlation ID propagation). -- **Public benchmark suite** vs raw httpx2 + tenacity, published with each release as part of the repo (closing the "is this thing slow?" objection pre-emptively). -- **HTTP/2 toggle** for `Httpx2Transport` (default off; documented opt-in). -- **JSON Schema / OpenAPI response validation** as an alternate `ResponseDecoder` for users who don't want pydantic/msgspec. - -### Vision (Future) - -The dream version that comes from being the category leader: - -- **LLM/AI-gateway preset.** A `LLMClient` subclass with token-accounting middleware, SSE/streaming parsers, vendor-failover middleware. Concrete answer to "I'm hand-rolling resilience on top of `openai-python`." -- **Sync API** as a parallel class hierarchy (Stainless pattern), if downstream demand justifies the maintenance double. -- **OpenAPI codegen target.** Generate typed `httpware`-based clients from a spec — each generated client is a distribution vector that brings the middleware ecosystem along. -- **In-house circuit breaker** (3-state, sliding-window, slow-call detection, jittered half-open) — only if the wrapping-`purgatory` path proves inadequate. Detailed reference design captured in the distillate. -- **Distributed resilience-state coordination** beyond a single Redis backend (gossip, multi-region failover). -- **Middleware marketplace / registry** — third-party middleware (auth providers, signing schemes for AWS SigV4/GCP/HMAC, tracing exporters) creating a network effect. - -## User Journeys - -### 1. Maria — Migrating a Service from `base-client` to `httpware` (Primary, success path) - -**Opening scene.** Maria is a senior backend engineer at a `modern-python` partner team. Her FastAPI service depends on `base-client` to call three internal APIs. She has been deferring an upgrade because the last time she opened the codebase she counted 19 `except httpx.*` clauses in tests, three `httpx._client.USE_CLIENT_DEFAULT` references in production code, and a `circuit-breaker-box` configuration nobody on the team understands. The `lovelydinosaur` discussion thread crosses her feed and the migration becomes someone's task — hers. - -**Rising action.** She reads the `httpware` README and the migration guide. The shape feels familiar (httpx-like method names) but the types are different — `httpware.AsyncClient` returns `httpware.Response`, not `httpx.Response`. She runs `grep -r 'httpx' .` on her consumer code and finds 47 hits. The migration guide gives her a per-symbol replacement table. She replaces `httpx.AsyncClient` construction with `httpware.AsyncClient(base_url=..., timeout=...)` (the default `Httpx2Transport` is constructed implicitly). She deletes the `circuit_breaker_box.Retrier` wrapper and the `tenacity` decorators — `httpware` ships retries and a retry budget by default. She rewrites her error handling: `except httpx.HTTPStatusError as e: if e.response.status_code == 404` becomes `except NotFoundError`. She thinks "wait, that's it?" - -**Climax.** She runs the test suite. Most tests fail with `AttributeError: 'RecordedTransport' has no attribute 'mock'`. She replaces `respx.mock` + 5-line route setup with a single dict literal: `RecordedTransport({("GET", "/users/1"): Response(status=200, json={...})})`. Suddenly her test files shrink by 30%. The test that simulated a `httpx.ReadTimeout` becomes `RecordedTransport({(...): TimeoutError()})`. The 19 exception types collapse to 5. - -**Resolution.** Her service runs. The retry-budget metrics show up in her Grafana dashboard automatically via OpenTelemetry. The migration took two afternoons. She archives the team's bookmark to circuit-breaker-box's repo. Her PR review comment from a teammate is "wait, where did all the test boilerplate go?" - -**Capabilities this journey reveals:** -- Migration guide with per-symbol replacement table from `base-client` and from raw `httpx` (both `encode/httpx` and `pydantic/httpx2` paths) -- `httpware.NotFoundError`/`RateLimitedError`/etc. with plain fields -- Defaults that "just work" (retries on, retry budget on, sane Timeout/Limits) -- OpenTelemetry semantic-convention auto-instrumentation -- `RecordedTransport` accepting both response and exception side-effects - -### 2. Dmitri — Greenfield Service Client (Primary, alternate goal) - -**Opening scene.** Dmitri is starting a new internal billing service. It calls four upstream services. He skims the `httpware` README during onboarding. - -**Rising action.** He writes a `BillingApiClient` subclass that holds an `httpware.AsyncClient` instance. Endpoint methods are 3-line `await self._client.get("/invoices/{id}", response_model=Invoice)`. He adds `default_headers={"X-Service": "billing"}` and `auth=lambda: get_token()`. Twenty minutes in, the client compiles, types check, and three endpoint methods are written. - -**Climax.** He needs request signing (HMAC) for one upstream. He reads the middleware authoring guide. He writes a 12-line `SignRequestMiddleware` class implementing the `Middleware` protocol. He passes it to the constructor: `middleware=[SignRequestMiddleware(secret)]`. It works on the first run. - -**Resolution.** The billing client is 47 lines of consumer code, has retries with full-jitter backoff, retry budget, OpenTelemetry tracing, typed responses, and signed requests. He never imported `httpx2`, `tenacity`, or `circuit-breaker-box`. He doesn't notice this; he just notices that he is done. - -**Capabilities this journey reveals:** -- `AsyncClient` constructor with idiomatic kwargs (`default_headers`, `default_query`, `auth`, `middleware`, `timeout`, `base_url`) -- Composable `Middleware` protocol with single `(req, next) -> Response` signature -- `auth: str | Callable | Middleware` union type -- Middleware authoring guide with end-to-end signing example - -### 3. Yulia — Production Outage Debug (Primary, edge case / failure recovery) - -**Opening scene.** 2 a.m. PagerDuty. Yulia's service is throwing `5xx` for ~15% of requests to an upstream payments API. The on-call dashboard shows elevated `http.client.request.duration` p99 but the upstream's status page is green. - -**Rising action.** Yulia opens the Grafana board (the one that `httpware` populates by default via OTel). She sees `circuit_breaker.rejections_total` is flat (good — no breaker tripped) but `retry_budget.tokens_remaining` is at 0. The retry budget is being exhausted, which means a large number of requests are hitting upstream failures and retries are being shed. She sees `http.client.request.duration` for the affected upstream — p99 of 4.2s, up from 600ms. - -**Climax.** She runs `client.with_options(timeout=Timeout(connect=5, read=15, write=10, pool=5)).with_options(retries=0).post(...)` from a debug shell to bypass retries and isolate the upstream. She gets a clean `ServiceUnavailableError` with `e.body` showing the upstream's actual response: "rate-limit exceeded by region quota." - -**Resolution.** She raises the upstream rate-limit ticket. The retry budget protected her service from spiraling into a retry storm during the incident. She updates the runbook to point at the `retry_budget.tokens_remaining` graph as the first thing to check during similar incidents. - -**Capabilities this journey reveals:** -- OpenTelemetry semantic-convention instrumentation (per-request duration, per-host counters) -- `client.with_options(...)` as the ergonomic per-call override for debugging -- Observability hooks for `retry_budget.tokens_remaining`, `retry.attempts_total`, `bulkhead.queued` -- Plain-field exceptions (`e.body`, `e.headers`) usable in ad-hoc debug shells -- Documentation showing the on-call playbook for observability signals - -### 4. Alex — LLM Gateway with Vendor Failover (Secondary user) - -**Opening scene.** Alex maintains an internal LLM proxy service that fronts OpenAI, Anthropic, and an internal model. It uses `openai-python` and `anthropic-sdk-python` directly. The hand-rolled retry logic doesn't compose with the SDKs' built-in retries, and a recent OpenAI degradation took down the proxy for 12 minutes before the team manually flipped traffic to Anthropic. - -**Rising action.** Alex reads the LLM/AI-gateway secondary-user section of the `httpware` docs. He decides to wrap the OpenAI and Anthropic SDKs at the HTTP layer using `httpware`. He configures two `AsyncClient` instances, each with `base_url` pointing at the respective vendor. He writes a `VendorFailoverMiddleware` that, on `ServiceUnavailableError` from one client, retries the request through a second client. - -**Climax.** During a synthetic test, Alex kills the OpenAI route on his test stand. The middleware catches the connection error, calls the Anthropic client, and the response returns within 800ms (vs. the previous 12-minute manual cutover). The retry budget on the OpenAI side prevents thundering-herd retries during the outage. - -**Resolution.** The LLM proxy is now resilient to single-vendor outages. The middleware is 60 LOC. Alex publishes it as `llm-failover-middleware` on PyPI — the first third-party `httpware` middleware in the ecosystem. - -**Capabilities this journey reveals:** -- Streaming response support (LLM responses are SSE/streaming) -- Per-attempt timeouts long enough for LLM workloads (`read=600s` configurable) -- Middleware composition allowing cross-client patterns (failover across two `AsyncClient` instances) -- Public middleware-authoring contract stable enough to publish third-party middleware against -- `httpware` not being prescriptive about what "the" LLM client looks like — composition over inheritance - -### 5. Sergey — Building Custom Tracing Middleware (Developer-author journey) - -**Opening scene.** Sergey is the platform engineer who owns observability standards at his company. The default `httpware` OpenTelemetry middleware emits standard semantic-convention attributes, but his org needs custom attributes (tenant ID from a context var, deployment region from env). - -**Rising action.** He reads the middleware authoring guide. He sees two paths: (a) write a full `Middleware` class, (b) use the `@before_request` and `@after_response` phase shortcuts. He picks (b) for simplicity. Two functions, eight lines total, decorated. He registers them on the client constructor. - -**Climax.** First request fires. Custom attributes show up in his Jaeger trace alongside the default attributes — both compose cleanly. He notices the default OTel middleware ran first, his custom additions ran after. He likes that. - -**Resolution.** He publishes the middleware as an internal package. Two other teams adopt it within a week. - -**Capabilities this journey reveals:** -- `@before_request` and `@after_response` decorators as the ergonomic on-ramp before full `Middleware` classes -- Documented middleware execution order so additive instrumentation composes predictably -- Default OTel middleware is itself middleware (replaceable, not magic) — Sergey's custom middleware sees the same hooks -- Middleware authoring guide with both phase-shortcut and full-class examples - -### Journey Requirements Summary - -Capabilities revealed across the five journeys: - -| Capability area | Journeys requiring it | -|---|---| -| Migration guide (per-symbol replacement, side-by-side examples) | 1 | -| `RecordedTransport` for tests (response + exception side-effects) | 1 | -| Sensible defaults (retries, retry budget, OTel, Timeout, Limits) | 1, 2 | -| Plain-field status-keyed exception hierarchy | 1, 3 | -| `AsyncClient` ergonomic constructor (`base_url`, `auth`, `middleware`, etc.) | 2, 4 | -| `Middleware` protocol + onion model | 2, 4, 5 | -| `@before_request` / `@after_response` phase shortcuts | 5 | -| Middleware authoring guide (phase shortcuts + full class) | 2, 5 | -| `auth: str \| Callable \| Middleware` union | 2 | -| OpenTelemetry semantic-convention auto-instrumentation | 1, 3, 5 | -| `client.with_options(...)` per-call override | 3 | -| Observability metrics: `retry_budget.tokens_remaining`, `retry.attempts_total`, `bulkhead.queued`, breaker rejections | 3 | -| Streaming response support (`async with client.stream(...)`) | 4 | -| Long-duration `read` timeout configurability | 4 | -| Public middleware contract stable for third-party publication | 4, 5 | -| Composable across multiple `AsyncClient` instances | 4 | -| Reasonable on-call documentation / runbook examples | 3 | - -## Innovation & Novel Patterns - -### Detected Innovation Areas - -1. **Retry budget brought to Python.** The Finagle/Envoy retry-budget pattern (token-bucket admission control over the whole client's retry traffic) is well-understood at scale, but no popular Python HTTP library ships it. `httpware` is plausibly the first Python library to make a retry budget a default-on, configurable primitive. This isn't a new pattern — it's a documented best practice that's been absent from the language's ecosystem. - -2. **Library-owned transport abstraction in async Python.** Stripe-python proved the pattern works for sync (its `HTTPClient` ABC switches between `RequestsClient`/`HTTPXClient`/`AIOHTTPClient`). `httpware` brings the same discipline to async — and applies it as a wrapper-framework, not a per-vendor SDK. Owning `Transport` as a first-class protocol that consumers depend on, rather than re-exposing httpx2, is uncommon in Python wrappers (the current `base-client` exemplifies the prevailing leaky pattern with `encode/httpx`). - -3. **Documented "extension slot" in the middleware ordering.** `httpware` commits to a specific onion order with a named, contract-bound slot for plug-in middleware (where a circuit breaker will live). Most middleware systems leave order informal or document it loosely; a named slot is a small but real innovation in surface design. It makes the question "where does my middleware go?" answerable by reading docs, not by experimentation. - -4. **Anti-leakage discipline applied beyond HTTP.** The same protocol-based pluggability applied to transport (`Transport` / `RecordedTransport`) is applied to validation (`ResponseDecoder` / pydantic adapter / msgspec adapter). One framework, two pluggable extension points, same design pattern. The novelty here is consistent execution, not the underlying idea. - -### Market Context & Competitive Landscape - -- **Polly (.NET)** and **resilience4j (JVM)** are the category leaders in their ecosystems. Python's analogous slot is empty — `tenacity` covers retry, `purgatory`/`pybreaker` cover breakers, but no library composes them into a coherent framework with consistent observability and async semantics. -- **Stainless-generated SDKs** (openai-python, anthropic-python) demonstrate that the typed-response + status-keyed-exception pattern works at scale. They deliberately omit middleware — that's the gap `httpware` fills for the framework use case (vs. the per-API SDK use case Stainless targets). -- **niquests** validates that the async HTTP transport landscape is mid-disruption. `httpware`'s abstraction means consumers don't pay a tax for being early or late on that disruption. -- **The `pydantic/httpx2` fork** (2026-05-11) resolves the encode/httpx maintenance gap with backed stewardship. `httpware` doesn't compete with httpx2 — it sits above it — so it's net-additive to whichever transport the consumer ends up on. - -### Validation Approach - -- **Migrate one production service first.** Hard-evidence test: a real consumer ships on `httpware` for ≥30 days. If migration takes >2 days for a team that already used `base-client`, the migration guide failed and the pattern needs work. -- **Property-based tests for the retry budget.** The hardest novel part of the design; concurrency invariants must hold under stress. ≥10,000 Hypothesis trials covering admission control under concurrent failure scenarios. -- **Public benchmark suite vs raw httpx2 + tenacity**, published with each release. Closes the "frameworks are slow" objection pre-emptively. Target: ≤15% per-request overhead on 5KB JSON payloads at 100 RPS. -- **External adoption signal.** ≥3 non-`modern-python` PyPI dependents within 6 months. This is the "did the combination resonate?" question being answered by the market, not by us. -- **Reference circuit-breaker middleware ships in growth phase.** Proves the extension slot is a real, usable contract — not aspirational. - -### Risk Mitigation (innovation-specific) - -Three risks tied directly to the innovative elements above; broader project-level risks (market, adoption, resourcing) are covered in *Project Scoping & Phased Development → Risk Mitigation Strategy*. - -- **If retry budget defaults are wrong**, they're constructor-overridable from day one. No correctness risk — just a tuning question. -- **If transport abstraction cannot cleanly cover niquests semantic differences** (timeout interpretation, streaming cancellation), v1.0 stays httpx2-only and the gap is resolved via type-narrowing or adapter shims when niquests is added in the Growth phase. Consumer migration cost stays the same. -- **If property-based tests find concurrency bugs in the retry budget**, fix them before v1.0 — shipping broken resilience primitives is the exact failure mode that drove this project (the `circuit-breaker-box` precedent). - -## Developer Tool Specific Requirements - -### Project-Type Overview - -`httpware` is a pip-installable Python library distributed via PyPI. It provides a public API surface (classes, protocols, decorators) consumed by application code at import time. It is not a CLI, has no GUI, requires no daemon, and does not generate code. Its install footprint is small (one wheel, pure-Python, ~1500-2000 LOC). - -### Language Matrix - -**Python versions supported (v1.0):** - -| Version | Status | Notes | -|---|---|---| -| 3.11 | Supported | Floor — required for `asyncio.TaskGroup` and exception groups | -| 3.12 | Supported | | -| 3.13 | Supported | Free-threaded build tested as best-effort, not promised | -| 3.14 | Supported when GA | Added to CI on release | - -**Python versions explicitly excluded:** - -- **3.10 and earlier.** Required for `TaskGroup` and `except*` (PEP 654). 3.10 EOL is October 2026; cost of supporting it is too high. -- **PyPy.** Tested as best-effort; not promised. Pydantic v2 has degraded performance on PyPy. -- **GraalPy / IronPython / other implementations.** Not in scope. - -**OS / platform support:** Pure Python wheels, no compiled extensions. Linux, macOS, Windows all supported equally. ARM and x86 supported equally. - -**Async-only.** No sync facade in v1.0. The library requires `asyncio` (or `anyio` via httpx2's internals). `trio` compatibility is best-effort because httpx2 supports it; not directly tested. - -### Installation Methods - -**Primary install:** -``` -pip install httpware -``` - -**Compatible package managers:** anything that reads `pyproject.toml`: -- `uv add httpware` / `uv pip install httpware` -- `poetry add httpware` -- `pdm add httpware` -- `pixi add --pypi httpware` -- `rye add httpware` - -**Build backend:** `uv-build` (matching `base-client`'s recent migration). PEP 517/518 compliant. - -**Distribution:** PyPI (primary), GitHub Releases (binary attestations). Releases are git-tagged and signed; provenance via PyPI Trusted Publishers / Sigstore attestation. SBOM published with each release. - -**Extras (optional install groups):** - -| Extra | Pulls in | Purpose | -|---|---|---| -| `httpware[msgspec]` | `msgspec` | Faster validator path (alternate `ResponseDecoder`) | -| `httpware[otel]` | `opentelemetry-api`, `opentelemetry-sdk` | OpenTelemetry instrumentation middleware | -| `httpware[niquests]` | `niquests` (post-v1.0) | `NiquestsTransport` backend | -| `httpware[all]` | everything above | Convenience | - -Base install (no extras) ships with httpx2, pydantic, and standard library only. - -### IDE Integration - -- **`py.typed` marker** ships with the package — type checkers treat `httpware` as fully typed inline. -- **Full type coverage:** every public symbol carries explicit annotations. `ty` (Astral) passes on `httpware/` and on a reference consumer project. -- **Generic-aware:** `response_model: type[T] | None = None` keyword refines the return type to `T` when supplied, `Response` when omitted. Verified via `ty`-checked examples in the test suite. -- **Hover docs:** every public class and method carries a one-line docstring with usage example. Auto-extracted to API reference via mkdocstrings. -- **Type checker:** `ty` (Astral) exercised in CI; matches `modern-python/modern-di` and the org's house convention. -- **No IDE plugin required.** PyCharm, VS Code (Pylance), Helix, Neovim — all benefit equally from the inline types. - -### API Surface - -Top-level public exports from `httpware`: - -| Symbol | Kind | Purpose | -|---|---|---| -| `AsyncClient` | class | The main client | -| `Request`, `Response` | dataclass | First-class request/response types | -| `Transport` | Protocol | Pluggable transport interface | -| `Httpx2Transport` | class | Default httpx2-backed transport | -| `RecordedTransport` | class | Test-only transport for mocking | -| `Middleware`, `Next` | Protocol | Onion middleware contract | -| `before_request`, `after_response`, `on_error` | decorator | Phase-shortcut helpers | -| `Retry`, `RetryBudget`, `Bulkhead`, `Timeout`, `Observability` | class | Built-in middleware | -| `ResponseDecoder` | Protocol | Pluggable validation interface | -| `PydanticDecoder`, `MsgspecDecoder` | class | Default decoders | -| `Limits`, `Timeout` (config types) | dataclass | Connection / timeout config | -| `ClientError`, `TransportError`, `TimeoutError`, `StatusError`, `BadRequestError`, `UnauthorizedError`, `ForbiddenError`, `NotFoundError`, `ConflictError`, `UnprocessableEntityError`, `RateLimitedError`, `InternalServerError`, `ServiceUnavailableError` | exception | Status-keyed hierarchy | - -Approximate count: ~25 public symbols. Stability tier: all exports are public-stable from v1.0; private helpers live in `httpware._internal` (underscore-prefixed, not re-exported). - -### Code Examples - -**Quickstart (10 LOC):** - -```python -from httpware import AsyncClient -from pydantic import BaseModel - -class User(BaseModel): - id: int - name: str - -async def main(): - async with AsyncClient(base_url="https://api.example.com") as client: - user = await client.get("/users/1", response_model=User) - print(user.name) -``` - -**Service-client subclass pattern:** - -```python -from dataclasses import dataclass -from httpware import AsyncClient - -@dataclass -class BillingClient: - client: AsyncClient - - @classmethod - def from_url(cls, base_url: str, *, token: str) -> "BillingClient": - return cls(client=AsyncClient(base_url=base_url, auth=token)) - - async def get_invoice(self, invoice_id: str) -> Invoice: - return await self.client.get(f"/invoices/{invoice_id}", response_model=Invoice) -``` - -**Custom middleware (signing):** - -```python -from httpware import Middleware, Next, Request, Response -import hmac, hashlib - -class SignRequestMiddleware: - def __init__(self, secret: bytes) -> None: - self._secret = secret - - async def __call__(self, req: Request, next: Next) -> Response: - sig = hmac.new(self._secret, req.body or b"", hashlib.sha256).hexdigest() - return await next(req.with_header("X-Signature", sig)) - -client = AsyncClient(base_url="...", middleware=[SignRequestMiddleware(SECRET)]) -``` - -**Test fixture (RecordedTransport):** - -```python -import pytest -from httpware import AsyncClient, RecordedTransport, Response, NotFoundError - -@pytest.fixture -def transport() -> RecordedTransport: - return RecordedTransport({ - ("GET", "/users/1"): Response(status=200, json={"id": 1, "name": "ada"}), - ("GET", "/users/2"): Response(status=404, json={"detail": "not found"}), - }) - -async def test_user_ok(transport): - client = AsyncClient(base_url="https://x", transport=transport) - user = await client.get("/users/1", response_model=User) - assert user.name == "ada" - assert transport.calls[0].url.path == "/users/1" - -async def test_user_missing(transport): - client = AsyncClient(base_url="https://x", transport=transport) - with pytest.raises(NotFoundError) as exc: - await client.get("/users/2", response_model=User) - assert exc.value.status == 404 -``` - -**Streaming:** - -```python -async with client.stream("GET", "/events") as resp: - async for line in resp.iter_lines(): - process(line) -``` - -### Migration Guide (deliverable outline) - -The migration guide is a v1.0 release blocker. Structure: - -1. **Why migrate** — 200 words. Link to brief, focus on: the `encode/httpx` → `pydantic/httpx2` stewardship transition (which `base-client` consumers need to handle anyway), and on resilience correctness. -2. **What's changing** — at-a-glance table: - - `httpx.AsyncClient` → `httpware.AsyncClient` (constructor kwargs largely compatible; httpware uses httpx2 under the hood) - - `httpx.Request` / `httpx.Response` → `httpware.Request` / `httpware.Response` - - `httpx.HTTPStatusError` → status-keyed exception (`NotFoundError` etc.) - - `httpx.codes.is_*` → status-keyed exception branching - - `httpx._client.USE_CLIENT_DEFAULT` → `httpware.Unset()` sentinel - - `httpx.Timeout(1)` → `httpware.Timeout(connect=5, read=30, ...)` - - `respx.mock` decorator → `RecordedTransport` fixture - - `circuit_breaker_box.Retrier` → built-in `Retry` middleware - - `tenacity.retry(...)` decorators → constructor `retries=N` or `Retry` middleware -3. **Step-by-step migration** — six concrete steps, each with a "before" and "after" code block: - 1. Replace `httpx.AsyncClient` construction - 2. Update method signatures returning `Response` - 3. Replace error handling (`except httpx.*` → `except StatusError` subclasses) - 4. Replace `respx` test fixtures with `RecordedTransport` - 5. Remove `circuit_breaker_box.Retrier` and `tenacity` decorators - 6. Verify OpenTelemetry/observability hookup -4. **Side-by-side reference appendix** — full before/after for one example service. -5. **Gotchas** — known semantic differences (e.g., exception fields are plain types, not `httpx.Response`; the underlying transport is now `httpx2`, not `httpx` — but this is invisible to consumer code through the `httpware.*` types). - -### Implementation Considerations - -- **Single-file core or split modules?** Split: `client.py`, `request.py`, `response.py`, `errors.py`, `middleware/__init__.py` (+ one file per built-in middleware), `transports/httpx2.py`, `transports/recorded.py`, `decoders/pydantic.py`, `decoders/msgspec.py`, `_internal/*`. -- **Concurrency model.** Single shared `AsyncClient` per event loop. Pool lifecycle bound to the client's async context manager. `from_url(...)` classmethod helper for one-line construction. -- **State immutability.** `Request`/`Response` are frozen dataclasses. `req.with_header(...)` returns a new instance. Prevents middleware action-at-a-distance bugs. -- **Cancellation.** `asyncio.CancelledError` propagates unchanged through middleware (it is *excluded* from retry/breaker failure classification). -- **Logging.** Library emits no `print` and no top-level logger configuration. Observability middleware emits structured logs via a configurable logger and OpenTelemetry spans/metrics — but only if the observability extra is installed. No log emission in the hot path of unconfigured installs. -- **Sensible failure mode for missing extras.** If a user passes `decoder=MsgspecDecoder()` without `httpware[msgspec]` installed, raise `ImportError` with a pointer to the install command. No silent fallbacks. - -## Project Scoping & Phased Development - -### MVP Strategy & Philosophy - -**MVP approach: Problem-solving MVP.** The minimum useful version of `httpware` is whatever lets one `modern-python` team migrate one production service off `base-client` and ship it. Everything else — niquests, circuit breakers, msgspec adapter, OpenAPI codegen — is post-MVP value, not MVP value. We don't ship a "platform" or a "revenue MVP"; we ship a working framework for a single concrete migration target, then expand once it survives contact with production. - -**Delivery mode: Phased.** Three phases: **MVP / Growth / Vision** — feature contents defined in the *Product Scope* section above. Phase boundaries are not changing in this step. - -**Why phased over single-release:** The dependency between phases is real and ordered: -- Growth features (niquests transport, reference circuit breaker, msgspec, FastAPI integration recipes) all require a stable MVP abstraction as their foundation. Shipping them simultaneously would force premature surface decisions. -- Vision features (LLM preset, OpenAPI codegen, in-house circuit breaker) require Growth-phase external adoption signals to justify the implementation cost. -- Phasing also lets us ship v1.0 quickly with a small public surface, then expand the surface in 2.x releases with clear backward-compatibility commitments. - -### Resource Requirements - -- **Maintainer count (current):** 1-2 active, plus drive-by contributors. Explicit named-maintainer accounting and bus-factor target deferred per maintainer preference; to be revisited before v1.0 cut. -- **Build effort (MVP):** Estimated 2-3 calendar months of part-time engineering for a primary author with code-review support. Planning estimate: 1500-2000 LOC core + tests (acknowledged as possibly conservative; realistic ceiling 4000-6000 LOC including streaming edge cases, exception mapping, and property-based test infrastructure). -- **Dependencies:** httpx2 (>=2.0.0, <3.0; httpx2 v2.0.0 GA published to PyPI on 2026-05-12), pydantic v2, opentelemetry-api (optional), msgspec (optional), purgatory (referenced from companion CB package only, not MVP). -- **Infrastructure spend:** Zero beyond free-tier GitHub Actions CI, PyPI publishing, Read the Docs (or GitHub Pages) docs hosting. -- **Skills required:** Mid-senior Python async; familiarity with httpx/httpx2 internals; light resilience-engineering exposure (Finagle/Polly literacy helpful for retry budget). No specialized domain expertise needed. - -### Phase Boundaries (recap) - -See *Product Scope* section above for full feature lists. Summary: - -| Phase | Goal | Feature highlights | Exit criteria | -|---|---|---|---| -| **MVP (v1.0)** | One service migrated, framework stable | Transport protocol, Httpx2Transport, single-call typed-response API, onion middleware, Retry + RetryBudget + Bulkhead + Timeout + Observability, status-keyed exceptions, RecordedTransport, streaming, migration guide | ≥1 production consumer in `modern-python` stable for 30 days | -| **Growth (v1.x → v2.x)** | Multi-transport proven, ecosystem seeded | NiquestsTransport, reference CB middleware (wraps purgatory), msgspec decoder, OTel auto-instrumentation, `client.gather`, FastAPI/Litestar recipes, public benchmark suite | All known `base-client` consumers migrated; ≥3 external PyPI dependents | -| **Vision (v3+)** | Category leader, codegen ecosystem | LLM-gateway preset, sync API hierarchy (if demand), OpenAPI codegen target, in-house CB (only if purgatory-wrap is inadequate), middleware registry, distributed resilience state | Cited externally as the recommended Python resilience-first HTTP framework | - -### Risk Mitigation Strategy - -Project-level risks beyond what the Innovation section covered: - -**Technical risks** - -- **Retry budget concurrency correctness.** Highest-novelty, highest-stakes piece. Mitigation: property-based test suite (Hypothesis) ≥10,000 trials covering concurrent failure scenarios; explicit invariants documented; retry budget is constructor-disabled (`retry_budget=None`) so users can opt out if a bug surfaces. -- **Transport semantic differences across backends.** Deferred risk — v1.0 is httpx2-only. NiquestsTransport in Growth phase validates the abstraction; if niquests's timeout/streaming semantics don't fit cleanly, we resolve via adapter shims without consumer-side breakage. -- **httpx2 private-API drift.** The current `base-client` imports from `httpx._client` and `httpx._types` (encode/httpx). `httpware`'s `Httpx2Transport` must avoid private-API usage entirely — verified by a CI check (`grep 'httpx2\._'` returns 0 inside `httpware/`). If httpx2 GA changes public APIs from the beta, we adapt one file (`transports/httpx2.py`); no consumer-visible change. - -**Market risks** - -- **External adoption may not materialize.** Mitigation: framed as upside, not a v1.0 gate. Primary success criterion is internal `modern-python` migration, which is within our control. External adoption signals (≥3 PyPI dependents) are Growth-phase, not MVP-phase. -- **The pydantic/httpx2 fork resolved the governance concern (2026-05-11) before v1.0 cut.** Reality check: this is the world we're in. Mitigation: the framework's value is multi-axis. Even with the strategic-risk argument now historical, the resilience composition, typed responses, RecordedTransport, and status-keyed exception hierarchy are standalone wins. The transport-agnostic layer becomes about good design hygiene rather than strategic insurance, but remains durable value (NiquestsTransport, future backends). -- **A vendor (Stainless, Pydantic AI, etc.) releases a competing framework.** Mitigation: low likelihood — Stainless's strategy is per-API SDKs not frameworks; Pydantic AI is LLM-specific. If a credible competitor emerges, we adapt positioning to lean into resilience and middleware composability (where competitors are weakest). - -**Resource risks** - -- **Bus-factor and maintainer attention.** Explicitly acknowledged. Mitigation strategy: (a) keep MVP core small (1500-4000 LOC) so the maintainable surface is finite; (b) keep extensions plug-in (Growth-phase features like circuit breaker, niquests, msgspec, codegen all live behind protocols and optional extras), so partial neglect doesn't break consumers; (c) sustainability section deferred per maintainer call but recognized as a real v1.0-cut decision point. -- **Maintainer attention shift mid-build.** If active development pauses post-MVP, consumers stay on a working httpx2-backed library indefinitely. No external dependencies force release cadence. Degradation is graceful — no daily releases needed for the library to keep working. -- **Funding.** None required. OSS project, no infrastructure costs beyond free-tier CI/docs hosting. Not a risk vector. - -## Functional Requirements - -### Client Construction & Lifecycle - -- **FR1:** A consumer can construct an `AsyncClient` with optional `base_url`, default headers, default query parameters, timeout, limits, auth, transport, decoder, and middleware list. -- **FR2:** A consumer can construct an `AsyncClient` via `AsyncClient.from_url(base_url, ...)` for one-line default configuration. -- **FR3:** A consumer can use the client as an async context manager (`async with`), binding its lifecycle to a code block and closing the transport on exit. -- **FR4:** A consumer can derive a new client with overridden defaults via `client.with_options(**overrides)` that shares the underlying transport and connection pool. -- **FR5:** A consumer can pass authentication as a static string, a synchronous callable returning a string, an async callable returning a string, or a custom `Middleware` instance. -- **FR6:** A consumer can configure connection limits (`max_connections`, `max_keepalive_connections`, `keepalive_expiry`) and timeouts (split `connect`/`read`/`write`/`pool` or a single value) at client construction. - -### Request & Response - -- **FR7:** A consumer can issue GET, POST, PUT, PATCH, DELETE, HEAD, and OPTIONS requests via dedicated methods, plus arbitrary methods via `client.request(method, url, ...)`. -- **FR8:** A consumer can override per-request headers, query parameters, cookies, timeout, and provide body via `json=`, `data=` (form), `files=` (multipart), or `content=` (raw). -- **FR9:** A consumer receives an `httpware.Response` exposing `status: int`, `headers: Mapping`, `content: bytes`, `text: str`, `json()`, `url`, and `elapsed`, with no references to the underlying transport's types. -- **FR10:** A consumer can request a typed response by passing `response_model=T` to any request method, receiving a value of type `T` directly. -- **FR11:** A consumer can issue a streaming request via `async with client.stream(method, url, ...) as resp`, consuming the body through `iter_bytes(chunk_size)`, `iter_text(chunk_size)`, or `iter_lines()`. - -### Transport Layer - -- **FR12:** The framework defines a `Transport` Protocol that any HTTP-client backend must satisfy to be usable. -- **FR13:** A consumer can supply a custom `Transport` implementation at client construction. -- **FR14:** The framework ships a default `Httpx2Transport` adapting `httpx2.AsyncClient` to the `Transport` Protocol. -- **FR15:** The framework guarantees that swapping the `Transport` implementation requires no changes to consumer code that does not directly reference transport-specific types (i.e., conforming consumer code remains valid). -- **FR16:** The framework's public exports do not include the underlying HTTP client's types; `httpx2.*` is not re-exported. - -### Middleware System - -- **FR17:** A consumer can supply an ordered list of `Middleware` instances at client construction; each is invoked for every request in declared order (outer to inner). -- **FR18:** A consumer can implement a `Middleware` by providing an async callable matching `(req: Request, next: Next) -> Response`. -- **FR19:** A consumer can author middleware using `@before_request`, `@after_response`, and `@on_error` decorators on simple async functions. -- **FR20:** The framework documents a stable middleware execution order (`Observability → RetryBudget → Retry → [extension slot] → Bulkhead → Timeout → Transport`) and a named extension slot for plug-in middleware. -- **FR21:** A consumer can short-circuit the middleware chain by not calling `next` and returning a synthesized `Response` directly. -- **FR22:** `Request` objects are immutable; a consumer mutates a request via `req.with_header(...)`, `req.with_url(...)`, etc., each returning a new instance. - -### Resilience - -- **FR23:** The framework retries failed requests according to a configurable policy specifying max attempts, backoff curve, retryable status codes, and retryable exception types. -- **FR24:** The framework retries only idempotent methods (GET, HEAD, PUT, DELETE) by default; POST and PATCH require explicit opt-in. -- **FR25:** The framework applies full-jitter exponential backoff between retry attempts and honors a `Retry-After` response header when present. -- **FR26:** The framework enforces a retry budget (token-bucket admission control) capping total retries-per-second across the client; rejected retry attempts surface the original error without further retry. -- **FR27:** A consumer can configure or disable the retry budget at client construction. -- **FR28:** The framework enforces a per-host bulkhead (concurrency cap) when configured; requests exceeding the cap queue or fail-fast per configuration. -- **FR29:** The framework enforces a per-attempt timeout; timed-out attempts raise `TimeoutError` and are eligible for retry. -- **FR30:** A consumer can plug a circuit-breaker middleware (or any other resilience primitive) into the documented extension slot without library changes; a built-in circuit breaker is not provided in v1.0. - -### Validation & Typed Responses - -- **FR31:** The framework defines a `ResponseDecoder` Protocol that adapts raw response bytes to a typed model. -- **FR32:** The framework ships a default pydantic-based `ResponseDecoder` that caches `TypeAdapter` instances per model type and validates JSON in a single pass. -- **FR33:** The framework ships an alternate msgspec-based `ResponseDecoder` available via the `httpware[msgspec]` install extra. -- **FR34:** A consumer can supply a custom `ResponseDecoder` at client construction. -- **FR35:** A consumer can decode responses into pydantic models, dataclasses, TypedDict, `list[T]`, `dict[K, V]`, primitives, and any other type the chosen decoder supports. - -### Error Handling - -- **FR36:** The framework raises `httpware`-owned exceptions only; consumer code does not need to import the underlying transport's exception types. -- **FR37:** The framework provides a status-keyed exception hierarchy: `BadRequestError`, `UnauthorizedError`, `ForbiddenError`, `NotFoundError`, `ConflictError`, `UnprocessableEntityError`, `RateLimitedError`, `InternalServerError`, `ServiceUnavailableError`, plus base classes `ClientStatusError` (4xx), `ServerStatusError` (5xx), and `StatusError` (any non-2xx). -- **FR38:** The framework provides `TransportError` for connection/network failures and `TimeoutError` for client-side timeouts, both distinct from status errors. -- **FR39:** Every exception exposes plain-typed fields: `status: int`, `body: bytes`, `headers: Mapping[str, str]`, `json: Any | None`, `request_method: str`, `request_url: str`. No transport-typed objects are attached. -- **FR40:** The framework excludes `asyncio.CancelledError` from automatic retry and from resilience-middleware failure accounting; cancellation propagates unchanged through the middleware chain. - -### Testing Support - -- **FR41:** The framework ships a `RecordedTransport` test double accepting a mapping of `(method, url_pattern) → Response | Exception` and exposing received requests as `.calls`. -- **FR42:** A consumer can construct a client with `transport=RecordedTransport({...})` to drive tests without network access or external mocking libraries. -- **FR43:** `RecordedTransport` supports both response side-effects (returning a stub `Response`) and exception side-effects (raising a stub exception); recorded calls are inspectable for method, URL, headers, and body. - -### Observability - -- **FR44:** The framework emits lifecycle hooks for: request start, request complete, retry attempted, retry budget exhausted, per-attempt timeout, and exception raised — each accepting a user-supplied callable. -- **FR45:** The framework ships an OpenTelemetry instrumentation middleware (available via `httpware[otel]`) that produces spans and metrics conforming to OpenTelemetry HTTP-client semantic conventions. -- **FR46:** A consumer can inspect the runtime state of the retry budget (remaining tokens, in-use ratio) via a public API on the `RetryBudget` middleware for `/healthz`-style integration. -- **FR47:** The framework does not configure global logging or emit logs in its hot path unless observability middleware is explicitly installed. - -## Non-Functional Requirements - -### Performance - -- **NFR1:** Per-request framework overhead — measured as the wall-clock delta of `client.get(url, response_model=User)` vs raw `httpx2.AsyncClient` + manual `pydantic.TypeAdapter(User).validate_json(...)` — is ≤15% on typical 5KB JSON payloads at 100 RPS sustained. Measured by the published benchmark suite on every release. -- **NFR2:** `TypeAdapter` instances are cached per `response_model`; the default pydantic decoder constructs zero `TypeAdapter` instances per request after warm-up. -- **NFR3:** The default `ResponseDecoder` uses `validate_json(response.content)` (single parse pass), not `validate_python(json.loads(content))` (two parse passes). -- **NFR4:** No synchronous I/O, no blocking calls (e.g., `requests`, `time.sleep`), and no GIL-heavy work on the framework hot path beyond what the chosen transport and decoder require. -- **NFR5:** Cold-start (first `import httpware` + first request) completes in ≤200ms on Python 3.11 on a developer-class machine (single-core baseline >2GHz). - -### Security - -- **NFR6:** TLS certificate verification is enabled by default. Disabling requires explicit `verify=False` per-client or per-request. -- **NFR7:** A configurable secret-redaction hook is invoked on every header and body fragment emitted to logs, OpenTelemetry spans, or `repr()` output. Default redacted-header allowlist includes `Authorization`, `Cookie`, `Set-Cookie`, `X-Api-Key`, `X-Auth-Token`, `Proxy-Authorization`. -- **NFR8:** No request or response body is emitted to logs or spans by default. Body capture is opt-in per middleware configuration. -- **NFR9:** Releases are published via PyPI Trusted Publishers with Sigstore attestation. A SBOM (CycloneDX or SPDX) is attached to each GitHub Release. -- **NFR10:** A `SECURITY.md` at the repo root documents the vulnerability disclosure channel and commits to a 90-day private-disclosure window before public disclosure. - -### Concurrency & Throughput - -- **NFR11:** A single `AsyncClient` instance supports concurrent requests up to its configured `max_connections` limit without framework-introduced lock contention beyond what the underlying transport requires. -- **NFR12:** `RetryBudget` token accounting is concurrency-safe: a Hypothesis property-based test suite of ≥10,000 concurrent-access trials passes without token-count drift, invariant violations, or race conditions. -- **NFR13:** Middleware execution is per-request and stateless by default. Any shared state across requests is the consumer's responsibility and must be explicit. -- **NFR14:** An `AsyncClient` instance is bound to its creating event loop; cross-loop sharing of a single client instance is not supported and is documented as undefined behavior. - -### Reliability & Correctness - -- **NFR15:** `asyncio.CancelledError` is never swallowed, transformed, or counted as a failure by any built-in middleware. It propagates through the entire middleware chain unchanged. -- **NFR16:** Streaming-response context managers guarantee the underlying connection is returned to the pool on consumer-raised exceptions (including `CancelledError`) and on normal exit. -- **NFR17:** All public types pass `ty` (Astral) type checking on Python 3.11+. A `py.typed` marker ships with the package. -- **NFR18:** No breaking changes to any public symbol within the v1.x release line. Deprecations carry a one-minor-version warning period (emitted via `DeprecationWarning`) before removal in v2.0. - -### Integration - -- **NFR19:** OpenTelemetry instrumentation conforms to the current OpenTelemetry HTTP-client semantic conventions (`http.request.method`, `url.full`, `http.response.status_code`, `http.client.request.duration`, etc.). Conformance is validated by a CI check that imports the OTel semconv schema and asserts emitted-attribute coverage. -- **NFR20:** Compatible with pydantic v2 (`>=2.0, <3.0`) and msgspec (`>=0.18`); a migration plan to pydantic v3 is documented when v3 ships, but compatibility is not pre-promised. -- **NFR21:** The library imports cleanly and operates correctly alongside FastAPI, Starlette, and Litestar (validated by a smoke-test CI job using each framework). Integration recipes ship as Growth-phase deliverables. -- **NFR22:** Project metadata follows PEP 621 (`pyproject.toml`); install and build succeed under `pip`, `uv`, `poetry`, and `pdm` using the `uv-build` PEP 517 backend. - -### Maintainability & Quality - -- **NFR23:** Line coverage on `httpware/` core modules (transports and decoders excluded, since both are largely adapter code) is ≥90%, enforced in CI. -- **NFR24:** Property-based tests (Hypothesis) cover concurrency-sensitive primitives: `RetryBudget`, `Bulkhead`, retry interleaving with timeouts, request immutability under middleware mutation. ≥10,000 trials per CI run. -- **NFR25:** CI runs on every push and pull request, exercising: `ruff` lint, `ty` type check on `httpware/` and on a reference consumer project, `pytest` with coverage, the property-based suite, and a smoke test against a real httpbin/httpbingo endpoint. diff --git a/docs/archive/product-brief-httpware-distillate.md b/docs/archive/product-brief-httpware-distillate.md deleted file mode 100644 index 80771f2..0000000 --- a/docs/archive/product-brief-httpware-distillate.md +++ /dev/null @@ -1,455 +0,0 @@ ---- -title: "Product Brief Distillate: httpware" -type: llm-distillate -source: "product-brief-httpware.md" -created: "2026-05-11" -updated: "2026-05-12" -purpose: "Token-efficient context for downstream PRD creation" ---- - -# httpware — Detail Pack - -Dense reference for downstream PRD / architecture work. Each bullet is self-contained. - -## Strategic context - -### httpx → httpx2 transition (as of 2026-05-12) - -**Resolved on 2026-05-11: Pydantic Services forked `encode/httpx` to `pydantic/httpx2`, restored issues/discussions, released v2.0.0b1 the same day.** - -Historical context (the conditions that led to the fork): - -- `encode/httpx` 0.28.1 shipped 2024-12-06 — 17 months without a release. -- `encode/httpx` issue tracker was disabled (`has_issues: false` via GitHub API). -- Last commit to `encode/httpx` `master`: 2026-02-23. Recent commits cosmetic (typos, docs, dependabot, third-party-list updates). One substantive change in 12 months: "Drop Python 3.8 support" (2025-06). -- **Discussion #3784 (2026-02-27)**, lead maintainer `lovelydinosaur`: *"I've closed off access to issues and discussions. I don't want to continue allowing an online environment with such an absurdly skewed gender representation. I find it intensely unwelcoming, and it's not reflective of the type of working environments I value."* — verbatim. -- Maintained fork `httpxyz` appeared 2026-03-25 from one frustrated user; never gained traction. -- **OpenAI and Anthropic SDKs pinned `httpx<1.0`** during the stalled period and have not migrated. - -Current state of `pydantic/httpx2` (verified 2026-05-12): - -- Repo created **2026-05-11**, owned by `pydantic` org. -- License: BSD-3-Clause (inherited from httpx). -- Issues **enabled**; 165 open at fork time. -- Latest release: **v2.0.0 GA, 2026-05-12** (initial v2.0.0b1 was published the same day as the fork, 2026-05-11; GA followed the next day). -- README: *"Pydantic Services is picking up stewardship under the HTTPX2 name so that users have a reliably maintained path forward — including timely security updates for a library that sits in the critical path of so many production systems."* -- Lead maintainer post-fork: **Marcelo Trylesinski (Kludex)** — FastAPI core team member. -- Original contributors (lovelydinosaur, florimondmanca, karpetrosyan, sethmlarson, cdeler, etc.) carried over via fork history. -- Same API as httpx 0.28 — drop-in for most consumers; httpx2 is httpx with stewardship, not a redesign. -- **Implication for httpware:** the "strategic risk" framing in the brief reflected the pre-fork situation. Post-fork, the project is driven by architectural debt in `base-client` and the gap in Python's resilience ecosystem, not by httpx maintenance concerns. - -### niquests evaluation - -- Repo: `jawah/niquests`. ~2,313 stars. Last push 2026-05-10. v3.18.8 released 2026-05-10. -- Maintainer: **Ahmed R. TAHRI ("Ousret")** dominant. Other recent contributors (Bartosz Magiera, Julien Brayere, Tatsh) are drive-by. **Bus factor ~1.** -- HTTP/1.1 + HTTP/2 + HTTP/3 + WebSocket + SSE by default. No extras required for HTTP/2. -- API is requests-compatible (sync-first); async via `AsyncSession` with `aget`/`apost`. -- Multiplexing is opt-in: `Session(multiplexed=True)`. -- Niquests' own benchmark: 1000 GETs to httpbingo.org — niquests 0.551s, aiohttp 1.351s, httpx 2.087s. Self-published, directional only. -- Mocking story: **`niquests-mock` is 3 stars**, single tiny project (`0x12th/niquests-mock`). Provides a `respx_mock` API alias but mature ecosystem does not exist. respx does NOT work with niquests. -- **Conclusion**: technically strong, fails the "wide community" bar. - -### Other candidates evaluated and rejected - -- **aiosonic**: solo (Johanderson Mogollon), no mocking ecosystem, sub-1k stars. Fail. -- **httpcore (direct)**: same encode-org cadence problem, too low-level, no first-class mocking. Fail. -- **urllib3 v2 + anyio**: not actually async at protocol level — would require threadpool. Fail constraint. -- **tornado.httpclient**: tornado-flavored, no respx equivalent, heavy dependency. Fail. -- **httpxyz**: 1 maintainer, brand new. Fail. - -### Trigger and motivation (from maintainer) - -- **General strategic concern**, not a specific CVE or hard deadline. -- The lovelydinosaur situation reinforces but did not trigger the work. -- Audience for the brief: maintainer + several teams using `base-client` in work projects (community-of-python is a real internal-multi-team library). - -## Current `base-client` surface (what's leaking and what hurts) - -- **Repo**: `community-of-python/base-client`. ~702 LOC total, 13 Python files. -- **Public surface that leaks httpx types** (file: `base_client/base_client.py`): - - `BaseClient.client: httpx.AsyncClient` — public dataclass field. Examples reach into it directly. - - `BaseClient.send(request: httpx.Request) -> httpx.Response` - - `BaseClient.prepare_request(...) -> httpx.Request` - - `BaseClient.validate_response(response: httpx.Response) -> None` -- **Private httpx imports** (CRITICAL coupling): - - `from httpx._client import USE_CLIENT_DEFAULT, UseClientDefault` (line 7) - - `from httpx._types import CookieTypes, HeaderTypes, QueryParamTypes, RequestContent, RequestData, RequestExtensions, RequestFiles, TimeoutTypes` (lines 8–17) -- **Error classes hold `httpx.Response`** (`base_client/errors.py`): `HttpStatusError`, `HttpClientError`, `HttpServerError` — all carry `response: httpx.Response` field. -- **`response_to_model`** (`base_client/response.py`): loose utility, takes `httpx.Response`, calls `response.json()`, builds `pydantic.TypeAdapter` **per call** (performance footgun documented by pydantic). -- **Tests assert against 19 specific httpx exception types** via `respx.mock(side_effect=...)`: RequestError, TransportError, ConnectTimeout, ReadTimeout, WriteTimeout, PoolTimeout, NetworkError, ConnectError, ReadError, WriteError, CloseError, ProtocolError, LocalProtocolError, RemoteProtocolError, ProxyError, UnsupportedProtocol, DecodingError, TooManyRedirects, HTTPError, plus a few more. -- **Examples use** `httpx.Timeout(1)` (1-second total timeout — too aggressive for any real API), `httpx.codes.is_server_error()`, `is_client_error()`, `is_success()`. -- **Dependencies**: `httpx` (no version pin), `pydantic`, `multidict`, `circuit-breaker-box`. Dev: `respx`, `pytest`, `pytest-asyncio`, `mypy`, `ruff`. -- **Tooling**: mypy strict, ruff (select=ALL, line-length=120), pytest with `asyncio_mode="auto"`. Python 3.10+ (CI tests 3.10–3.13). - -## `circuit-breaker-box` bug inventory (verified by source read) - -For the future-iteration circuit-breaker design — these are the failure modes to avoid: - -| # | Issue | Location | Severity | -|---|---|---|---| -| 1 | No Half-Open state — recovery is TTL-only; all in-flight requests stampede on TTL expiry | `circuit_breaker_base.py:6-18`, `circuit_breaker_in_memory.py:18-21` | Critical | -| 2 | First failure not counted in `Retrier` — only attempts ≥2 increment the counter | `retrier.py:50-51` (`if attempt.retry_state.attempt_number > 1`) | Critical | -| 3 | Redis `EXPIRE` called on every increment — refreshes TTL on each failure so breaker never auto-recovers under sustained load | `circuit_breaker_redis.py:42-43` | Critical | -| 4 | Non-atomic increment in async memory backend — read-then-write straddles `await` | `circuit_breaker_in_memory.py:23-29` | High | -| 5 | Off-by-one in availability check — `failures_count <= max_failure_count` allows one more failure than configured | `circuit_breaker_in_memory.py:33`, `circuit_breaker_redis.py:57` | Medium | -| 6 | No response-based failure detection — only exceptions count; 5xx fast-returns invisible | `retrier.py` (no result hook) | High | -| 7 | No slow-call detection — 30s successful response doesn't trip | architectural | Medium | -| 8 | Redis backend doesn't fail open — Redis outage propagates `RedisConnectionError` to callers | `circuit_breaker_redis.py:28-67` | High | -| 9 | TTL refresh in cachetools in-memory path — same root cause as #3 | `circuit_breaker_in_memory.py:25, 28` | High | -| 10 | No state-transition hooks / events / counters | `circuit_breaker_base.py` | Medium | - -- Library is 2-state (Closed/Open), not 3-state. Implements a per-host failure-counting TTL ban, not a faithful circuit breaker. -- Repo maintainer profile: NikitaKozlovtcev (14 commits) + lesnik512 (5). Last release ~9 months stale. - -## API design patterns to adopt (from cross-language survey) - -### Stainless skeleton (openai-python, anthropic-sdk-python) - -- **Private `_client`**, never exposed. Lazy default `httpx.AsyncClient`; user can inject via `http_client=` kwarg. -- **Resource pattern**: each endpoint group is a class holding a back-pointer to the base client. Endpoint methods call `self._post("/path", body=..., options=..., cast_to=Model)`. -- **`cast_to`/`response_model` kwarg** is the typed-response mechanism. Single call site, `TypeVar` carries the type through. -- **`with_options(**overrides) -> Self`** returns a new client sharing the pool but with new defaults. Steal verbatim. -- **`NotGiven` sentinel** (`not_given`) distinct from `None`, because JSON `null` vs "omit the field" must be representable. -- **`.with_raw_response.create(...)`** returns `APIResponse[T]` (lets users see headers/status; `.parse()` to get the model). -- **`.with_streaming_response.create(...)`** returns a context manager that never buffers the body. -- **`extra_headers=`, `extra_query=`, `extra_body=`** escape hatches on every endpoint method. -- **Two parallel class hierarchies** (`SyncAPIClient` / `AsyncAPIClient`). No `asyncify` magic; literally separate code paths. v1.0 of httpware ships async-only — defer sync hierarchy entirely. -- **Stainless deliberately rejects middleware** — retries are hand-rolled in the request loop. httpware breaks with this — middleware is the framework's extension axis. - -### Exception hierarchy (single tier, plain fields) - -``` -ClientError (base, never raised) -├─ TransportError (network/DNS/TLS) -├─ TimeoutError -└─ StatusError (any non-2xx) - ├─ ClientStatusError (4xx) ─ BadRequest, Unauthorized, Forbidden, NotFound, - │ Conflict, UnprocessableEntity, RateLimited - └─ ServerStatusError (5xx) ─ InternalServerError, ServiceUnavailable -``` - -- Each exception carries **plain fields**: `status: int`, `body: bytes`, `headers: Mapping[str, str]`, `json: Any | None`, `request_method: str`, `request_url: str`. -- **No `httpx2.Response` attached.** Tests assert on `error.status` and `error.json["code"]`, never on transport exception types. -- Mapping from transport exceptions happens at the seam (in `Httpx2Transport`). - -### Middleware (onion model + phase shortcuts) - -Canonical interface: - -```python -class Middleware(Protocol): - async def __call__(self, req: Request, next: Next) -> Response: ... -``` - -- Phase shortcut helpers wrap user functions into a `Middleware`: `@before_request`, `@after_response`, `@on_error`. -- Composition order (outer → inner): `Observability → RetryBudget → Retry → [extension slot] → Bulkhead → Timeout → Transport`. -- Rationale for ordering: - - Observability outermost: must see rejections too. - - RetryBudget outside Retry: budget gates whether retry happens at all. - - Retry outside [extension slot]: each retry attempt is a fresh check at the extension point (where a CB would sit). - - Bulkhead inside the extension slot: tripped CB rejects without touching the semaphore. - - Timeout innermost: per-attempt deadline. -- **Cross-language inspiration**: OkHttp `Interceptor.intercept(Chain)`, reqwest-middleware (Rust), ky hooks (TS — phases: `beforeRequest`, `beforeRetry`, `afterResponse`, `beforeError`). - -### Auth flow - -```python -client = Client(auth="static-key") # string -client = Client(auth=lambda: get_fresh_token()) # callable for short-lived tokens -client = Client(auth=MyOAuthMiddleware(...)) # full middleware -``` - -- Type the kwarg as `str | Callable[[], str] | Middleware`. Coerce internally. - -### Streaming - -```python -async with client.stream("GET", "/events") as resp: - async for line in resp.iter_lines(): - process(line) -``` - -- Mirror httpx2 public API exactly (muscle memory transfers; same as httpx), but `resp` type is `httpware.Response`, not `httpx2.Response`. -- Context manager guarantees pool return on exit, even if user code raises. - -## Anti-patterns (do not do these) - -1. Returning the underlying transport response (`hvac` returns `requests.Response` — every caller writes ad-hoc parsing). -2. Importing from private modules (current `base-client` does this; brittle to minor versions). -3. Exposing the HTTP client as a public field (`BaseClient.client`). -4. A single God exception with N subclasses tied to httpx exception types. -5. Two-step `prepare_request` → `send` as the public API. -6. Decorator-based endpoint definitions (Refit-in-Python attempts like `uplink`, `apiwrappers` — mypy can't follow them). -7. `response_to_model` as a loose utility outside the response object. -8. Sync facade via `nest_asyncio` / `asgiref.async_to_sync`. -9. Middleware that mutates a shared dict (Flask `before_request` idiom). Request objects must be immutable — each middleware returns a new one. -10. Auto-deserializing every response without opt-out. `response_model=` is explicit; default returns raw wrapped response. - -## Performance specifics - -### Pool / timeout / limits defaults - -- **`Limits` defaults**: `max_connections=100, max_keepalive_connections=20, keepalive_expiry=5.0` (matches httpx/httpx2 defaults; document that high-concurrency services need 1000/100 like OpenAI/Anthropic). -- **`Timeout` defaults**: `Timeout(connect=5.0, read=30.0, write=30.0, pool=5.0)` — split-value defaults. **Never `Timeout(1)`** (current base-client examples are dangerous). -- **OpenAI/Anthropic SDK reference**: `Limits(max_connections=1000, max_keepalive_connections=100)`, `Timeout(timeout=600, connect=5.0)` — 10-min read for LLM streams. -- Critical rule: if users wrap calls in `asyncio.Semaphore(N)`, then `N ≤ max_connections` or they get `PoolTimeout` instead of clean queueing. - -### Response validation - -- **Cache `TypeAdapter`** per `response_model` (module-level `functools.lru_cache`). Pydantic docs explicit: "Creating a TypeAdapter for a given type comes with some non-trivial overhead... it is recommended to create a TypeAdapter for a given type just once and reuse it." -- **Use `validate_json(response.content)`** instead of `validate_python(response.json())` — one parse pass instead of two. ~2× faster end-to-end. -- **Numbers**: - - `orjson` decode: ~3-6× faster than stdlib `json`. - - `msgspec.json.decode(buf, type=MyStruct)`: ~3-4× faster than orjson alone (parse+validate single pass). - - `msgspec` vs `pydantic v2` decode-and-validate: msgspec ~12× faster. - - Pydantic v1 → msgspec: ~85× faster. -- For typical 5KB payloads, parsing is sub-ms and dominated by network. For high-throughput (>1k req/s) or large responses (>100KB), JSON parsing becomes a real CPU bottleneck. - -### Connection lifecycle - -- **Single shared client per event loop**, created at startup, closed at shutdown. Per-request clients re-do TCP+TLS handshakes (1-3 RTT) and discard keepalive sockets. -- **Connection pools are bound to an event loop** — not shareable across loops. `@lru_cache`'d async clients break under `asyncio.run()` patterns. -- `BaseClient.from_url(base_url, timeout=..., limits=..., **kwargs)` factory helper builds sensible default `AsyncClient`. -- Add `__aenter__`/`__aexit__` that delegate to underlying client (ergonomics, no semantic change). - -### Async HTTP footguns - -- **Blocking DNS**: both httpx and niquests use `asyncio.getaddrinfo` (ThreadPoolExecutor wrapper around blocking `getaddrinfo`). Saturated default executor (32 threads) → DNS stalls. CPython issue #112169. Niquests has configurable resolver (DoH/DoT/DoQ). -- **SSL GIL contention**: CPython `SSLSocket.read` does GIL round-trip per 16KB TLS record (issue 37355). Bites above ~10k req/s. Mitigations: HTTP/2 (fewer handshakes), persistent connections, uvloop. -- **`asyncio.gather` exceeding pool size**: 500 requests at 100-connection pool → `PoolTimeout`. Expose `client.gather(requests, max_concurrency=N)` using `TaskGroup` on 3.11+. -- **`asyncio.CancelledError`** should NOT count as a failure (caller-initiated abort, not upstream problem). Excluded from breaker failure classification. -- **Don't touch `.text`/`.content`/`.json()` in framework hot path** — current `base-client` does this in DEBUG log path, breaks streaming and leaks body to logs. Bug to avoid. - -### HTTP/2 reality check - -- httpx: HTTP/2 opt-in extra (`pip install httpx[http2]` + `AsyncClient(http2=True)`). h2+httpcore overhead is real; httpx HTTP/2 sometimes slower than HTTP/1.1 for small payloads. -- niquests: HTTP/2 + HTTP/3 by default, multiplexing opt-in via `Session(multiplexed=True)`. -- Wins are 30-50% for many-small-requests-same-host workloads with ≥10ms RTT. Don't enable by default; expose as opt-in. -- Open question: no public benchmark found for "wrapped httpx with HTTP/2 vs HTTP/1.1 on small JSON workload at 50-200 concurrency." - -## Circuit-breaker reference design (deferred to post-v1.0) - -For when the in-house breaker eventually ships: - -- **States**: 3-state (Closed, Open, Half-Open) + admin `Disabled` flag. -- **Trip condition**: count-based sliding window (ring buffer of last N call outcomes), `failure_rate_threshold` over a `minimum_calls` floor (e.g., 0.5 over 20 calls in a window of 100). Optional `slow_call_duration_threshold` promotes slow successes to "failures." -- **Recovery**: half-open admits up to `half_open_max_calls` (default 5). Close when all N succeed; reopen on first failure. **`break_duration` with full jitter applied automatically** (`uniform(0.5, 1.5) * break_duration`) — cheapest correctness win, almost no Python lib defaults this. -- **Granularity**: per-host by default (matches current `base-client`), accept `key: Callable[[Request], str]` for per-route. -- **Failure classification**: 5xx + connection/timeout/network errors = failure. 4xx = success (configurable). `asyncio.CancelledError` = excluded. User-supplied `failure_predicate` overrides. -- **Atomicity**: `asyncio.Lock` per-key, lazily created. Read-modify-write of counts always inside the lock, never across `await`. -- **Backends**: in-memory default; pluggable `Storage` protocol; Redis storage **fails open** on backend errors (current circuit-breaker-box doesn't — fail-closed bug). -- **Hooks**: `on_state_change(key, from_state, to_state)`, `on_call_rejected(key)`, `on_call_recorded(key, outcome)`. -- **Estimated size**: ~400 LOC (likely 600-800 realistic with proper tests). -- **Composition with retry**: breaker check happens **per-attempt inside retry loop**, not around the whole loop. Resilience4j/Polly ordering. - -## Retry / retry-budget design (in v1.0) - -### Retry - -- Default: full-jitter exponential backoff, base 0.5s, max 8s, max attempts 3 (matches OpenAI/Anthropic SDK defaults — those use `delay * jitter, jitter ∈ [0.75, 1.25]`). -- **Only retry idempotent methods by default** (GET/HEAD/PUT/DELETE). POST requires explicit opt-in (idempotency-key pattern, see Stripe). -- Classifier: connection errors (always retry), 429 (retry with `Retry-After`/`retry-after-ms` honored), 5xx (retry except 501), 4xx (never retry). -- Use `tenacity.AsyncRetrying` rather than `Retrying` in async paths. -- Anti-pattern: `tenacity.wait_fixed(1)` — fixed waits cause synchronized thundering-herd retry storms. - -### Retry budget (Finagle pattern — the differentiator) - -- Token-bucket admission control. Default: `min_per_sec=10, ratio=0.2, ttl=10s` (Finagle defaults). -- Caps total retries-per-second across the whole client. A flapping endpoint can't trigger retries from every concurrent call. -- **Single biggest production-readiness gap in Python today.** No Python lib ships this. - -## Test mocking design (`RecordedTransport`) - -### Primary path - -```python -import pytest -from httpware import AsyncClient, RecordedTransport, Response - -@pytest.fixture -def fake_transport() -> RecordedTransport: - return RecordedTransport({ - ("GET", "/users/1"): Response(status=200, json={"id": 1, "name": "ada"}), - ("GET", "/users/2"): Response(status=404, json={"detail": "not found"}), - }) - -async def test_get_user_ok(fake_transport): - client = AsyncClient(base_url="https://x", transport=fake_transport) - user = await client.get("/users/1", response_model=User) - assert user.name == "ada" - assert fake_transport.calls[0].url.path == "/users/1" -``` - -### Layers of mocking support (in order of preference) - -1. **`RecordedTransport`** — primary. Zero httpx knowledge in tests. -2. **Middleware injection** — mount a "respond from fixture" middleware that short-circuits before reaching the network. -3. **respx pass-through** — default transport is httpx-backed, respx still works. Documented but not encouraged. - -### Tests in v1.0 must cover (replacing current `base-client` test patterns) - -- Mock 200 + JSON body -- Mock 4xx (assert specific exception, e.g., `NotFoundError`) -- Mock 5xx (retry behavior, RetryBudget exhaustion) -- Mock connection error (TransportError raised) -- Mock timeout (TimeoutError raised) -- Streaming response (chunk iteration, early close) -- Pool exhaustion (Bulkhead behavior) - -## Concrete API sketch - -### Construction - -```python -from httpware import AsyncClient - -client = AsyncClient( - base_url="https://api.example.com", - default_headers={"User-Agent": "myapp/1.0"}, - timeout=10.0, # or Timeout(connect=5, read=30, ...) - retries=3, - auth="bearer-token-here", # or callable, or Middleware - middleware=[TracingMiddleware()], - response_decoder=PydanticDecoder(), # or MsgspecDecoder, or custom -) -``` - -### Simple GET - -```python -resp = await client.get("/healthz") -assert resp.status == 200 -print(resp.text) -``` - -### Typed response (primary path) - -```python -from pydantic import BaseModel - -class User(BaseModel): - id: int - name: str - -user = await client.get("/users/1", response_model=User) -users = await client.get("/users", response_model=list[User]) -``` - -### Error handling - -```python -from httpware import RateLimitedError, ServerStatusError - -try: - user = await client.get("/users/999", response_model=User) -except RateLimitedError as e: - retry_after = float(e.headers.get("retry-after", "1")) -except ServerStatusError as e: - log.warning("upstream %s: %s", e.status, e.json) - raise -``` - -### Custom middleware (onion) - -```python -from httpware import Middleware, Next, Request, Response - -class SignRequestMiddleware: - def __init__(self, secret: str) -> None: - self._secret = secret - - async def __call__(self, req: Request, next: Next) -> Response: - req = req.with_header("X-Signature", hmac_sign(req.body, self._secret)) - return await next(req) -``` - -### Phase-shortcut middleware - -```python -from httpware import before_request, after_response - -@before_request -async def add_correlation_id(req: Request) -> Request: - return req.with_header("X-Correlation-ID", contextvars_get_id()) - -@after_response -async def log_slow_responses(req: Request, resp: Response) -> Response: - if resp.elapsed > 1.0: - log.warning("slow request %s %s: %.2fs", req.method, req.url, resp.elapsed) - return resp -``` - -### Sub-client (ky.extend-style) - -```python -users_api = client.with_options( - base_url=str(client.base_url) + "/users", - default_headers={"X-Service": "users"}, -) -``` - -### Backend swap (the design payoff) - -```python -from httpware.transports.niquests import NiquestsTransport -client = AsyncClient(base_url="...", transport=NiquestsTransport()) -``` - -### Streaming - -```python -async with client.stream("GET", "/events") as resp: - assert resp.status == 200 - async for line in resp.iter_lines(): - process(line) -``` - -## Scope signals (from user, throughout conversation) - -- **Decoupled flavor**: (a) single backend, swappable internally. NOT (b) pluggable backends à la SQLAlchemy dialects. -- **Initial backend**: `httpx2 >=2.0.0, <3.0` (Pydantic Services stewardship line; same API as httpx 0.28, drop-in). Updated 2026-05-13 after httpx2 v2.0.0 GA shipped (2026-05-12); original decision was "stay on httpx 0.28," now obsolete. No swap to niquests in v1.0. -- **Trigger**: general strategic concern. -- **Audience**: maintainer + several teams in `modern-python` / `community-of-python`. Library is public on PyPI. -- **base-client fate**: deprecated. New library supersedes it; consumers migrate when they want; no automated migration shim. -- **New library scope**: same use case as base-client (HTTP client framework for service clients). Broader scope is an open question, not promised. -- **Name**: `httpware`. Org: `github.com/modern-python`. -- **Strong architecture/redesign**: in scope. Public API redesign is wanted, not just internal cleanup. -- **Circuit breaker**: explicitly dropped from v1.0, but design must accommodate plug-in via middleware extension slot. -- **Validator pluggability**: explicitly requested. `ResponseDecoder` protocol. - -## Rejected approaches - -- **Stay on httpx, abstract it out, defer the swap.** Considered as "Framing 1 — Decouple first, decide later." Initially adopted, then upgraded to greenfield rebuild. -- **Migrate to niquests immediately.** Rejected: bus-factor-1, weak mocking ecosystem, fails community constraint. -- **Pluggable backends (multiple transports shipped simultaneously, user-selected).** Rejected: ~2-3× the work, harder to test, not needed for the use case. Kept as single-backend-swappable-internally. -- **Continue maintaining `base-client` and fix in place.** Rejected: too much breaking-change surface (httpx leakage everywhere, including private-API usage), redesign is wanted independently. -- **Fork httpx into a community-maintained variant.** Considered as a "strategic alternative" during the brief phase but not chosen. **Update 2026-05-12:** Pydantic Services actually did this — released `pydantic/httpx2` on 2026-05-11. We're now adopting it as the default backend rather than forking ourselves. -- **Contribute upstream to niquests to grow its maintainer pool.** Considered by review; not chosen. Brief doesn't address. -- **Depend on `purgatory` or `pybreaker` for the circuit breaker in v1.0.** Considered. Decision: drop CB entirely from v1.0, design middleware extension point so a future plug-in (likely `purgatory`-wrapping) works cleanly. -- **Add sustainability/governance section (named maintainers, hours, succession plan).** Deferred ("too early for this"). -- **Sync API in v1.0.** Excluded. Decision point not "permanent vs. post-v1.0" — leaning post-v1.0 if ever. -- **Decorator-based endpoint definitions (Refit-style).** Rejected: Python codegen is fragile, IDE autocomplete breaks. -- **`asyncify`/`asgiref.async_to_sync` style sync facade over async.** Rejected: threading bugs, breaks signal handling, kills uvloop. - -## Open questions / deferred decisions - -- **Sustainability story** — explicit named maintainers, hours/week, succession policy. Deferred per maintainer ("too early"). Worth revisiting before v1.0 cut. -- **Governance** — license (probably MIT or Apache-2.0), CONTRIBUTING.md, CLA/DCO, release-cadence commitment, CVE disclosure channel. Not specified in brief. -- **Migration cost quantification** — how many `base-client` consumers actually exist, how many lines of consumer code touch httpx types. Currently described qualitatively. Worth a `grep` across known consumers. -- **Scope: same as base-client vs. broader.** Maintainer flagged as open. Specifically: should the new library address SSE / WebSocket use cases? Auth-flow library territory (OAuth refresh, SigV4 signing)? -- **HTTP/2 default decision.** No public benchmark for "wrapped httpx HTTP/2 vs HTTP/1.1 on small JSON, 50-200 concurrency." Default off; document the toggle. Worth measuring on real workload before flipping. -- **`circuit-breaker-box` deprecation.** With base-client deprecated, what's the path for the breaker lib itself? Archive? Hand off? Fix the bugs? Not addressed in brief. -- **OpenAPI codegen story.** Explicitly out of v1.0. Open question whether `httpware` ships a generator target later (would be a powerful distribution vector — opportunity reviewer flagged). -- **FastAPI/Litestar partnership.** Mentioned by opportunity reviewer as the single largest adoption lever. Not in brief. Worth raising in PRD. -- **OpenTelemetry semantic-convention specifics.** Brief commits to OTel. PRD should pin specific spans/attributes/metric names (`http.client.request.duration`, etc.). -- **Public benchmark suite.** Skeptic reviewer suggested publishing latency overhead, throughput, memory vs raw httpx2 + tenacity, on every release. Cheap insurance against "this framework is slow" accusations. Not in brief. -- **Reversibility / sunset plan.** What happens at 6 months if external adoption misses the ≥3-projects target. Not stated. - -## References (for downstream verification) - -- httpx state: `gh api repos/encode/httpx`, `gh api repos/encode/httpx/commits`, `gh api repos/encode/httpx/discussions/3784` -- niquests state: `gh api repos/jawah/niquests` -- circuit-breaker-box source: `github.com/community-of-python/circuit-breaker-box` (commit `e1cb058`) -- Cross-language design references: openai/openai-python (`src/openai/_base_client.py`), anthropics/anthropic-sdk-python, stripe/stripe-python (`stripe/_http_client.py`), encode/httpx (`httpx/_client.py`, `httpx/_transports/`), sindresorhus/ky, mardiros/purgatory, danielfm/pybreaker, sony/gobreaker, Polly (.NET), resilience4j (JVM) -- Performance refs: Pydantic Performance docs (TypeAdapter caching), msgspec benchmarks (jcristharif.com/msgspec/benchmarks.html), AWS Builders Library (jitter/backoff), CPython issues #112169 (DNS) and #37355 (SSL GIL) -- Finagle retry budget: `github.com/twitter/finagle/blob/develop/finagle-core/src/main/scala/com/twitter/finagle/service/RetryBudget.scala` diff --git a/docs/archive/product-brief-httpware.md b/docs/archive/product-brief-httpware.md deleted file mode 100644 index fb23ab7..0000000 --- a/docs/archive/product-brief-httpware.md +++ /dev/null @@ -1,127 +0,0 @@ ---- -title: "Product Brief: httpware" -status: "complete" -created: "2026-05-11" -updated: "2026-05-12" -inputs: - - "Conversation with maintainer (Artur Shiriev / krenix512@proton.me)" - - "Source scan: /Users/kevinsmith/src/pypi/base-client" - - "Source scan: github.com/community-of-python/circuit-breaker-box" - - "GitHub API: encode/httpx, encode/httpx Discussion #3784, jawah/niquests, pydantic/httpx2" - - "Survey of openai-python, anthropic-sdk-python, stripe-python, hvac" - - "Survey of reqwest, ky, OkHttp, Polly, resilience4j, gobreaker, purgatory, pybreaker" ---- - -# Product Brief: httpware - -## Executive Summary - -`httpware` is a new Python async HTTP client framework for building resilient service clients. It supersedes `base-client` (community-of-python), which will be deprecated, and ships under the `modern-python` org. - -The library exists because every Python team building backend service clients hits the same wall: the underlying HTTP client leaks its types through your public API, retries and circuit breakers are bolted on as separate libraries with poor composition, and the test story forces consumers to learn transport-level mocking. The dominant async HTTP client was released-stalled for 17 months under `encode/httpx`, then forked to `pydantic/httpx2` on 2026-05-11 with stewardship picked up by Pydantic Services — `httpware` ships on the stewardship-renewed `httpx2` line. - -`httpware` solves the underlying design problems: the public API is **transport-agnostic** — the underlying HTTP client is an implementation detail behind a small `Transport` protocol, swappable from httpx2 to niquests (or anything else) without consumer changes. Resilience is a **composable middleware concern**: retries, timeouts, bulkheads, and a Finagle-style **retry budget** ship built-in; circuit breakers and other failure-policy primitives have a stable extension point and a reference design but are deliberately out of v1.0 (see Scope). Tests use a `RecordedTransport` and never see the underlying client. - -Now is the right time because `base-client`'s current design (httpx types leaking through the public API, dependency on `httpx._client` private modules, a circuit-breaker library with verified critical bugs) is unsalvageable without breaking changes. The `httpx2` transition is a forcing function: rebuilding on the stewardship-renewed line is the moment to also fix the architectural debt and missing resilience primitives in one motion. - -## The Problem - -Three pain points motivate this work, in order of severity. - -**1. The httpx → httpx2 transition.** `encode/httpx` 0.28.1 shipped 2024-12-06 — no release for 17+ months. The issue tracker was disabled. On 2026-02-27 the lead maintainer publicly stated they were stepping back from community engagement (Discussion #3784). On **2026-05-11** Pydantic Services forked the project as `pydantic/httpx2`, restored community channels, and released `v2.0.0b1` the same day. The strategic concern is now resolved — but every `base-client` consumer is still pinned to `encode/httpx`. Moving to the stewardship-renewed line is itself a breaking change for downstream services if we don't isolate the dependency. Rebuilding the wrapper is the moment to do it right. - -**2. `base-client`'s public API leaks httpx everywhere.** Today, `BaseClient` exposes `httpx.AsyncClient` as a public dataclass field; method signatures take and return `httpx.Request` / `httpx.Response`; error classes hold `httpx.Response`; the implementation imports from `httpx._client` and `httpx._types` (private modules). Tests assert against 19 specific httpx exception types via `respx.mock(side_effect=...)`. **Every consumer is tightly coupled to httpx, including via httpx's private API.** Migrating to httpx2 — or anything else — is a breaking change for every downstream service that uses `base-client`. - -**3. Resilience is bolted on poorly.** `base-client` depends on `circuit-breaker-box`, which a source-level read confirms has **five critical or high-severity bugs**: no half-open state at all (recovery thundering herd), first failure not counted in retries, Redis backend refreshes the TTL on every increment (the breaker never auto-recovers under sustained load), non-atomic in-memory increment, and an off-by-one in the availability check. It is not a faithful circuit-breaker implementation. Separately, no popular Python library currently ships a **retry budget** — the single most effective control against retry storms, well-understood at Finagle/Envoy scale and absent from Python entirely. The Python ecosystem has the right *parts* (`tenacity`, `purgatory`, `pybreaker`) but no canonical framework that composes them with a coherent ordering, observability, and async semantics. - -## The Solution - -`httpware` is a small, opinionated framework with six design pillars: - -1. **Own the abstractions.** `httpware.Request`, `httpware.Response`, `httpware.Transport`, `httpware.Middleware` are first-class types defined by the library. The underlying HTTP client (`httpx2` by default) sits behind the `Transport` protocol. No consumer code ever imports `httpx2`. - -2. **Single-call, typed-response API.** `await client.get("/users/1", response_model=User)` returns a typed `User`. No two-step prepare/send, no fluent builder. Matches the Stainless pattern that openai-python and anthropic-python proved at scale. - -3. **First-class middleware, onion model.** Every request flows through `Observability → RetryBudget → Retry → [extension slot] → Bulkhead → Timeout → Transport`. Users add custom middleware (auth refresh, tracing, signing, or a third-party circuit breaker) on the same axis. Built-in primitives are themselves middleware — composable, replaceable, removable. - -4. **Pluggable validation.** `response_model=` accepts any type — pydantic is the default validator, but a `ResponseDecoder` protocol lets users plug in `msgspec`, `attrs`, plain dataclasses, or anything else. The library does not hard-couple to one validation library, avoiding the same leakage problem we're solving for the transport. - -5. **Retry budget as the flagship resilience feature.** The single thing that turns retry from a footgun into a safe default. Token-bucket admission control over the whole client's retry traffic (Finagle defaults: 20% retry ratio + 10/sec floor + 10s TTL). Caps retry storms before they happen; degrades gracefully when the budget is exhausted. Almost no Python library ships this — `httpware` makes it on by default. - -6. **Tests speak the library's language, not the transport's.** A `RecordedTransport({(method, url): Response})` is the primary test path. Consumers never write respx routes and never assert against httpx2 exception types. Mocking is a 3-line fixture. - -## What Makes This Different - -The Python landscape has three rough cohorts: - -- **Stainless-generated SDKs** (openai-python, anthropic-python): excellent typed responses, granular exception hierarchy, but deliberately no middleware system — retries are hand-rolled inside the request loop, circuit breakers are not supported, and the library exists only because it's generated from an OpenAPI spec. Not a framework for building your own service clients. -- **Raw httpx2 / niquests**: low-level transport. No resilience built in. Tests use transport-level mocking. Public API tied to the chosen client. -- **`base-client` and similar thin wrappers**: shims that re-expose the underlying client's types and bolt on one or two resilience libraries with poor composition. - -`httpware` occupies the gap: a **framework** with the Stainless typed-response ergonomics, plus first-class middleware-based resilience (which Stainless deliberately omits), plus genuine transport-agnosticism. The retry budget is a category-leading feature. The honest moat is design quality and the combination — none of the pieces are individually novel, but the combination doesn't exist in Python today. - -The fuller positioning: **`httpware` is to Python what Polly is to .NET and resilience4j is to the JVM** — a canonical resilience-first HTTP framework. Python has no equivalent category leader; the transport-agnostic abstraction is the proof point that lets `httpware` stand independent of any one underlying client's fate. - -## Who This Serves - -**Primary**: Backend Python teams in `modern-python` and partner orgs building async service-to-service clients (FastAPI services calling other internal or third-party APIs). Several teams already depend on `base-client`; they need a path forward. - -**Secondary**: Teams building **LLM and AI-gateway clients**. AI service traffic is the highest-volume, highest-failure HTTP workload in Python in 2026: long streaming responses, aggressive rate-limits, retry storms when a provider degrades, multi-vendor failover. The middleware model is literally what teams hand-roll on top of openai/anthropic SDKs today — `httpware` makes it a one-line declaration. - -**Tertiary**: The wider Python community building service clients on PyPI. `httpware` is open-source and credible as a default choice for new projects that today would reach for `httpx2 + tenacity + a-circuit-breaker-library + a custom wrapper`. - -**Success for a user looks like**: defining a service client in under 50 lines that gets resilient retries, retry budgeting, observability, and typed responses for free; swapping the underlying HTTP client (httpx2 → niquests, when the time comes) with a single-line change; writing tests that don't know what an `httpx2.Request` is. - -## Success Criteria - -**v1.0 release criteria** - -- All public API surface uses `httpware.*` types. `grep -r 'import httpx2' httpware/ examples/ tests/` returns zero hits outside the `httpware.transports.httpx2` module. -- Built-in middleware ships and is documented: Retry, RetryBudget, Bulkhead, Timeout, Observability (with OpenTelemetry instrumentation). -- Middleware extension point documented and validated by a reference circuit-breaker middleware (built atop `purgatory` or equivalent) shipped as an example or companion package, not as a v1.0 dependency. -- `RecordedTransport` covers the common test patterns from current `base-client` consumers (success, error status, exception, retry behavior). -- `ResponseDecoder` protocol shipped with pydantic and msgspec adapters. -- At least one consumer service inside `modern-python` is migrated from `base-client` to `httpware` and running in production. - -**Adoption criteria (6 months post-1.0)** - -- All known `base-client` consumers migrated; `base-client` archived. -- ≥3 external (non-`modern-python`) projects on PyPI depending on `httpware`. - -## Scope - -**In scope for v1.0** - -- Async-only API (no sync facade). -- `Transport` protocol + default `Httpx2Transport`. (Niquests transport: nice-to-have, not blocking.) -- Single-call request API with `response_model=` typed responses. -- `ResponseDecoder` protocol — pluggable validator. Default pydantic adapter; msgspec adapter shipped. User can plug attrs, dataclasses, or anything else. -- Middleware system (onion model with phase-shortcut helpers) and a documented extension point in the resilience layer (the "extension slot" between Retry and Bulkhead) where third-party middleware (e.g. circuit breakers) cleanly compose. -- Built-in middleware: Retry (full-jitter exponential backoff, idempotency-aware), RetryBudget (Finagle defaults), Bulkhead (`asyncio.Semaphore`-based), Timeout, Observability hooks with first-class OpenTelemetry semantic-convention support. -- Exception hierarchy keyed by status (`NotFoundError`, `RateLimitedError`, etc.) with plain fields (`status: int`, `body: bytes`, `headers`, `json`). No transport-typed objects on exceptions. -- `RecordedTransport` for tests. -- `with_options(...)` returning a new client sharing the pool. -- Streaming via `async with client.stream(...) as resp`. -- Security defaults: TLS verification on, configurable secret-redaction hook on outgoing logs/spans, documented CVE disclosure channel. -- Python 3.11+ (drop 3.10 to use `TaskGroup`). - -**Explicitly out of v1.0 (designed for, not implemented in v1.0)** - -- **In-house circuit breaker.** The middleware extension slot is the contract — a circuit-breaker middleware can be plugged in without library changes. A reference implementation (likely wrapping `purgatory`) ships as an example or companion package. The library's design notes capture the intended state machine (3-state, sliding-window, slow-call, jittered half-open) so the future implementation has a target. -- **Sync API.** Async-only at v1.0. Sync support, if it comes, ships as a parallel class hierarchy (à la Stainless). Users with mixed sync/async codebases (Celery workers, scripts, migrations) keep using `requests` or httpx2 synchronously for now. -- **Niquests transport.** The `Transport` protocol guarantees it's a future-proof addition; shipping it is post-v1.0. -- **Distributed resilience-state coordination** beyond a single Redis backend. No gossip protocols. -- **OpenAPI-driven codegen.** -- **Backwards-compatibility shim for `base-client`.** Migration is documented but not automated. - -## Technical Approach - -A short engineering note (not the executive read): - -The reference implementation is a **clean greenfield rewrite**, not a fork of `base-client`. Default `Transport` wraps `httpx2.AsyncClient` and adapts types at the seam. The default `ResponseDecoder` is a pydantic adapter that caches `TypeAdapter` instances per `response_model` and uses `validate_json(content)` rather than `validate_python(json())` — roughly 2× parse-and-validate throughput, and a fix for a documented performance footgun in current `base-client`. A msgspec adapter ships alongside for users who want msgspec's faster path. Default `Limits` and `Timeout` shipped by the library are sensible for service workloads (`Timeout(connect=5, read=30, write=30, pool=5)`, `Limits(max_connections=100, max_keepalive=20)`), not the `Timeout(1)` in current `base-client` examples. - -## Vision (2-3 years) - -`httpware` becomes the default choice in the `modern-python` ecosystem for any service that talks to another service over HTTP. It earns adoption outside the org because no other Python library combines its typed-response ergonomics, first-class resilience, and transport-agnosticism. - -The transport-agnostic design isn't strategic insurance against any one client's fate (the httpx → httpx2 transition demonstrated that the ecosystem can self-correct), but it remains durable value: any consumer of `httpware` rides out future transport changes without code rewrites, and a future `NiquestsTransport` (or any other backend) drops into the same `Transport` slot. diff --git a/docs/archive/stories/1-1-project-scaffold-and-tooling.md b/docs/archive/stories/1-1-project-scaffold-and-tooling.md deleted file mode 100644 index 14ddc4d..0000000 --- a/docs/archive/stories/1-1-project-scaffold-and-tooling.md +++ /dev/null @@ -1,225 +0,0 @@ ---- -story_key: 1-1-project-scaffold-and-tooling -epic: 1 -story: 1 -title: Project scaffold and tooling -status: done -created: 2026-05-12 -completed: 2026-05-13 -input_documents: - - docs/prd.md - - docs/architecture.md - - docs/epics.md ---- - -# Story 1.1: Project scaffold and tooling - -## Story - -**As a** `httpware` maintainer, -**I want** a fully-configured project skeleton with the org's conventions, -**So that** subsequent stories can implement library code without fighting tooling. - -## Acceptance Criteria - -**AC1.** **Given** a fresh checkout of a new GitHub repo at `modern-python/httpware`, **When** I run `uv init --lib httpware` followed by the org-convention port from `modern-python/modern-di`, **Then** the repo has `src/httpware/__init__.py`, `src/httpware/py.typed`, and a `pyproject.toml` declaring `httpx2>=2.0.0,<3.0` and `pydantic>=2.0,<3.0` as dependencies. - -**AC2.** **And** extras `[msgspec]`, `[otel]`, `[niquests]`, `[all]` are declared. - -**AC3.** **And** dev/lint dep groups match `modern-di` (pytest, pytest-cov, pytest-asyncio, pytest-repeat, pytest-benchmark; ruff, ty, eof-fixer, typing-extensions); plus `hypothesis` in dev for property-based tests. - -**AC4.** **And** `[tool.ruff]`, `[tool.pytest.ini_options]` match `modern-di` with `target-version = "py311"`. - -**AC5.** **And** root files exist: `Justfile`, `LICENSE` (MIT), `SECURITY.md`, `CONTRIBUTING.md`, `CHANGELOG.md`, `CLAUDE.md`, `context7.json`, `.gitignore`. - -**AC6.** **And** `.github/workflows/ci.yml` runs `ruff check`, `ty`, `pytest --cov` on Python 3.11–3.14. - -**AC7.** **And** `uv build` produces a wheel and `pip install dist/*.whl` succeeds in a clean venv (smoke-import: `python -c "import httpware"` exits 0). - -## Tasks/Subtasks - -- [x] **Task 1: Initialize uv library scaffold** - - [x] 1.1: Run `uv init --lib` in `/Users/kevinsmith/src/pypi/httpware/` - - [x] 1.2: Verify `src/httpware/__init__.py` exists - - [x] 1.3: Add `src/httpware/py.typed` zero-byte marker (auto-created by `uv init --lib`) - -- [x] **Task 2: Configure pyproject.toml** - - [x] 2.1: Set `[project]` metadata (name, description, authors, requires-python>=3.11, license=MIT, classifiers for Python 3.11-3.14, `Typing :: Typed`) - - [x] 2.2: Declare main dependencies: `httpx2>=2.0.0,<3.0` (tightened from `b1` after smoke install verified GA is on PyPI), `pydantic>=2.0,<3.0` - - [x] 2.3: Declare extras: `[project.optional-dependencies]` for `msgspec`, `otel`, `niquests`, `all` - - [x] 2.4: Configure `[build-system]` with `uv_build>=0.11,<0.12` (upper-bounded per uv's recommendation) - - [x] 2.5: Configure `[tool.uv.build-backend]` (module-name = "httpware", module-root = "src") - - [x] 2.6: Add `[project.urls]` for repository and docs - -- [x] **Task 3: Port modern-di conventions to pyproject.toml** - - [x] 3.1: Fetched live `modern-python/modern-di/pyproject.toml`, `Justfile`, `.github/workflows/{ci,publish}.yml`, `.gitignore`, `CLAUDE.md`, `context7.json` via `gh api` - - [x] 3.2: Copied `[tool.ruff]` (line-length=120, target-version="py311", fix=true, unsafe-fixes=true) and `[tool.ruff.lint]` (select=ALL, ignore set: D1, S101, TCH, FBT, D203, D213, COM812, ISC001) - - [x] 3.3: Copied `[tool.pytest.ini_options]` (asyncio_mode="auto", asyncio_default_fixture_loop_scope="function"); adjusted `pythonpath` to `["src"]` and `--cov` to `src/httpware` for src/-layout - - [x] 3.4: Copied `[tool.coverage]` config - - [x] 3.5: Added dev dep group: pytest, pytest-cov, pytest-asyncio, pytest-repeat, pytest-benchmark, hypothesis - - [x] 3.6: Added lint dep group: ruff, ty, eof-fixer, typing-extensions - -- [x] **Task 4: Add root configuration files** - - [x] 4.1: `LICENSE` — MIT, copyright "Modern Python contributors" - - [x] 4.2: `Justfile` with `install`, `lint`, `lint-ci`, `test`, `test-branch`, `publish` recipes (verbatim from modern-di) - - [x] 4.3: `SECURITY.md` documenting GitHub Security Advisories disclosure channel and 90-day private-disclosure window (per NFR10) - - [x] 4.4: `CONTRIBUTING.md` — development workflow + architecture invariants - - [x] 4.5: `CHANGELOG.md` — Keep-a-Changelog format with `Unreleased` section populated with scaffold-story changes - - [x] 4.6: `CLAUDE.md` — AI-agent guidance pointing at base-client/docs/{prd,architecture,epics}.md, CI-enforced invariants, code conventions, module layout, 5 protocol seams - - [x] 4.7: `context7.json` — minimal config matching modern-di style (URL only; public_key TBD) - - [x] 4.8: `.gitignore` — Python standard ignores from modern-di + `uv.lock` in ignore list (library convention) - -- [x] **Task 5: GitHub Actions CI workflow** - - [x] 5.1: `.github/workflows/ci.yml` runs ruff lint (via `just lint-ci`), pytest with coverage upload to Codecov - - [x] 5.2: Python matrix: 3.11, 3.12, 3.13, 3.14 (matches our floor; modern-di had 3.10 included but our PRD raised the floor) - - [x] 5.3: Uses `astral-sh/setup-uv@v3` + `extractions/setup-just@v2` (matches modern-di) - -- [x] **Task 6: Verify build and install** - - [x] 6.1: `uv build` produced `dist/httpware-0-py3-none-any.whl` (3.0K) and `dist/httpware-0.tar.gz` (3.0K) - - [x] 6.2: Installed wheel into clean tempdir venv via `uv venv --python 3.11` + `uv pip install`; 12 packages resolved (httpware + httpx2==2.0.0 + pydantic==2.13.4 + transitive deps) - - [x] 6.3: Smoke-import: `python -c "import httpware"` exits 0; `__all__ == []` - - [x] 6.4: `ruff format`, `ruff check`, `ty check` all pass on empty `src/httpware/__init__.py`; `pytest` collected 0 tests (expected — scaffold story has no library code yet) - - [x] 6.5: Initialized git repo (auto-created by `uv init --lib`), staged all scaffold files, committed as `fe4df95 chore: initial project scaffold (Story 1.1)` on `main` - -## Dev Notes - -**Architecture references** (all in `docs/`): - -- `architecture.md` § Starter Template Evaluation — selected starter is `uv init --lib` + port from `modern-python/modern-di` -- `architecture.md` § Project Structure & Boundaries — full directory tree and root-file enumeration -- `architecture.md` § Implementation Patterns — naming, structure, type-hint style, etc. -- `prd.md` § Developer Tool Specific Requirements — language matrix, install methods, IDE integration - -**Key design decisions affecting this story:** - -- **Build backend:** `uv_build` (PEP 517 compliant). `[build-system] requires = ["uv_build"]`, `build-backend = "uv_build"`. -- **Layout:** src/-layout. `[tool.uv.build-backend] module-name = "httpware"`, `module-root = "src"`. (modern-di uses flat layout; we use src/ because it's the safer default for new repos and `uv init --lib` defaults to it.) -- **Type checker:** `ty` (Astral), NOT mypy. Suppression comments are `# ty: ignore[]`. -- **Python floor:** 3.11+ (`TaskGroup`, `except*`). -- **License:** MIT (matches modern-di and modern-python org default). - -**Reference URLs to fetch live during implementation:** - -- `https://api.github.com/repos/modern-python/modern-di/contents/pyproject.toml` — authoritative ruff/pytest/coverage config -- `https://api.github.com/repos/modern-python/modern-di/contents/Justfile` — recipes to port -- `https://api.github.com/repos/modern-python/modern-di/contents/.github/workflows` — CI workflow structure - -**No tests required for this story** beyond the build-verify smoke checks. Tests for library functionality begin in Story 1.2. - -**Definition of Done for this story:** - -- All AC criteria pass -- `uv build` succeeds -- Clean-venv install + smoke import works -- `ruff check`, `ty check` pass on empty src/ tree (the package exists but contains no code yet, so these should pass trivially) -- Git repo initialized with first commit - -## Dev Agent Record - -### Implementation Plan - -1. **Initialize via `uv init --lib`** in the empty `/Users/kevinsmith/src/pypi/httpware/` directory — single command sets up `src/httpware/__init__.py`, `py.typed`, minimal `pyproject.toml`, `.gitignore`, `.python-version`, `README.md`, and a fresh `.git/` repo. -2. **Fetch the authoritative `modern-python/modern-di` config files** via `gh api` (pyproject.toml, Justfile, .github/workflows/{ci,publish}.yml, .gitignore, CLAUDE.md, context7.json) and use them as the verbatim source for org conventions, adjusting only: - - Python floor: `>=3.10,<4` → `>=3.11,<4` (per PRD Story 1.1 AC and architecture decision to use `TaskGroup`/`except*`) - - ruff `target-version`: `py310` → `py311` - - Layout: flat (`modern_di/`) → src/ (`src/httpware/`); set `[tool.uv.build-backend] module-root = "src"` - - pytest `pythonpath`: `["."]` → `["src"]`; `--cov` source: `.` → `src/httpware` - - Drop Python 3.10 from CI matrix -3. **Override `__init__.py`** content from the default `def hello()` stub to a docstring + empty `__all__`. -4. **Override `.python-version`** from `3.14` (uv's default) to `3.11` (our declared floor). -5. **Add new content not in modern-di**: dependencies (`httpx2>=2.0,<3.0`, `pydantic>=2.0,<3.0`), install extras, `LICENSE`, `SECURITY.md`, `CONTRIBUTING.md`, `CHANGELOG.md`, populated `README.md`. -6. **Validate**: `uv sync` → `ruff format` → `ruff check` → `ty check` → `pytest` → `uv build` → clean-venv install + smoke import. -7. **Commit**: stage everything except gitignored files (`.python-version`, `.venv`, `uv.lock`), single initial commit on `main`. - -### Debug Log - -- `uv init --lib` auto-created `py.typed` (good — no need to add manually as originally planned in subtask 1.3). -- Initial `httpx2>=2.0.0b1,<3.0` constraint (from the original AC1) caused `uv pip install` (without `--prerelease=allow`) to skip httpx2 in clean-venv smoke test. Tightened to `>=2.0.0,<3.0` after verifying `httpx2==2.0.0` GA was published on PyPI on 2026-05-12. AC1, base-client/docs/{prd,architecture,epics}.md, and this story file have all been updated in a follow-up commit to match the implemented constraint. -- `uv build` warned about `[build-system] requires = ["uv_build"]` lacking an upper bound. Pinned to `>=0.11,<0.12` to silence the warning and prevent future breakage when uv_build 0.12 ships. -- `git add` initially refused `.python-version` (in `.gitignore` from modern-di's convention); kept it untracked. Same for `uv.lock` (added to `.gitignore` after initial commit attempt staged it). - -### Completion Notes - -**AC verification — all 7 satisfied:** - -- **AC1** ✓ `src/httpware/__init__.py` and `src/httpware/py.typed` exist; `pyproject.toml` declares `httpx2>=2.0.0,<3.0` and `pydantic>=2.0,<3.0`. -- **AC2** ✓ Extras `[msgspec]`, `[otel]`, `[niquests]`, `[all]` all declared in `[project.optional-dependencies]`. -- **AC3** ✓ Dev group: pytest, pytest-cov, pytest-asyncio, pytest-repeat, pytest-benchmark, hypothesis. Lint group: ruff, ty, eof-fixer, typing-extensions. Match modern-di + hypothesis addition. -- **AC4** ✓ `[tool.ruff]` and `[tool.pytest.ini_options]` ported verbatim from modern-di with `target-version = "py311"` (raised from `py310`). -- **AC5** ✓ All eight root files exist: Justfile, LICENSE, SECURITY.md, CONTRIBUTING.md, CHANGELOG.md, CLAUDE.md, context7.json, .gitignore. -- **AC6** ✓ `.github/workflows/ci.yml` runs ruff/ty/pytest with coverage upload on Python 3.11–3.14 matrix (3.10 dropped per our floor; 3.14 included since GA is on `actions/setup-python`). -- **AC7** ✓ `uv build` produced wheel+sdist; clean-venv install of the wheel resolved 12 packages (httpware + httpx2 2.0.0 + pydantic 2.13.4 + transitive); `python -c "import httpware"` exited 0 with `__all__ == []`. - -**Definition of Done:** - -- All Tasks/Subtasks marked `[x]` -- All 7 AC pass -- `ruff format` + `ruff check` + `ty check` + `pytest` all pass locally -- `uv build` succeeds; wheel installs and imports cleanly in a fresh venv -- File List complete; Change Log updated -- Initial commit on `main` (`fe4df95`) - -**Deviations from PRD/Architecture docs (worth noting for future cleanup):** - -- Original AC1 specified `httpx2>=2.0.0b1,<3.0` (written when only the beta was published). Story tightened to `>=2.0.0,<3.0` after verifying GA shipped on PyPI 2026-05-12; planning artifacts (PRD, Architecture, Epics, AC1) updated to match in a follow-up commit. Deviation resolved. -- `module-name = "httpware"` and `module-root = "src"` (src/ layout) chosen over modern-di's flat layout, per architecture decision §Starter Template (rationale: src/ layout prevents test code from accidentally importing local source). - -**Tests written:** None — scaffold story has no library code. Tests begin in Story 1.2. - -## File List - -Files added (14 total): - -- `.github/workflows/ci.yml` — CI workflow (ruff/ty/pytest, Python 3.11-3.14 matrix) -- `.gitignore` — modern-di convention + project-specific ignores -- `CHANGELOG.md` — Keep-a-Changelog format -- `CLAUDE.md` — AI-agent guidance for working in this repo -- `CONTRIBUTING.md` — contributor workflow and architecture invariants -- `Justfile` — install/lint/test/publish recipes (verbatim from modern-di) -- `LICENSE` — MIT -- `README.md` — project overview, install, quickstart, highlights -- `SECURITY.md` — disclosure policy with 90-day window -- `context7.json` — context7 docs index pointer -- `docs/stories/1-1-project-scaffold-and-tooling.md` — this story file -- `pyproject.toml` — full project config (deps, extras, ruff, pytest, coverage, dep groups) -- `src/httpware/__init__.py` — package init with docstring and empty `__all__` -- `src/httpware/py.typed` — zero-byte typing marker - -Generated/transient (not committed): - -- `.python-version` — gitignored per modern-di convention -- `uv.lock` — gitignored per modern-di convention (library project) -- `.venv/` — gitignored -- `dist/` — gitignored - -## Change Log - -| Date | Change | Notes | -|---|---|---| -| 2026-05-12 | Story created | Extracted from `base-client/docs/epics.md` Story 1.1; reorganized into tasks/subtasks for dev workflow. | -| 2026-05-13 | Story completed | All 7 AC pass; initial commit `fe4df95` on `main`; lint+typecheck+build+smoke-install verified. | - -## Status - -`done` - -### Review Findings - -_Code review run: 2026-05-13. Reviewers: Blind Hunter, Edge Case Hunter, Acceptance Auditor. 6 patches applied, 3 dismissed by maintainer (not errors), 6 deferred, 20 dismissed as noise (2 decision-needed items were resolved → dismissed: `.gitignore plan.md` blacklist is intentional convention; AC6 lint-on-single-version matches modern-di canon and is accepted)._ - -- [x] [Review][Patch] CHANGELOG declared `httpx2>=2.0.0b1,<3.0` while pyproject shipped `>=2.0.0,<3.0`. [`CHANGELOG.md:13`] — applied -- [x] [Review][Patch] CHANGELOG `[Unreleased]` link was `compare/HEAD...HEAD` — replaced with `commits/main` until first tag. [`CHANGELOG.md:20`] — applied -- [x] [Review][Dismissed] `version = "0"` placeholder. [`pyproject.toml:29`] — dismissed by maintainer, not an error -- [x] [Review][Patch] `[all]` extra refactored to self-reference siblings: `all = ["httpware[msgspec,otel,niquests]"]`. [`pyproject.toml:42-47`] — applied -- [x] [Review][Patch] README "Optional extras" snippet — added `pip install httpware[niquests]`. [`README.md:22-26`] — applied -- [x] [Review][Dismissed] CI `timeout-minutes`. [`.github/workflows/ci.yml:14, 26`] — dismissed by maintainer, not an error -- [x] [Review][Dismissed] CI explicit `permissions:` block. [`.github/workflows/ci.yml:1-12`] — dismissed by maintainer, not an error -- [x] [Review][Patch] Duplicate `--cov` flag — dropped `--cov=src/httpware` from CI invocation; `addopts` is now the single source of `--cov` source. [`.github/workflows/ci.yml:45`] — applied -- [x] [Review][Patch] File List header `15 total` → `14 total`. [`docs/stories/1-1-project-scaffold-and-tooling.md:171`] — applied - -- [x] [Review][Defer] Codecov upload fails on fork PRs without `CODECOV_TOKEN`; matches modern-di pattern, accepted tradeoff. [`.github/workflows/ci.yml:46-52`] — deferred, pre-existing -- [x] [Review][Defer] `just publish` does not validate `GITHUB_REF_NAME` / `PYPI_TOKEN`; local invocation could corrupt the version. [`Justfile:25-29`] — deferred, release-flow hygiene -- [x] [Review][Defer] `uv_build>=0.11,<0.12` is a one-minor-version window that will expire fast. [`pyproject.toml:54`] — deferred, will bump on release -- [x] [Review][Defer] Python 3.14 in CI matrix; httpx2 / pydantic / uv_build wheels may not yet exist on 3.14, causing the matrix entry to fail. [`.github/workflows/ci.yml:30-33`] — deferred, wait-and-see -- [x] [Review][Defer] `[tool.ruff.lint] select = ["ALL"]` paired with unpinned `ruff`/`ty` — any new ruff release adds rules and breaks CI overnight. [`pyproject.toml:70-72, 84-85`] — deferred, matches modern-di -- [x] [Review][Defer] No `[test]` extra declared; CI relies on `--all-extras`, so any future heavy extra is pulled into every CI run. [`pyproject.toml:35-47`] — deferred, scope creep concern diff --git a/docs/archive/stories/1-2-core-data-types.md b/docs/archive/stories/1-2-core-data-types.md deleted file mode 100644 index cfa6839..0000000 --- a/docs/archive/stories/1-2-core-data-types.md +++ /dev/null @@ -1,254 +0,0 @@ ---- -story_key: 1-2-core-data-types -epic: 1 -story: 2 -title: Core data types -status: done -created: 2026-05-13 -input_documents: - - docs/prd.md - - docs/architecture.md - - docs/epics.md - - docs/stories/1-1-project-scaffold-and-tooling.md ---- - -# Story 1.2: Core data types - -## Story - -**As a** library author, -**I want** immutable `Request`, `Response`, `Limits`, `Timeout`, and `ClientConfig` types, -**So that** every other module has stable primitives to build on. - -## Acceptance Criteria - -**AC1.** **Given** the scaffold from Story 1.1, **When** I implement `src/httpware/request.py`, **Then** `Request` is a `@dataclass(frozen=True, slots=True)` with exactly these fields, in this order, with these types: `method: str`, `url: str`, `headers: Mapping[str, str]`, `params: Mapping[str, str]`, `cookies: Mapping[str, str]`, `body: bytes | None`, `extensions: Mapping[str, Any]`. `method` and `url` are required (no defaults); the four `Mapping` fields default to an empty mapping via `field(default_factory=dict)`; `body` defaults to `None`. - -**AC2.** **And** `Request` has methods `with_header(name: str, value: str) -> Request`, `with_url(url: str) -> Request`, `with_body(body: bytes | None) -> Request`, and `with_query(params: Mapping[str, str]) -> Request`. Each method returns a new instance via `dataclasses.replace(...)`. `with_header` adds-or-replaces a single header (case-insensitive match on the header name is **not** required in this story — the matching helper lands in Story 2.3); `with_query` replaces the full `params` mapping with the supplied one (merge semantics are also Story 2.3's job). - -**AC3.** **And** `Response` (in `src/httpware/response.py`) is a `@dataclass(frozen=True, slots=True)` with exactly these fields, in this order, with these types: `status: int`, `headers: Mapping[str, str]`, `content: bytes`, `url: str`, `elapsed: float`. All five fields are required (no defaults). - -**AC4.** **And** `Response.text` is a `@property` that decodes `content` as text. Charset is taken from the `charset=` parameter of the `Content-Type` header if present, else `"utf-8"`. `Response.json()` is a method (not a property — it can raise) that calls `json.loads(content)` and returns the parsed value (`Any`). Neither value is stored on the instance; both are computed on each access. Slots must not declare backing fields for either. - -**AC5.** **And** `src/httpware/config.py` defines three frozen dataclasses with these exact defaults: `Timeout(connect: float = 5.0, read: float = 30.0, write: float = 30.0, pool: float = 5.0)`; `Limits(max_connections: int = 100, max_keepalive_connections: int = 20, keepalive_expiry: float = 5.0)`; `ClientConfig(base_url: str | None = None, default_headers: Mapping[str, str] = field(default_factory=dict), default_query: Mapping[str, str] = field(default_factory=dict), timeout: Timeout = field(default_factory=Timeout), limits: Limits = field(default_factory=Limits))`. All three are `@dataclass(frozen=True, slots=True)`. `ClientConfig` is intentionally narrow at this point; transport / decoder / middleware / auth / redactor fields land in later stories (1.4–5.3) via additive `dataclasses.replace`-compatible extension. - -**AC6.** **And** `src/httpware/__init__.py` re-exports `Request`, `Response`, `Limits`, `Timeout`, `ClientConfig` and lists them in `__all__` (sorted alphabetically). `from httpware import Request, Response, Limits, Timeout, ClientConfig` succeeds. - -**AC7.** **And** `uv run ty check` and `uv run ruff check` pass with zero diagnostics on the new modules. - -**AC8.** **And** unit tests under `tests/` cover, at minimum: (a) `Request` is frozen — attempting to assign to a field raises `FrozenInstanceError`; (b) every `with_*` method returns a new instance and leaves the original unchanged; (c) two `Request` instances with identical field values compare equal under `==`; (d) `Response.text` decodes UTF-8 by default and honors a non-UTF-8 `Content-Type` charset (e.g. `text/plain; charset=latin-1`); (e) `Response.json()` round-trips a small object; (f) `Limits()`, `Timeout()`, and `ClientConfig()` constructed with no args match the defaults in AC5; (g) `Timeout`, `Limits`, `ClientConfig` are frozen. `uv run pytest` exits 0 with all tests passing. - -## Tasks/Subtasks - -- [x] **Task 1: Implement `Request` in `src/httpware/request.py`** (AC1, AC2) - - [x] 1.1: Module docstring (one short line per `docs/architecture.md#Docstring Style`). - - [x] 1.2: Define `Request` as `@dataclass(frozen=True, slots=True)` with the seven fields from AC1 in the specified order and types. Use `from collections.abc import Mapping` and `from typing import Any` — no `from __future__ import annotations`. - - [x] 1.3: Implement `with_header`, `with_url`, `with_body`, `with_query` as instance methods. Each method body is a single `return dataclasses.replace(self, =)` expression (for `with_header`, build the new headers dict first: `{**self.headers, name: value}`). - - [x] 1.4: Public class docstring (one line). No private docstrings required. - -- [x] **Task 2: Implement `Response` in `src/httpware/response.py`** (AC3, AC4) - - [x] 2.1: Module docstring. - - [x] 2.2: Define `Response` as `@dataclass(frozen=True, slots=True)` with the five fields from AC3 in order. - - [x] 2.3: Add `@property def text(self) -> str` that parses the charset out of `self.headers.get("content-type", "")` (case-insensitive header lookup — accept either casing in the tests) and decodes `self.content` with that encoding, falling back to `"utf-8"`. - - [x] 2.4: Add `def json(self) -> Any` that returns `json.loads(self.content)`. - - [x] 2.5: Do **not** add `text` or `json` to `__slots__` (they are not stored). With `slots=True` and `@property`, the field-vs-property distinction is correct; the property is defined on the class. - -- [x] **Task 3: Implement `Limits`, `Timeout`, `ClientConfig` in `src/httpware/config.py`** (AC5) - - [x] 3.1: Module docstring. - - [x] 3.2: Define `Timeout` with the four `float` fields and defaults from AC5. - - [x] 3.3: Define `Limits` with the three fields and defaults from AC5. - - [x] 3.4: Define `ClientConfig` with the five fields and defaults from AC5. Use `field(default_factory=Timeout)` and `field(default_factory=Limits)` for the nested config defaults — never bare `Timeout()` as a default value. - - [x] 3.5: All three classes are `@dataclass(frozen=True, slots=True)`. - -- [x] **Task 4: Update `src/httpware/__init__.py`** (AC6) - - [x] 4.1: Add absolute-path explicit imports: `from httpware.config import ClientConfig, Limits, Timeout` and `from httpware.request import Request` and `from httpware.response import Response`. - - [x] 4.2: Replace `__all__: list[str] = []` with the sorted public list: `["ClientConfig", "Limits", "Request", "Response", "Timeout"]`. - -- [x] **Task 5: Add unit tests under `tests/`** (AC8) - - [x] 5.1: Create `tests/` directory at repo root. **Deviation:** the story planned to skip `tests/__init__.py`, but ruff's `INP001` (implicit-namespace-package) fires under `select = ["ALL"]` and the project's ignore list does not exempt it. Added an empty `tests/__init__.py` to satisfy the lint gate. Pytest discovery still works the same way. - - [x] 5.2: `tests/conftest.py` — placeholder docstring (`"""Shared pytest fixtures for the httpware test suite."""`); no fixtures yet. - - [x] 5.3: `tests/test_request.py` — frozen check, `with_*` immutability on all four mutators (add + replace for `with_header`), equality, independent default mappings. - - [x] 5.4: `tests/test_response.py` — `.json()` round-trip, `.text` UTF-8 default, Unicode default decode, parametrized case-insensitive charset (`content-type` / `Content-Type` / `CONTENT-TYPE` all decode latin-1), missing-charset fallback, equality, frozen. - - [x] 5.5: `tests/test_config.py` — defaults asserted via dataclass equality (`assert Timeout() == Timeout(connect=5.0, …)`) rather than per-field comparisons, which sidesteps `PLR2004` (magic-value-in-comparison) cleanly; nested-default assertion (`cfg.timeout == Timeout()`); frozen for all three. - -- [x] **Task 6: Validate locally** (AC7, AC8 — DoD) - - [x] 6.1: `just lint` passes (eof-fixer, ruff format, ruff check, ty check). - - [x] 6.2: `just test` passes: 24/24 tests in 0.12s; coverage 100% on `request.py`, `response.py`, `config.py`, `__init__.py` (well above the NFR23 floor of 90%). - - [x] 6.3: `uv run python -c "from httpware import Request, Response, Limits, Timeout, ClientConfig; print('ok')"` → `ok`. - - [x] 6.4: `CHANGELOG.md` `Unreleased` section now lists the new core data types. - -## Dev Notes - -### Architecture references (authoritative — read these before coding) - -- `docs/architecture.md` § Data Architecture (Decision 1) — frozen+slots, `dataclasses.replace`, `Response.content: bytes` primitive, `.text`/`.json()` as lazy/computed. -- `docs/architecture.md` § Configuration & Lifecycle (Decision 9) — `ClientConfig` is the immutable bag `AsyncClient` will hold; `with_options` uses `dataclasses.replace`. Story 1.2 ships the minimal `ClientConfig`; later stories extend it. -- `docs/architecture.md` § Naming Patterns — module names `snake_case` (`request.py`, `response.py`, `config.py`), classes `PascalCase`. -- `docs/architecture.md` § Type-Hint Style — **no** `from __future__ import annotations`; use PEP 604 unions (`int | None`), PEP 585 generics (`list[T]`, `dict[K, V]`), `collections.abc.Mapping` not `typing.Mapping`. Suppression comments are `# ty: ignore[]` only. -- `docs/architecture.md` § Structure Patterns — `config.py` is documented to also hold `Redactor`, but `Redactor` is **out of scope for this story** (lands in Story 5.3). Do not add it. -- `docs/architecture.md` § Public API Export Discipline — single `__all__` in `httpware/__init__.py`; explicit imports, no wildcards. -- `CLAUDE.md` § Code conventions — exception keyword-only, snake_case methods, no `a` prefix on async methods (not applicable here — no async in this story). - -### Key design points - -**Why `frozen=True, slots=True`:** Decision 1. `slots=True` cuts per-instance memory and prevents attribute typos. `frozen=True` enables structural sharing via `dataclasses.replace` (each `with_*` returns a new instance; the old one is safe to keep aliased). - -**Why `dataclasses.replace` rather than `evolve`/`attrs`/manual `__init__`:** Decision 1 explicitly rejects `attrs` (extra dep) and `pydantic.BaseModel` (NFR1 overhead budget). `dataclasses.replace` is stdlib and one line per method. - -**Why `collections.abc.Mapping` for header/param fields:** Read-only contract advertised in the type; the runtime value is whatever the constructor receives (typically a `dict`). Defensive copying / `MappingProxyType` wrapping is **not** required in this story — the cost (forces a copy on every header rewrite) outweighs the benefit at v0. If a future story needs structural immutability, it'll be a follow-up. - -**Default-factory rule:** Every `Mapping[...]` field uses `field(default_factory=dict)`. Never use a literal `{}` as a default — `@dataclass` will raise at class-creation time. Same for `Timeout` / `Limits` nested defaults in `ClientConfig`: `field(default_factory=Timeout)` (the type, not `Timeout()`). - -**`Response.text` charset parsing:** Keep it simple. `headers.get("content-type", "")` (try lowercase first; fall back to title-case `Content-Type` if needed — Python's `dict.get` is exact-match, so a tiny helper is fine, e.g. `_get_header_ci(headers, "content-type")`). Then `"charset="` substring search and split. Do **not** pull in `email.message.Message` or `httpx2`'s parser — those are deferred to the transport layer. If charset is missing or malformed, fall back to `"utf-8"`. - -**Slots + `@property`:** Properties on slotted dataclasses work, but you must NOT name a `__slots__` entry the same as a property (already enforced by the spec — `text` and `json` are not fields, just methods on the class). - -**`json` method, not property:** `json()` can raise `json.JSONDecodeError`. Methods make raising obvious in the call site (`resp.json()` vs `resp.json`). httpx does this too; matches consumer mental model. - -**Equality is automatic:** `@dataclass(eq=True)` is the default. Don't override `__eq__`. Two `Request` instances with the same field values (including same mapping contents) compare equal because dict equality is by content. - -**Hashability:** `frozen=True` makes the dataclass auto-generate `__hash__` — but only if all fields are themselves hashable. Mapping fields backed by `dict` are **not** hashable. The auto-generated `__hash__` will therefore raise `TypeError` at hash time, not at construction time. This is fine — consumers shouldn't be hashing `Request`/`Response`. Don't add a custom `__hash__` to "fix" it. - -### What lives where after this story - -| File | New / modified | Contents | -|---|---|---| -| `src/httpware/request.py` | **new** | `Request` frozen+slots dataclass + 4 `with_*` methods. | -| `src/httpware/response.py` | **new** | `Response` frozen+slots dataclass + `.text` property + `.json()` method. | -| `src/httpware/config.py` | **new** | `Timeout`, `Limits`, `ClientConfig` frozen+slots dataclasses. | -| `src/httpware/__init__.py` | **modify** | Add 5 explicit imports; replace empty `__all__` with sorted list of 5. | -| `tests/conftest.py` | **new** | Empty placeholder (`""`). | -| `tests/test_request.py` | **new** | AC8 (a)–(c) coverage. | -| `tests/test_response.py` | **new** | AC8 (d)–(e) coverage + frozen check. | -| `tests/test_config.py` | **new** | AC8 (f)–(g) coverage. | -| `CHANGELOG.md` | **modify** | One-line bullet under `Unreleased`. | - -### Read-before-edit (per architect's guidance) - -Files this story modifies: - -- `src/httpware/__init__.py` — currently a one-line docstring plus `__all__: list[str] = []`. The story replaces `__all__` and adds 5 explicit imports. Nothing else lives there yet. -- `CHANGELOG.md` — `Unreleased` already has the Story 1.1 scaffold bullet. Append one new bullet; do not reformat existing entries. - -Files this story creates (no prior state to preserve): `src/httpware/request.py`, `src/httpware/response.py`, `src/httpware/config.py`, `tests/conftest.py`, `tests/test_request.py`, `tests/test_response.py`, `tests/test_config.py`. - -### Carryover from Story 1.1 - -- `httpx2==2.0.0` GA is on PyPI and pinned `>=2.0.0,<3.0` in `pyproject.toml`. **Not used in this story** — no transport code yet. -- `pydantic>=2.0,<3.0` declared. **Not imported in this story** — that's Story 1.5 (`PydanticDecoder`). -- `uv.lock` is gitignored (library convention) — running `uv sync` locally is fine, the lock won't be committed. -- `--cov=src/httpware` is already in `[tool.pytest.ini_options]`, so coverage is on by default; no separate `--cov` invocation needed. -- `asyncio_mode = "auto"` is set — but there are **no async tests** in this story, so it doesn't matter. - -### Anti-patterns to reject (CI-enforced or will fail review) - -- ❌ `from __future__ import annotations` (architecture invariant; Python 3.11+ floor). -- ❌ `Optional[X]` / `Union[X, Y]` — use `X | None` / `X | Y` (PEP 604). -- ❌ `typing.Mapping`, `typing.Dict`, `typing.List` — use `collections.abc.Mapping`, `dict`, `list` (PEP 585). -- ❌ `# type: ignore` or `# mypy: ignore` — use `# ty: ignore[]` only. -- ❌ `print()` anywhere — there's no reason to print in this story. -- ❌ Mutable default values: `headers: Mapping[str, str] = {}` — must be `field(default_factory=dict)`. -- ❌ Hand-rolled `__eq__` / `__init__` — the dataclass machinery generates them. -- ❌ Storing `.text` / `.json` as fields — they are computed accessors per AC4 and Decision 1. -- ❌ Importing `httpx2` or `pydantic` from any of the three new modules — they have no business there yet (will be enforced by Story 6.4's CI grep gate; pre-empt it now). -- ❌ Adding `Redactor` to `config.py` — that's Story 5.3. -- ❌ Adding any field to `ClientConfig` beyond the five in AC5 (e.g., `transport`, `decoder`, `middleware`, `auth`, `redactor`) — those land in later stories. - -### Testing standards summary - -- pytest function-style tests; no `unittest.TestCase`. -- `pytest-asyncio` auto mode is on but unused here. -- Property-based tests (Hypothesis) **not required** in this story — concurrency-sensitive primitives (`RetryBudget`, `Bulkhead`) come later. The Hypothesis dep is already installed; ignore it. -- Coverage target: ≥90% on the three new modules (NFR23). Equality and `with_*` assertions should hit every line of `Request`; the property + json method should hit every line of `Response`; default-construction assertions should hit every line of `Limits`/`Timeout`/`ClientConfig`. - -### Definition of Done - -- All 8 ACs verified. -- All Task/Subtask checkboxes are `[x]`. -- `ruff format`, `ruff check`, `ty check`, `pytest` all pass locally. -- File List below is updated to reflect every changed and new file. -- Change Log has a new dated entry. -- `__all__` and explicit imports in `__init__.py` match (no public symbol re-exported without being in `__all__`, and vice versa). -- Status set to `review`. - -## Dev Agent Record - -### Implementation Plan - -Followed TDD per module: write failing tests → confirm collection failure (RED) → implement module → confirm tests pass (GREEN). Execution order Task 5 (test stubs) → Tasks 1+2+3 (impl) → Task 4 (wire `__init__.py` exports) → Task 6 (validate). `ClientConfig` does not depend on `Request`/`Response`, but the modules are small enough that strict parallelism did not buy anything; wrote them sequentially. - -### Debug Log - -- **Forward-reference return types:** Without `from __future__ import annotations`, the class body cannot reference `Request` inside method annotations (the name is not bound until the class is fully built). Resolved by using `typing.Self` for all four `with_*` methods. Works for any subclass too, which is the right semantics. -- **`ruff format` reformatted `"hello".encode()` → `b"hello"`** in `tests/test_response.py` (UP012, the `pre-format` rewrite). Cosmetic; left as ruff produced it. -- **`INP001` (implicit-namespace-package) on every test file** — `tests/` needs an `__init__.py` under this project's `select = ["ALL"]` ruff config (the ignore list does not exempt INP). The story's Task 5.1 explicitly said to skip it; that guidance was wrong against the project's actual lint config. Added an empty `tests/__init__.py`; documented the deviation in the subtask checkbox. No effect on pytest discovery (`pythonpath = ["src"]` already handles src-layout). -- **`PLR2004` (magic-value-in-comparison) on per-field default asserts** in `test_config.py` — fired on every literal in `assert lim.max_connections == 100` etc. Refactored to dataclass-equality form: `assert Limits() == Limits(max_connections=100, max_keepalive_connections=20, keepalive_expiry=5.0)`. The expected values are now constructor args, not bare literals, so PLR2004 doesn't fire — and the assertion is also denser and reads better. -- **`json` method on `Response`** triggered `ANN401` (dynamic-type `Any` in return annotation). Justified — `json.loads` returns `Any` by definition; suppressed with `# noqa: ANN401` on that line only. -- **IDE pyright noise:** the editor's bundled pyright kept flagging `httpware.config`/`httpware.request`/`httpware.response` as unresolved imports even when ty and pytest succeeded. This is an IDE-side caching issue — the project's authoritative type checker is `ty`, which passes clean. No code change required. - -### Completion Notes - -**AC verification — all 8 satisfied:** - -- **AC1** ✓ `Request` in `src/httpware/request.py` is `@dataclass(frozen=True, slots=True)` with the seven fields in spec order: `method: str`, `url: str`, `headers: Mapping[str, str]`, `params: Mapping[str, str]`, `cookies: Mapping[str, str]`, `body: bytes | None`, `extensions: Mapping[str, Any]`. Mapping fields default to `field(default_factory=dict)`; `body` defaults to `None`; `method`/`url` required. -- **AC2** ✓ All four `with_*` methods present, each one line on `dataclasses.replace`. `with_header` builds the new mapping via `{**self.headers, name: value}` (replace-when-present semantics by virtue of dict-update). Return type is `Self` (preserves subclass identity if anyone subclasses `Request` later). -- **AC3** ✓ `Response` in `src/httpware/response.py` is `@dataclass(frozen=True, slots=True)` with the five required fields in order: `status: int`, `headers: Mapping[str, str]`, `content: bytes`, `url: str`, `elapsed: float`. -- **AC4** ✓ `Response.text` is `@property` — case-insensitive header lookup via `_get_content_type` helper, parses `charset=` from the value, falls back to `"utf-8"`. `Response.json()` is a method (not a property) returning `json.loads(self.content)`. Neither is in slots; both are computed each call. -- **AC5** ✓ `Timeout`, `Limits`, `ClientConfig` in `src/httpware/config.py` are all `@dataclass(frozen=True, slots=True)`. Defaults exactly match the spec. `ClientConfig.timeout` and `.limits` use `field(default_factory=Timeout)` / `field(default_factory=Limits)` — never bare instances. `Redactor` is intentionally not present (Story 5.3). -- **AC6** ✓ `src/httpware/__init__.py` re-exports all five via explicit absolute imports; `__all__` is the sorted list `["ClientConfig", "Limits", "Request", "Response", "Timeout"]`. Smoke import succeeds. -- **AC7** ✓ `just lint-ci` reports zero diagnostics across `eof-fixer`, `ruff format --check`, `ruff check --no-fix`, `ty check`. -- **AC8** ✓ 24 tests cover every documented case: frozen on all 5 dataclasses, `with_*` immutability + new-instance identity + replace-semantics on `with_header`, equality, independent default mappings, charset case-insensitivity (parametrized over three casings), UTF-8 / latin-1 / missing-charset paths, JSON round-trip, default constructor returns documented defaults. `pytest` exits 0; coverage 100%. - -**Deviations from the story spec (worth flagging for reviewer):** - -- Added `tests/__init__.py` (story Task 5.1 said skip) — required by `INP001` under `select = ["ALL"]`. Added to File List below. -- Used `Self` for `with_*` return annotations rather than a quoted forward-ref `"Request"` — both are valid; `Self` is the post-PEP-673 idiom and works for any subclass. -- Test_config refactored to use dataclass equality instead of per-field assertions — denser and sidesteps `PLR2004`. Same coverage; same assertions. - -**Tests added:** 24 (8 in `test_request.py`, 9 in `test_response.py`, 7 in `test_config.py`). All pass. Coverage 100% on the three new modules and `__init__.py`. - -## File List - -- `src/httpware/request.py` — new -- `src/httpware/response.py` — new -- `src/httpware/config.py` — new -- `src/httpware/__init__.py` — modified (explicit imports + `__all__`) -- `tests/__init__.py` — new (empty; required by ruff INP001) -- `tests/conftest.py` — new (placeholder) -- `tests/test_request.py` — new -- `tests/test_response.py` — new -- `tests/test_config.py` — new -- `CHANGELOG.md` — modified (Unreleased entry) - -## Change Log - -| Date | Change | Notes | -|---|---|---| -| 2026-05-13 | Story created | Drafted from `docs/epics.md` Story 1.2 and `docs/architecture.md` Decisions 1 and 9; expanded the epics' single multi-clause AC into AC1–AC8 for traceability with tasks/subtasks; added explicit anti-pattern list and scope-limit on `ClientConfig`. | -| 2026-05-13 | Story implemented | All 8 ACs satisfied; 24 tests pass; coverage 100% on `request.py`, `response.py`, `config.py`, `__init__.py`; `just lint-ci` clean. Two deviations from spec (tests/__init__.py + dataclass-equality assertion style) documented in Completion Notes. Status → `review`. | - -## Status - -`done` - -### Review Findings - -_Code review run: 2026-05-13. Reviewers: Blind Hunter, Edge Case Hunter, Acceptance Auditor. Acceptance Auditor reported **no AC violations**. 5 patches applied, 11 deferred, 36 dismissed as noise. Post-patch verification: `just lint-ci` clean, 27 tests pass (+3 new), 100% coverage retained._ - -- [x] [Review][Patch] `Response.text` now wraps `bytes.decode` in `try/except LookupError` and falls back to UTF-8 when the declared charset is unknown. [`src/httpware/response.py:42-46`] — applied -- [x] [Review][Patch] `Response` class docstring now documents `elapsed` as wall-clock seconds from request send to response receipt. [`src/httpware/response.py:29-32`] — applied -- [x] [Review][Patch] Added `test_response_text_strips_quotes_around_charset` (parametrized over `"latin-1"` and `'latin-1'`) covering the quote-stripping branch. [`tests/test_response.py`] — applied -- [x] [Review][Patch] Equality tests now also assert `!=` against per-field variants (Request and Response). [`tests/test_request.py`, `tests/test_response.py`] — applied -- [x] [Review][Patch] Completion Notes AC8 — "frozen on all 4 dataclasses" → "all 5". [`docs/stories/1-2-core-data-types.md:200`] — applied - -- [x] [Review][Defer] Charset parser robustness — whitespace inside quotes, mismatched quotes, multiple `charset=` directives, `charset=` substring inside a quoted boundary param. [`src/httpware/response.py:21-26`] — deferred, pragmatic v0; revisit when transport tests reveal real-world breakage -- [x] [Review][Defer] Header name/value validation (CR/LF injection, `None`, empty string) on `Request.with_header`. [`src/httpware/request.py:21-23`] — deferred to header-handling story (2.3 or later) -- [x] [Review][Defer] URL validation — `with_url("")` accepts empty, `base_url` has no trailing-slash normalization. [`src/httpware/request.py:25-27`, `src/httpware/config.py:27-33`] — deferred, lands when transport composes URLs -- [x] [Review][Defer] `with_query(None)` is currently accepted and breaks downstream iteration. [`src/httpware/request.py:33-35`] — deferred, type system already says `Mapping[str, str]` -- [x] [Review][Defer] `Timeout` / `Limits` accept negative or zero values silently (no `__post_init__`). [`src/httpware/config.py:10-22`] — deferred, validation lands with transport integration -- [x] [Review][Defer] `params: Mapping[str, str]` cannot express multi-valued query strings (`?tag=a&tag=b`). [`src/httpware/request.py:8`] — deferred, type widening tracked separately -- [x] [Review][Defer] `body: bytes | None` precludes streaming / async-iterable bodies. [`src/httpware/request.py:11`] — deferred, intentional minimal scope; revisit in transport stories -- [x] [Review][Defer] No `with_headers` / `with_cookie` / `with_extension` merge helpers; only `with_header` (single) and `with_query` (replace). [`src/httpware/request.py:20-35`] — deferred to Story 2.3 (merge/case-insensitive helpers) -- [x] [Review][Defer] `Response.json()` ignores declared charset; `json.loads(bytes)` auto-detects only UTF-8/16/32 by BOM. [`src/httpware/response.py:44-45`] — deferred, most APIs are UTF-8; revisit if a real backend breaks -- [x] [Review][Defer] `Request.body` / `Response.content` (bytes) render verbatim in the default dataclass `__repr__`, leaking large payloads / secrets into logs. [`src/httpware/request.py`, `src/httpware/response.py`] — deferred to Story 5.3 (`Redactor`) -- [x] [Review][Defer] No `@final` decorator — frozen+slots subclassing is fragile and the spec doesn't bless it; an explicit `@final` would prevent the footgun. [`src/httpware/request.py`, `src/httpware/response.py`, `src/httpware/config.py`] — deferred, no current subclasser diff --git a/docs/archive/stories/1-3-exception-hierarchy-with-plain-fields.md b/docs/archive/stories/1-3-exception-hierarchy-with-plain-fields.md deleted file mode 100644 index 503812b..0000000 --- a/docs/archive/stories/1-3-exception-hierarchy-with-plain-fields.md +++ /dev/null @@ -1,387 +0,0 @@ ---- -story_key: 1-3-exception-hierarchy-with-plain-fields -epic: 1 -story: 3 -title: Exception hierarchy with plain fields -status: done -created: 2026-05-13 -completed: 2026-05-13 -input_documents: - - docs/prd.md - - docs/architecture.md - - docs/epics.md - - docs/stories/1-2-core-data-types.md ---- - -# Story 1.3: Exception hierarchy with plain fields - -## Story - -**As a** consumer developer, -**I want** a status-keyed exception hierarchy with plain typed fields, -**So that** I can catch `NotFoundError` etc. without importing `httpx2` and without inspecting transport types. - -## Acceptance Criteria - -**AC1.** **Given** the `Request`/`Response`/config types from Story 1.2, **When** I implement `src/httpware/errors.py`, **Then** the module defines exactly these classes in this inheritance shape: - -- `ClientError(Exception)` — root of the entire `httpware` exception tree -- `TransportError(ClientError)` -- `TimeoutError(ClientError)` — deliberately shadows `builtins.TimeoutError`; see Dev Notes -- `StatusError(ClientError)` - - `ClientStatusError(StatusError)` - - `BadRequestError(ClientStatusError)` - - `UnauthorizedError(ClientStatusError)` - - `ForbiddenError(ClientStatusError)` - - `NotFoundError(ClientStatusError)` - - `ConflictError(ClientStatusError)` - - `UnprocessableEntityError(ClientStatusError)` - - `RateLimitedError(ClientStatusError)` - - `ServerStatusError(StatusError)` - - `InternalServerError(ServerStatusError)` - - `ServiceUnavailableError(ServerStatusError)` - -**AC2.** **And** `StatusError.__init__` is **keyword-only** (positional arguments raise `TypeError`) and takes exactly these parameters in this order: `status: int`, `body: bytes`, `headers: Mapping[str, str]`, `json: Any | None`, `request_method: str`, `request_url: str`. Each parameter is stored as an instance attribute of the same name and same type. `__init__` calls `super().__init__(...)` with a short summary message (suggested format: `f"{status} {request_method} {request_url}"`) so that `str(exc)` and `repr(Exception)` chaining behave sensibly; the message form is not contractual. - -**AC3.** **And** every `StatusError` subclass (the 9 leaves plus `ClientStatusError` and `ServerStatusError`) inherits `StatusError.__init__` unchanged — leaves do **not** override `__init__`. `issubclass(NotFoundError, ClientStatusError) and issubclass(NotFoundError, StatusError) and issubclass(NotFoundError, ClientError)` is true; `issubclass(InternalServerError, ServerStatusError) and issubclass(InternalServerError, StatusError) and issubclass(InternalServerError, ClientError)` is true. - -**AC4.** **And** the module defines a module-level `STATUS_TO_EXCEPTION: Mapping[int, type[StatusError]]` with **exactly these entries** (no others): - -| status | class | -|---|---| -| 400 | `BadRequestError` | -| 401 | `UnauthorizedError` | -| 403 | `ForbiddenError` | -| 404 | `NotFoundError` | -| 409 | `ConflictError` | -| 422 | `UnprocessableEntityError` | -| 429 | `RateLimitedError` | -| 500 | `InternalServerError` | -| 503 | `ServiceUnavailableError` | - -The dict's declared type annotation is `Mapping[int, type[StatusError]]`. The unknown-status fallback rule — 4xx → `ClientStatusError`, 5xx → `ServerStatusError` — is **documented in the module docstring and the `STATUS_TO_EXCEPTION` docstring/comment**; the resolution logic itself is the caller's responsibility (Story 1.4 inlines it at the transport seam). Story 1.3 ships the dict and the fallback **classes**, not a lookup helper. - -**AC5.** **And** `__repr__` on any `StatusError` instance returns exactly: `f"<{type(self).__name__} status={self.status} method={self.request_method} url={self.request_url}>"`. The class name comes from `type(self).__name__` (so `NotFoundError(...).__repr__()` starts with ` None:` — the bare `*` enforces kwargs-only without naming a stand-in positional. Use `# noqa: A002` if ruff flags the `json` parameter name shadowing the stdlib module (it shouldn't, but be prepared). The `Any` annotation may trigger `ANN401`; suppress narrowly via `# noqa: ANN401` on the same line. - - [x] 2.3: Body of `__init__`: assign all 6 fields to `self.`, then call `super().__init__(f"{status} {request_method} {request_url}")`. Use plain attribute assignment — `StatusError` is **not** a dataclass (Exception subclasses do not play well with `@dataclass(frozen=True)`). - - [x] 2.4: Define `def __repr__(self) -> str:` returning exactly `f"<{type(self).__name__} status={self.status} method={self.request_method} url={self.request_url}>"`. Use `type(self).__name__` so subclasses are formatted correctly. - - [x] 2.5: Do **not** override `__str__`. `Exception.__str__` already returns the message passed to `super().__init__()`. - - [x] 2.6: Annotate the 6 attributes as class-level type hints (so `ty` sees the types without requiring docstring round-trips): - ```python - status: int - body: bytes - headers: Mapping[str, str] - json: Any - request_method: str - request_url: str - ``` - Place them above `__init__` per ruff's preferred ordering. The `json` attribute annotation is `Any` even though the parameter accepts `Any | None` — this avoids `ty` complaining about union-narrowing on access. - -- [x] **Task 3: Implement category bases and 9 leaf classes** (AC1, AC3) - - [x] 3.1: `class ClientStatusError(StatusError):` — one-line docstring "Base for 4xx HTTP status errors."; no body otherwise. - - [x] 3.2: `class ServerStatusError(StatusError):` — one-line docstring "Base for 5xx HTTP status errors."; no body otherwise. - - [x] 3.3: Define the 7 client-side leaves as bare subclasses of `ClientStatusError`, each with a one-line docstring naming the status code: `BadRequestError` (400), `UnauthorizedError` (401), `ForbiddenError` (403), `NotFoundError` (404), `ConflictError` (409), `UnprocessableEntityError` (422), `RateLimitedError` (429). Each body is just `pass` or a docstring. - - [x] 3.4: Define the 2 server-side leaves as bare subclasses of `ServerStatusError`: `InternalServerError` (500), `ServiceUnavailableError` (503). Same shape — docstring only. - - [x] 3.5: Confirm no leaf overrides `__init__` or `__repr__` — they inherit `StatusError`'s machinery unchanged. - -- [x] **Task 4: Define `STATUS_TO_EXCEPTION`** (AC4) - - [x] 4.1: Place after the leaf classes (forward references would otherwise require strings). - - [x] 4.2: Declare with explicit annotation: `STATUS_TO_EXCEPTION: Mapping[int, type[StatusError]] = {…}`. Use a plain dict literal as the runtime value; the `Mapping` annotation advertises read-only intent. Do **not** wrap in `types.MappingProxyType` for v0 (extra ceremony, no real benefit — Story 5.3 will revisit immutability if needed). - - [x] 4.3: Populate with the 9 entries from the AC4 table, in numerical order. Add a single-line comment above the dict noting the fallback rule (unknown 4xx → `ClientStatusError`, unknown 5xx → `ServerStatusError`); do **not** implement the lookup logic in this module. - -- [x] **Task 5: Define `TransportError` and `TimeoutError`** (AC6) - - [x] 5.1: `class TransportError(ClientError):` — one-line docstring "Connection / network / protocol failure raised before a response was received.". Body: `pass` or docstring only. - - [x] 5.2: `class TimeoutError(ClientError):` — one-line docstring "Client-side timeout (connect / read / write / pool).". Body: `pass` or docstring only. **Comment the deliberate shadow of `builtins.TimeoutError`** (one line) so future readers don't "fix" it. - -- [x] **Task 6: Update `src/httpware/__init__.py`** (AC7) - - [x] 6.1: Add explicit absolute-path imports from `httpware.errors`: all 14 exception classes and `STATUS_TO_EXCEPTION`. Single multi-line `from httpware.errors import (...)` block is fine; alphabetize the imported names for diff-friendliness. - - [x] 6.2: Replace `__all__` with the final sorted list (21 entries — see Dev Notes → Public API Surface for the canonical order). Verify alphabetical order character-by-character (Python's `sorted()` is case-sensitive, but every symbol here is `PascalCase` or `UPPER_SNAKE` so simple lex order suffices; `STATUS_TO_EXCEPTION` sorts after `S…` classes — see canonical list below). - - [x] 6.3: Update the module docstring only if needed (no change required — the existing line still applies). - -- [x] **Task 7: Write `tests/test_errors.py`** (AC9) - - [x] 7.1: Imports at top: `pytest`, every exception class under test, `STATUS_TO_EXCEPTION`. Import from `httpware` (top-level), not from `httpware.errors`, to incidentally test AC7's re-export. - - [x] 7.2: Hierarchy tests (AC9.a): one parametrized test over `[(NotFoundError, ClientStatusError, StatusError, ClientError), (InternalServerError, ServerStatusError, StatusError, ClientError), …]` asserting `issubclass(leaf, parent)` for every intermediate parent in the chain. Include a test asserting `issubclass(TransportError, ClientError)` and `issubclass(TimeoutError, ClientError)`. - - [x] 7.3: Kwargs-only enforcement (AC9.b): `with pytest.raises(TypeError): NotFoundError(404, b"", {}, None, "GET", "/x")`. Match `TypeError` message loosely (`pytest.raises(TypeError, match="positional")` is too strict — drop the `match=`). - - [x] 7.4: Field-storage test (AC9.c): construct one `NotFoundError` with all 6 kwargs; assert each `exc. == `. Use a small sample object for `json` (e.g., `{"error": "not found"}`). - - [x] 7.5: `__repr__` exact-match test (AC9.d): two cases — `NotFoundError` 404/`GET`/`/users/1` → exact string `""`; `InternalServerError` 500/`POST`/`/x` → `""`. Build expected strings as f-strings in the test for readability; use `==` not `match=` (we want exact, not regex). - - [x] 7.6: `__repr__` redaction test (AC9.e): construct `NotFoundError(status=404, body=b"secret-token-abc", headers={"Authorization": "Bearer s3cret"}, json=None, request_method="GET", request_url="/x")`; assert each of `"secret-token-abc"`, `"Authorization"`, `"s3cret"` is **not** in `repr(exc)`. Multi-line assert with three separate `assert "…" not in r` statements reads cleaner than a single chained one. - - [x] 7.7: `STATUS_TO_EXCEPTION` mapping test (AC9.f): parametrize over the 9 (code, class) pairs; assert `STATUS_TO_EXCEPTION[code] is cls`. Separate assert: `assert len(STATUS_TO_EXCEPTION) == 9` (catches accidental additions). - - [x] 7.8: Fallback semantics test (AC9.g): `assert STATUS_TO_EXCEPTION.get(418, ClientStatusError) is ClientStatusError` and `assert STATUS_TO_EXCEPTION.get(504, ServerStatusError) is ServerStatusError`. Note: this is *documenting the intended caller idiom*, not testing module logic. - - [x] 7.9: Re-export smoke test (AC9.h): `from httpware import ClientError, StatusError, NotFoundError, STATUS_TO_EXCEPTION, TransportError, TimeoutError`; assert `NotFoundError is httpware.errors.NotFoundError` (import both forms and compare with `is`). - -- [x] **Task 8: Validate locally** (AC8, AC9 — DoD) - - [x] 8.1: `just lint` passes (eof-fixer, ruff format, ruff check, ty check). - - [x] 8.2: `just test tests/test_errors.py` passes; full `just test` passes (no regression on the 24 existing tests from Story 1.2); coverage on `errors.py` is 100% (term-missing output shows no uncovered lines). - - [x] 8.3: `uv run python -c "from httpware import ClientError, StatusError, NotFoundError, STATUS_TO_EXCEPTION; print('ok')"` → `ok`. - - [x] 8.4: `CHANGELOG.md` `Unreleased` → `Added`: one new bullet — e.g., `Status-keyed exception hierarchy with plain typed fields: ClientError, TransportError, TimeoutError, StatusError, ClientStatusError/ServerStatusError bases, 9 leaf classes (BadRequestError … ServiceUnavailableError), STATUS_TO_EXCEPTION lookup dict (Story 1.3).` — append below the existing Story 1.2 bullet, do not reformat existing entries. - - [x] 8.5: Set the `status:` front-matter and the trailing `## Status` block to `review`. Update the File List below to reflect the actually-changed files. - -### Review Findings (2026-05-13) - -Adversarial code review (Blind Hunter + Edge Case Hunter + Acceptance Auditor). Acceptance Auditor verdict: **PASS** — all 9 ACs satisfied, anti-patterns clean, 100% coverage on `errors.py`. The following items surfaced from the Blind / Edge Case layers and need decisions or follow-up. - -**Decisions resolved → patched:** - -- [x] [Review][Decision→Patch] **Pickling broken — Dev Notes intent contradicted.** Resolution: added `_reconstruct_status_error` module-level helper and `StatusError.__reduce__` returning `(callable, (cls, status, body, dict(headers), json, method, url))`. New tests cover `pickle.dumps`/`pickle.loads` and `copy.deepcopy` round-trips with full field/`repr`/`str` equality. _Source: blind+edge_ -- [x] [Review][Decision→Patch] **URL credentials leak in `__repr__` and `str(exc)`.** Resolution: added `_strip_userinfo` helper using `urllib.parse.urlsplit`/`urlunsplit`. `__repr__` and the summary message passed to `Exception.__init__` both run the URL through it, dropping `user:pass@` while preserving scheme, host, port, path, query. Tests cover credential redaction in `repr`/`str`, port preservation, and `@`-in-path no-op. Query-string secrets remain untouched (documented in module docstring; deferred to Story 5.3 `Redactor`). _Source: blind+edge_ -- [x] [Review][Decision→Patch] **`httpware.TimeoutError` does NOT catch `builtins.TimeoutError` / asyncio timeouts.** Resolution: revisited Decision 3 — `TimeoutError` now multi-inherits `(ClientError, builtins.TimeoutError)`. `except builtins.TimeoutError` (asyncio.wait_for's form) and `except OSError` now catch httpware-raised timeouts; a bare `builtins.TimeoutError()` is NOT a `httpware.TimeoutError` (one-way relationship). Two new tests lock both directions. CHANGELOG calls out the architecture Decision 3 revisit. _Source: blind+edge_ -- [x] [Review][Decision→Patch] **Fallback idiom misclassifies 1xx/2xx/3xx.** Resolution: module docstring and the comment above `STATUS_TO_EXCEPTION` now explicitly state the fallback assumes `400 <= status < 600` and that callers must guard non-error codes (1xx/2xx/3xx) before consulting the dict. Story 1.4 inherits this contract. _Source: edge_ -- [x] [Review][Decision→Dismiss] **`headers: Mapping[str, str]` cannot represent multi-valued `Set-Cookie`.** Resolution: accept the v0 contract; revisit only on real consumer complaint. No code change. _Source: edge_ -- [x] [Review][Decision→Patch] **Mutable headers/dict aliasing.** Resolution: `__init__` now wraps headers in `MappingProxyType(dict(headers))`. Two new tests cover defensive-copy semantics (caller mutation does not leak) and read-only semantics (`exc.headers[k] = v` raises `TypeError`). _Source: blind_ -- [x] [Review][Decision→Patch] **Spec self-contradicts: canonical `__all__` order vs RUF022.** Resolution: updated Dev Notes → Public API Surface to publish the RUF022-shipped order (ALL-CAPS first) and documented why (CI is the gate, not `sorted()`). _Source: auditor_ - -**Patches applied:** - -- [x] [Review][Patch] Added `__reduce__` to `StatusError` + pickle/deepcopy round-trip tests. [`src/httpware/errors.py:121-133`, `tests/test_errors.py:250-291`] -- [x] [Review][Patch] Strip userinfo in `__repr__` and `Exception.__init__` summary message + redaction/port/`@`-in-path tests. [`src/httpware/errors.py:25-38, 113, 117-120`, `tests/test_errors.py:172-217`] -- [x] [Review][Patch] Multi-inherit `TimeoutError(ClientError, builtins.TimeoutError)` + isinstance tests in both directions + CHANGELOG note on Decision 3 revisit. [`src/httpware/errors.py:69-79`, `tests/test_errors.py:56-66`, `CHANGELOG.md`] -- [x] [Review][Patch] Module docstring + `STATUS_TO_EXCEPTION` comment scoped fallback to `400 <= status < 600`. [`src/httpware/errors.py:1-16, 174-178`] -- [x] [Review][Patch] Defensive headers copy via `MappingProxyType(dict(headers))` + caller-mutation and read-only tests. [`src/httpware/errors.py:111`, `tests/test_errors.py:106-136`] -- [x] [Review][Patch] Updated Dev Notes canonical `__all__` to RUF022-shipped order. [story file] -- [x] [Review][Patch] Added test for missing-field kwargs rejection (`test_status_error_rejects_missing_kwarg`). [`tests/test_errors.py:74-76`] -- [x] [Review][Patch] Added direct-construction tests for `StatusError`, `ClientStatusError`, `ServerStatusError` (the fallback path Story 1.4 will rely on). [`tests/test_errors.py:220-248`] -- [x] [Review][Patch] Added docstring note in `StatusError.__init__` warning subclasses to call `super().__init__(...)` and documenting the headers defensive copy. [`src/httpware/errors.py:100-107`] - -**Deferred (out of Story 1.3 scope — see `docs/deferred-work.md`):** - -- [x] [Review][Defer] `request_method` / `request_url` CRLF / log-injection — transport seam should validate before crafting the request. [`errors.py:55,58`] — deferred to Story 1.4 review. -- [x] [Review][Defer] Header case-folding / case-sensitivity contract — transport seam concern; how `httpx2.Headers` maps to `Mapping[str, str]`. [`errors.py:33`] — deferred to Story 1.4. -- [x] [Review][Defer] `request_method` casing normalization — transport seam concern; `repr` echoes `"get"` as-is. [`errors.py:35`] — deferred to Story 1.4. - -**Dismissed as noise / handled / out of charter (15 items):** STATUS_TO_EXCEPTION 408/502/504 omissions (anti-pattern); `MappingProxyType` wrapping (anti-pattern); `ClientError` root name (architecture-mandated); `` repr style (AC5-mandated exact format); identity-equality on exceptions (standard behavior); status range / field type runtime validation (anti-pattern, "type checker is the gate"); `# noqa: A001` vs `A004` codes (both correct); `json: Any` vs `Any | None` annotation asymmetry (Dev Notes-documented); `json` parameter / attribute name (architecture-mandated); `len(STATUS_TO_EXCEPTION) == len(_STATUS_MAPPING)` tautology (deliberate strengthening per completion notes); 400–599 key-range assertion (paranoia); CHANGELOG "9 leaf classes" phrasing (accurate — 9 leaves + 6 bases = 15 classes); subclass attribute shadowing (normal Python); `__str__` not tested (spec says message form non-contractual); `from httpware import *` not tested (named-import test (h) already covers the re-export contract). - -## Dev Notes - -### Architecture references (authoritative — read these before coding) - -- `docs/architecture.md` § **Decision 3 — Exception mapping at the seam** (lines 214–223). Story 1.3 ships the **destination** of the mapping (the classes + the dict); the **mapping itself** (httpx2 → httpware) is Story 1.4. Do not import `httpx2` from `errors.py`. -- `docs/architecture.md` § **Exception Construction** (lines 480–499). Authoritative source for: kwargs-only, mandatory 6 fields, `__repr__` format, no bare `Exception` raises. Treat this section as the literal spec. -- `docs/architecture.md` § **Pattern Examples — exception construction with keyword args** (lines 575–597). Shows the caller idiom Story 1.4 will use; informative but not part of this story. -- `docs/prd.md` **FR36–FR40** (lines 630–634). FR37 names every class the AC1 hierarchy must include; FR39 requires plain-typed fields on every exception; FR40 (CancelledError propagation) is out of scope for this story — it lives in the middleware/resilience modules. -- `docs/prd.md` **NFR7–NFR8** (lines 662–663). Default-redacted-header allowlist and "no body in default emissions". The Story 1.3 `__repr__` satisfies NFR8 by never including body or headers; full `Redactor` integration is Story 5.3. -- `docs/epics.md` Story 1.3 (lines 304–319) — authoritative AC source. -- `CLAUDE.md` § Code conventions — exception keyword-only, snake_case methods, `# ty: ignore[…]` only. -- `docs/stories/1-2-core-data-types.md` — same project conventions; this story follows the identical authoring style and test structure (function-style pytest, parametrize for combinatorial assertions, dataclass-equality form where useful). - -### Key design points - -**`ClientError` is the root.** A consumer who wants to catch every framework-raised error writes `except httpware.ClientError`. This includes `TransportError`, `TimeoutError`, and every `StatusError` leaf. Place `ClientError` at the top of `errors.py`. - -**`StatusError.__init__` is keyword-only.** Use a bare-`*` separator: `def __init__(self, *, status, body, …)`. Do **not** use `KW_ONLY` from `dataclasses` — `StatusError` is not a dataclass (Exception + `@dataclass(frozen=True)` is a footgun on Python 3.11 because Exception's own `__init__` writes to `self.args`, which a frozen dataclass forbids; the failure mode is non-obvious). Plain attribute assignment in `__init__` is clearer and dodges the whole problem. - -**Why no dataclass for exceptions.** Exceptions need to be raise-able and `pickle`-able and to interact with `__cause__` / `__context__` cleanly. Frozen dataclasses interfere with all three. A 10-line hand-written `__init__` plus `__repr__` is simpler than fighting the dataclass machinery — and it's what every major Python library does for typed-field exceptions (httpx, anthropic, openai-python). - -**`json` parameter name.** Yes, it shadows the stdlib `json` module locally in the `__init__` signature. The architecture mandates this name (line 495) and the PRD (FR39) reinforces it. If ruff's `A002` (builtin-argument-shadowing) fires, suppress on the function-def line: `# noqa: A002`. Don't rename to `json_` or `body_json`. - -**`Any` annotation for `json`.** The field can hold any JSON-decoded value (dict, list, str, int, float, bool, None) — there's no narrower type. `# noqa: ANN401` may be needed at the parameter-annotation site (matches the precedent set by `Response.json()` in Story 1.2). Storage annotation is `Any` (not `Any | None`) so attribute access doesn't require narrowing — `None` is a valid `Any` value. - -**`__repr__` precise format.** The exact string is `` — note: no quotes around the URL, no comma-after-elements, single space between key=value pairs. The architecture's line 497 is the contract. Test it with `==`, not regex. - -**Why `__repr__` cannot include body/headers.** NFR8 ("no body in default emissions") combined with NFR7 (redaction allowlist) means a raw `body` in `repr` would leak secrets in tracebacks, log lines, and Sentry payloads. The user can still inspect `e.body` / `e.headers` deliberately — which Story 5.3's `Redactor` will guard at emission points. For Story 1.3, the safe path is: `repr` is minimal; raw attributes remain accessible. - -**`TimeoutError` shadows `builtins.TimeoutError`.** Inside `httpware/errors.py` and downstream `from httpware import TimeoutError` consumers, the name resolves to `httpware.errors.TimeoutError`, not the builtin (which is `OSError`-based). This is **deliberate** — Decision 3 demands one client-facing timeout type, and forcing users to write `httpware.TimeoutError` everywhere defeats the ergonomics goal. Add a one-line comment in `errors.py` ("Deliberately shadows builtins.TimeoutError; see Decision 3.") so future readers do not "fix" it. The shadowing is local to the module and to consumer imports; it does not affect `asyncio.TimeoutError` (which `asyncio` raises internally and re-imports from `builtins` in 3.11+). - -**`TransportError`/`TimeoutError` are bare in this story.** The Story 1.3 AC does not specify required fields for them. Story 1.4 (Httpx2Transport) constructs them at the mapping seam and will decide the constructor shape there. **Do not** add a `request_method` / `request_url` / `__init__` to them in this story — that's premature design and breaks the YAGNI rule. The story explicitly defers this (AC6). - -**`STATUS_TO_EXCEPTION` is a plain dict, annotated `Mapping`.** The annotation advertises read-only contract; the runtime value is mutable but no caller mutates it (and Story 5.3 won't either). Skip `MappingProxyType` — the wrapping cost and import noise outweigh the benefit, and the architecture doesn't ask for it. - -**Fallback logic lives at the call site.** The architecture pattern example (line 581) inlines: -```python -exc_class = STATUS_TO_EXCEPTION.get( - resp.status_code, - ClientStatusError if resp.status_code < 500 else ServerStatusError, -) -``` -Story 1.4 will write that line inside `Httpx2Transport.__call__`. **Do not** add a `lookup_status_exception(status: int) -> type[StatusError]` helper to `errors.py` in this story — the AC4 wording ("documented in the module docstring") explicitly puts the fallback rule in prose, not code. A helper would be reasonable in a refactor later; not now. - -**No bare `Exception` raises anywhere.** This is a global rule (architecture line 499). The errors module itself does not `raise`; it only declares classes. The rule applies to dev code that uses these classes. - -### Public API Surface (canonical `__all__` after this story) - -After Story 1.3, `httpware/__init__.py` must re-export exactly these 21 symbols in the order ruff's `RUF022` produces (ALL-CAPS / `UPPER_SNAKE` constants first, then mixed-case classes alphabetically). RUF022 is the CI gate, not Python's `sorted()` — they disagree on uppercase/lowercase ordering. - -```python -__all__ = [ - "STATUS_TO_EXCEPTION", - "BadRequestError", - "ClientConfig", - "ClientError", - "ClientStatusError", - "ConflictError", - "ForbiddenError", - "InternalServerError", - "Limits", - "NotFoundError", - "RateLimitedError", - "Request", - "Response", - "ServerStatusError", - "ServiceUnavailableError", - "StatusError", - "Timeout", - "TimeoutError", - "TransportError", - "UnauthorizedError", - "UnprocessableEntityError", -] -``` - -That's 21 entries: 5 from Story 1.2 (`ClientConfig`, `Limits`, `Request`, `Response`, `Timeout`) + 15 new exception classes + 1 new dict (`STATUS_TO_EXCEPTION`). The CI gate is `ruff check`; trust RUF022's order. (Earlier drafts of this story used `sorted()`'s order, which puts `STATUS_TO_EXCEPTION` between `Response` and `ServerStatusError` — that order fails CI. The list above is the shipped order.) - -### What lives where after this story - -| File | New / modified | Contents | -|---|---|---| -| `src/httpware/errors.py` | **new** | `ClientError` root; `TransportError`/`TimeoutError` bare subclasses; `StatusError` with kwargs-only 6-field `__init__` and `__repr__`; `ClientStatusError`/`ServerStatusError` category bases; 9 leaf classes (`BadRequestError` … `ServiceUnavailableError`); `STATUS_TO_EXCEPTION` dict. | -| `src/httpware/__init__.py` | **modify** | Add `from httpware.errors import (...)` block (15 classes + `STATUS_TO_EXCEPTION`); replace `__all__` with the 21-entry sorted list. | -| `tests/test_errors.py` | **new** | All of AC9 (a)–(h). | -| `CHANGELOG.md` | **modify** | Append one bullet under `Unreleased` → `Added`. | - -### Read-before-edit (per architect's guidance) - -Files this story modifies (read current state before editing): - -- `src/httpware/__init__.py` — currently 8 lines: module docstring, three `from httpware. import …` lines for the Story 1.2 types, blank line, `__all__ = [...]` with the 5 Story 1.2 entries. The story replaces `__all__` (now 21 entries) and adds one more `from httpware.errors import (...)` block above the existing imports (alphabetical by submodule: `config`, `errors`, `request`, `response`). -- `CHANGELOG.md` — `Unreleased` → `Added` already has bullets for Stories 1.1 and 1.2. Append one new bullet at the end of the `Added` list; do not reformat or reorder existing entries. - -Files this story creates (no prior state to preserve): `src/httpware/errors.py`, `tests/test_errors.py`. - -### Carryover from Story 1.2 - -- `from __future__ import annotations` is **forbidden** (architecture invariant; CI gate). Use `Self` (PEP 673) for any return type that references the class — though in this story, no class returns an instance of itself, so `Self` is not needed. -- `tests/__init__.py` already exists (empty; satisfies ruff `INP001`). Do not delete it; do not require a `tests/__init__.py` in this story's task list. -- `--cov=src/httpware` is in `[tool.pytest.ini_options]`; coverage runs by default. -- `asyncio_mode = "auto"` is set. **No async tests in this story** — no transport, no middleware, no client. Pure synchronous class definition + tests. -- `pythonpath = ["src"]` is set; src-layout imports work without further fuss. - -### Anti-patterns to reject (will fail review or CI) - -- ❌ `from __future__ import annotations`. -- ❌ `import httpx2` / `from httpx2 import …` anywhere in `errors.py`. The mapping seam is Story 1.4's job in `transports/httpx2.py`. -- ❌ `@dataclass(frozen=True)` on `StatusError` or any exception class — incompatible with `Exception.__init__`'s `args`-mutation. -- ❌ Positional `StatusError(404, b"", {}, None, "GET", "/x")` construction — must be kwargs-only. -- ❌ Including `body` or `headers` (raw bytes or any header key/value) in `__repr__`. NFR8. -- ❌ Adding `__init__` to `TransportError` or `TimeoutError` in this story (deferred to Story 1.4 — AC6). -- ❌ Adding a `lookup_status_exception(status)` helper, a `from_status()` classmethod, a `to_dict()` method, or any other utility beyond what AC1–AC7 mandate. **Scope discipline.** -- ❌ `Optional[X]` / `Union[X, Y]` — use `X | None` / `X | Y`. -- ❌ `typing.Mapping`, `typing.Dict` — use `collections.abc.Mapping`, `dict`. -- ❌ `# type: ignore`, `# mypy: ignore`, or bare `# noqa` (without a rule code). -- ❌ `print()` anywhere. -- ❌ Renaming the `json` parameter to `json_` or `body_json` to avoid `A002` — suppress instead. -- ❌ Wrapping `STATUS_TO_EXCEPTION` in `types.MappingProxyType`. -- ❌ Adding entries to `STATUS_TO_EXCEPTION` beyond the 9 listed (e.g., 402, 405, 406, 410, 502, 504) — those resolve to the fallback bases by design. -- ❌ Overriding `__init__` or `__repr__` on any leaf class — they inherit from `StatusError` cleanly. - -### Testing standards summary - -- pytest function-style tests; no `unittest.TestCase`. -- Parametrize combinatorial assertions (hierarchy, status-to-class mapping) — keeps the test file dense and readable. -- `pytest.raises(TypeError)` for the kwargs-only check; **no `match=` regex** (the message is interpreter-specific). -- Use `is` not `==` when comparing class objects: `STATUS_TO_EXCEPTION[404] is NotFoundError`. -- Coverage target: 100% on `errors.py` (the module is ~30 lines and every line is reachable from these tests). Coverage on `__init__.py` should remain 100% after the new imports. -- No Hypothesis tests in this story (no concurrency, no property-based invariants to check). -- No async tests in this story. - -### Definition of Done - -- All 9 ACs verified (each AC mapped to at least one test in `tests/test_errors.py`, or to a check in `just lint`). -- All Task/Subtask checkboxes are `[x]`. -- `ruff format`, `ruff check`, `ty check`, `pytest` all pass locally with zero diagnostics. -- File List below is updated to reflect every changed and new file. -- Change Log has a new dated entry. -- `httpware/__init__.py`'s `__all__` and explicit imports are in lockstep (every entry in `__all__` is imported; every imported symbol is in `__all__`). -- Front-matter `status:` and trailing `## Status` are both set to `review`. - -### Open questions / things to flag for reviewer - -- **`STATUS_TO_EXCEPTION` in `__all__`** — Story 1.3 AC7 says "`__all__` lists every exception", which arguably excludes the dict. This story exports it anyway because the architecture (line 533, "Single source of truth: `httpware/__init__.py` defines `__all__` listing every public symbol") treats `__all__` as the full public surface. If the reviewer prefers it left out of `__all__` (importable but not re-advertised), drop the `"STATUS_TO_EXCEPTION"` entry and the `Public API Surface` list shrinks to 20 entries. Flag for review. -- **Parent class of `TransportError` and `TimeoutError`** — the AC lists them without explicit parens. This story interprets them as `(ClientError)` because (a) `except httpware.ClientError` should catch every framework error and (b) FR36 ("framework raises `httpware`-owned exceptions only") is the strongest single signal. If the reviewer prefers them rooted elsewhere (e.g., `(Exception)` directly), it's a one-character change per class. Flag for review. -- **`StatusError` message form** — AC2 says "calls `super().__init__(...)` with a short summary message"; the suggested form `f"{status} {request_method} {request_url}"` is not contractual. If the reviewer prefers e.g. `f"{request_method} {request_url} -> {status}"` or no message at all, change the one line. - -## Change Log - -| Date | Change | Notes | -|---|---|---| -| 2026-05-13 | Story created | Drafted from `docs/epics.md` Story 1.3 and `docs/architecture.md` Decision 3 + Exception Construction section; expanded the epic's single multi-clause AC into AC1–AC9 for traceability with tasks; added canonical `__all__` list, anti-pattern catalog, three explicit open questions for reviewer. | -| 2026-05-13 | Story implemented | All 9 ACs satisfied; 29 new tests in `tests/test_errors.py` (56 total project-wide, all pass); coverage 100% on `errors.py` (and 100% retained on the 4 prior modules); `just lint-ci` clean. Four minor deviations from spec (RUF022 `__all__` order, unused A002 suppression removed, added PLR0913, len-against-parametrize) documented in Completion Notes. Status → `review`. | - -## Dev Agent Record - -### Implementation Plan - -Followed TDD per the story Task order: wrote `tests/test_errors.py` first to lock in AC9 (a)–(h), confirmed `ModuleNotFoundError` on collection (RED), then implemented `src/httpware/errors.py` with `ClientError` root → `TransportError`/`TimeoutError` bare subclasses → `StatusError` with kwargs-only 6-field `__init__` and `__repr__` → `ClientStatusError`/`ServerStatusError` bases → 9 leaf classes → `STATUS_TO_EXCEPTION` dict. Wired `src/httpware/__init__.py` last — added the `from httpware.errors import (...)` block and rebuilt `__all__` to the 21-symbol surface. Validated with `just lint-ci` and full `pytest` regression at each step. - -### Debug Log - -- **`ruff format` collapsed `__repr__` body** from a 3-line f-string concatenation into a single-line f-string. Cosmetic; left as ruff produced it. -- **`A004` (import-shadowing-builtin) fired on `TimeoutError`** in both `src/httpware/__init__.py` and `tests/test_errors.py`. Suppressed with `# noqa: A004` on the import line, per architecture Decision 3 (deliberate shadow of `builtins.TimeoutError`). -- **`A002` (builtin-argument-shadowing) on `json` parameter** in `StatusError.__init__` did **not** fire — `json` is a stdlib module, not a builtin, so A002's scope doesn't catch it. The pre-emptive `# noqa: A002` from the story file flagged as `RUF100` (unused noqa); removed it. -- **`PLR0913` (too-many-arguments)** fired on `StatusError.__init__` because the kwargs-only `*` separator does not exempt the parameters from the count. Suppressed with `# noqa: PLR0913` on the `def` line — the 6 fields are mandated by the spec and not negotiable. -- **`PLR2004` (magic-value-in-comparison) on `assert exc.status == 404`** in field-storage test — refactored to extract inputs to local variables (`status = 404`; `assert exc.status == status`); same coverage, no literals in comparisons. -- **`PLR2004` on `assert len(STATUS_TO_EXCEPTION) == 9`** — replaced with `assert len(STATUS_TO_EXCEPTION) == len(_STATUS_MAPPING)`. This is stronger than the original: any drift between the dict and the parametrize table now fails the test. -- **`ty` complained about positional construction** in `test_status_error_rejects_positional_args` — the call deliberately raises `TypeError` at runtime, so `ty`'s static check (`missing-argument` and `too-many-positional-arguments`) needs both rule codes suppressed: `# ty: ignore[missing-argument, too-many-positional-arguments]`. -- **`RUF022` sorted `__all__`** with `STATUS_TO_EXCEPTION` first (ALL-CAPS constants ahead of mixed-case classes), which differs from the spec's canonical list that placed it between `Response` and `ServerStatusError`. Per the spec's own guidance ("If sorted() produces a different order than what's written above, trust sorted() and update the file"), accepted ruff's ordering — CI is authoritative. -- **IDE pyright noise:** the editor's bundled pyright kept flagging `httpware.errors` as unresolved. Same caching issue documented in Story 1.2's Debug Log — `ty` and `pytest` both pass clean. No code change. - -### Completion Notes - -**AC verification — all 9 satisfied:** - -- **AC1** ✓ `src/httpware/errors.py` defines the exact 15-class hierarchy: `ClientError` root; `TransportError`/`TimeoutError` direct subclasses; `StatusError` with `ClientStatusError`/`ServerStatusError` category bases and 9 leaves (`BadRequestError`/`UnauthorizedError`/`ForbiddenError`/`NotFoundError`/`ConflictError`/`UnprocessableEntityError`/`RateLimitedError` under client; `InternalServerError`/`ServiceUnavailableError` under server). -- **AC2** ✓ `StatusError.__init__` uses a bare-`*` to enforce kwargs-only. Six fields stored in spec order via plain `self.` assignment, then `super().__init__(f"{status} {request_method} {request_url}")`. -- **AC3** ✓ All 9 leaves plus `ClientStatusError`/`ServerStatusError` are bare subclasses (docstring only); no `__init__` or `__repr__` overrides. Verified by `test_leaf_inherits_full_chain` parametrized over 9 leaves. -- **AC4** ✓ `STATUS_TO_EXCEPTION: Mapping[int, type[StatusError]]` declared with the explicit annotation; plain dict literal; 9 entries exactly matching the AC4 table. Fallback rule documented in the module docstring; no lookup helper added (deferred to Story 1.4's transport seam). -- **AC5** ✓ `__repr__` returns `f"<{type(self).__name__} status=... method=... url=...>"` — uses `type(self).__name__` so leaves render their own class name. `test_repr_format_4xx_leaf` and `test_repr_format_5xx_leaf` lock the exact strings; `test_repr_does_not_leak_body_or_headers` asserts NFR8 (body/header content cannot appear in `repr`). -- **AC6** ✓ `TransportError` and `TimeoutError` are bare `(ClientError)` subclasses — no `__init__` override. Story 1.4 will construct them at the `Httpx2Transport` seam. -- **AC7** ✓ `src/httpware/__init__.py` re-exports all 15 new classes plus `STATUS_TO_EXCEPTION` via an explicit absolute-path `from httpware.errors import (...)` block; `__all__` has all 21 symbols sorted per `ruff` RUF022 (ALL-CAPS first). -- **AC8** ✓ `just lint-ci` reports zero diagnostics across eof-fixer / `ruff format --check` / `ruff check --no-fix` / `ty check`. -- **AC9** ✓ `tests/test_errors.py` covers (a) hierarchy parametrized over 9 leaves + `TransportError`/`TimeoutError`, (b) kwargs-only `TypeError`, (c) 6-field storage, (d) `__repr__` exact match for 4xx and 5xx leaves, (e) redaction (body / "Authorization" / value all absent from `repr`), (f) parametrized status-to-class mapping over the 9 entries plus length assertion against the parametrize list, (g) fallback `.get(...)` semantics for both 4xx and 5xx, (h) re-export `is`-identity from `httpware` top-level vs `httpware.errors`. 29 new tests added; 56/56 total pass; coverage 100% on `errors.py` (and 100% retained on the rest). - -**Deviations from the story spec (worth flagging for reviewer):** - -- `__all__` is sorted by ruff RUF022 with `STATUS_TO_EXCEPTION` first (ALL-CAPS constants ahead of mixed-case class names), not in the order documented in Dev Notes → Public API Surface. The spec itself authorized trusting `sorted()` if it differs; this story trusts ruff's RUF022 since it's the CI gate. -- `# noqa: A002` removed from the `json` parameter line — ruff did not actually flag it (A002 doesn't target stdlib-module names), so the pre-emptive suppression was unused (RUF100). The story file flagged this as a possibility ("Use `# noqa: A002` if ruff flags... it shouldn't, but be prepared"). -- Added `# noqa: PLR0913` on `StatusError.__init__` — not in the story file's anti-pattern list, but unavoidable: kwargs-only `*` does not exempt parameters from the argument count. -- `test_status_to_exception_has_only_nine_entries` asserts `== len(_STATUS_MAPPING)` rather than `== 9`. The spec called for catching "accidental additions" — comparing against the parametrize list makes that test self-reinforcing. - -**Tests added:** 29 (in `tests/test_errors.py`). All pass. Coverage 100% on `errors.py`; 100% retained on the 4 prior modules. - -## File List - -- `src/httpware/errors.py` — new -- `src/httpware/__init__.py` — modified (added `from httpware.errors import (...)` block; `__all__` expanded from 5 to 21 entries) -- `tests/test_errors.py` — new -- `CHANGELOG.md` — modified (one new bullet under `Unreleased` → `Added`) - -## Status - -`done` diff --git a/docs/archive/stories/1-4-transport-protocol-and-httpx2transport-adapter.md b/docs/archive/stories/1-4-transport-protocol-and-httpx2transport-adapter.md deleted file mode 100644 index d2ed0d7..0000000 --- a/docs/archive/stories/1-4-transport-protocol-and-httpx2transport-adapter.md +++ /dev/null @@ -1,471 +0,0 @@ ---- -story_key: 1-4-transport-protocol-and-httpx2transport-adapter -epic: 1 -story: 4 -title: Transport protocol and Httpx2Transport adapter -status: review -created: 2026-05-13 -input_documents: - - docs/prd.md - - docs/architecture.md - - docs/epics.md - - docs/stories/1-2-core-data-types.md - - docs/stories/1-3-exception-hierarchy-with-plain-fields.md - - docs/deferred-work.md ---- - -# Story 1.4: Transport protocol and Httpx2Transport adapter - -## Story - -**As a** library author, -**I want** a `Transport` protocol and a default `Httpx2Transport` implementation, -**So that** the entire library talks to one abstraction and `httpx2` is confined to a single file. - -## Acceptance Criteria - -**AC1.** **Given** the data types (Story 1.2) and the exception hierarchy (Story 1.3), **When** I implement `src/httpware/transports/__init__.py`, **Then** the module defines a `Transport` class decorated with `@runtime_checkable` and inheriting `Protocol` (from `typing`), with **exactly** these three method signatures and no other public attributes: - -```python -async def __call__(self, request: Request) -> Response: ... -def stream(self, request: Request) -> AbstractAsyncContextManager[StreamResponse]: ... -async def aclose(self) -> None: ... -``` - -`AbstractAsyncContextManager` is imported from `collections.abc`. The module's `__all__` is `["Transport"]`. The module docstring is one short line describing the protocol's role as the middleware↔transport seam (Seam 1 in the architecture). - -**AC2.** **And** the `StreamResponse` type referenced in `Transport.stream`'s signature is added to `src/httpware/response.py` as a **minimal stub** to unblock typing: a `@dataclass(frozen=True, slots=True)` class with **exactly three public fields** — `status: int`, `headers: Mapping[str, str]`, `url: str` — and no methods. A one-line docstring identifies it as a placeholder for Story 4.1 (full streaming machinery — `_stream`, `_release`, `iter_bytes`, `iter_text`, `iter_lines` — arrives in Epic 4). `StreamResponse` is re-exported from `httpware/__init__.py` and added to `__all__`. - -**AC3.** **And** I implement `src/httpware/transports/httpx2.py` with a class `Httpx2Transport` whose `__init__` is keyword-only and accepts exactly these parameters (each with the default shown): - -```python -def __init__( - self, - *, - client: httpx2.AsyncClient | None = None, - limits: Limits | None = None, - timeout: Timeout | None = None, -) -> None: ... -``` - -If `client` is supplied, `limits` and `timeout` are ignored (and a `ValueError` is raised if either is non-None alongside `client`, so the precedence is explicit, not silent). If `client` is `None`, a default `httpx2.AsyncClient` is constructed **lazily on first use** (event-loop binding — architecture line 749) with `httpx2.Limits(...)` and `httpx2.Timeout(...)` translated from the supplied `Limits` / `Timeout` (or their dataclass defaults if both are `None`). The constructed client is stored on a private `_client` attribute (typed `httpx2.AsyncClient | None`). Story 1.4 uses a "close everything in `aclose()`" policy: the transport closes whatever `_client` references regardless of who created it; the ownership-respecting variant lands in Story 1.7 (architecture Decision 9). Lazy init is guarded by a private `_init_lock: asyncio.Lock` so concurrent first-calls share one client. - -**AC4.** **And** `Httpx2Transport.__call__(request)` translates `Request` → `httpx2.Request` with this mapping: - -| httpware `Request` field | httpx2 `Request` argument | -|---|---| -| `method` | `method` (passed through; **uppercased at the seam** — see AC10) | -| `url` | `url` (passed through unmodified) | -| `headers` | `headers` (passed through as a `dict(request.headers)` copy) | -| `params` | `params` (passed through as a `dict(request.params)` copy) | -| `cookies` | `cookies` (passed through as a `dict(request.cookies)` copy) | -| `body` | `content=` (httpx2's bytes-input argument; passed through as-is, including `None`) | -| `extensions` | `extensions` (passed through as a `dict(request.extensions)` copy if non-empty, else `None`) | - -Then awaits `await client.send(httpx2_request)` (no `stream=`, no `auth=`, no `follow_redirects=` overrides — those land in later stories). The implementation **measures elapsed wall-clock seconds itself** using `time.monotonic()` deltas around the `send` call (rationale: `httpx2.Response.elapsed` is unreliable in `MockTransport` test paths and tied to the response-close lifecycle; portable measurement at the seam is cleaner and matches the `Response.elapsed` contract from Story 1.2). - -**AC5.** **And** on success (response received), `__call__` translates `httpx2.Response` → `httpware.Response` with this mapping (and **does not** call `resp.aclose()` explicitly — `client.send` with the default `stream=False` already buffers `resp.content` into memory): - -```python -Response( - status=resp.status_code, - headers=dict(resp.headers), # httpx2 returns lowercased keys (see AC11) - content=resp.content, - url=str(resp.url), - elapsed=monotonic_elapsed, # measured at the seam, NOT resp.elapsed -) -``` - -**AC6.** **And** if `resp.status_code` is **not in `range(200, 400)`** (i.e., 4xx or 5xx — 1xx and 3xx are treated as successful responses and returned to the caller; see Open Questions for the 3xx rationale), `__call__` raises a `StatusError` subclass instead of returning. The class is resolved via the architecture's documented idiom (architecture line 581): - -```python -exc_class = STATUS_TO_EXCEPTION.get( - resp.status_code, - ClientStatusError if resp.status_code < 500 else ServerStatusError, -) -raise exc_class( - status=resp.status_code, - body=resp.content, - headers=dict(resp.headers), - json=_try_decode_json(resp), - request_method=request.method, # the httpware Request method, uppercased - request_url=request.url, # the httpware Request URL (unredacted; errors.py redacts in __repr__) -) -``` - -A private module-level helper `_try_decode_json(resp: httpx2.Response) -> Any | None` attempts to parse `resp.content` as JSON when the response's `content-type` (case-insensitively located) starts with `application/json`; on success returns the decoded value, on `json.JSONDecodeError` or any non-JSON content type returns `None`. The helper never raises and never inspects body bytes if the content type is wrong. - -**AC7.** **And** `__call__` translates `httpx2` exceptions to `httpware` exceptions at the seam — **no `httpx2` exception is allowed to escape**. The mapping is implemented as exactly three `except` clauses on the `await client.send(...)` call, in this order: - -```python -try: - httpx2_resp = await self._get_client().send(httpx2_req) -except httpx2.TimeoutException as exc: - raise TimeoutError() from exc -except httpx2.HTTPError as exc: - raise TransportError() from exc -except httpx2.InvalidURL as exc: - raise TransportError() from exc -``` - -Rationale & coverage: - -- **Order matters.** `httpx2.TimeoutException` is a subclass of `httpx2.HTTPError` (via `TransportError → RequestError → HTTPError`); catching it first ensures all 4 timeout leaves (`ConnectTimeout`, `ReadTimeout`, `WriteTimeout`, `PoolTimeout`) map to `httpware.TimeoutError`. -- **`httpx2.HTTPError` covers the remaining 8 entries from the architecture's table:** `ConnectError`, `NetworkError`, `ProxyError`, `UnsupportedProtocol`, `ProtocolError`, `RemoteProtocolError`, `LocalProtocolError`, `DecodingError`, plus `TooManyRedirects` and `httpx2.CloseError`/`ReadError`/`WriteError`/`StreamError` — all `HTTPError` descendants. -- **`httpx2.InvalidURL` is a direct `Exception` subclass** in httpx2 — **NOT** under `HTTPError` — so it requires its own except clause. The architecture's mapping table (line 218) listed it under `TransportError` but did not flag the inheritance gap; this story closes it. -- All three clauses use `raise ... from exc` so the original `httpx2` exception remains in `__cause__` for debugging without becoming part of the `httpware` public surface. -- `TimeoutError()` and `TransportError()` are constructed **without** the 6 status fields — those fields belong to `StatusError`. Story 1.3 AC6 made the bare-class shape explicit ("constructed at the `Httpx2Transport` seam in Story 1.4; field requirements for those two are deferred to that story's mapping work"); this story confirms the shape (no fields beyond `args`). - -**AC8.** **And** `Httpx2Transport.stream(request)` exists with the protocol's signature — `def stream(self, request: Request) -> AbstractAsyncContextManager[StreamResponse]` — and its body raises `NotImplementedError("Streaming arrives in Epic 4 (Story 4.1).")` synchronously (i.e., the `NotImplementedError` is raised when `stream(...)` is *called*, before any `async with`). The method is annotated to return `AbstractAsyncContextManager[StreamResponse]` even though it always raises; this keeps the `Transport` protocol's `isinstance` check passing. - -**AC9.** **And** `Httpx2Transport.aclose()` is idempotent and safe whether or not the client was lazily constructed: - -- If `_client` is `None` (lazy default never used): does nothing. -- If `_client` is set and `_owned` is True (default constructed by us): calls `await self._client.aclose()`, then sets `self._client = None`. -- If `_client` is set and `_owned` is False (user supplied via `client=`): calls `await self._client.aclose()` (the architecture's later ref-counting in Story 1.7 governs whether `AsyncClient.__aexit__` calls this at all; at the transport layer we close what we have). -- After `aclose()`, subsequent calls to `aclose()` are no-ops (idempotency); subsequent calls to `__call__` or `stream()` raise `TransportError` (the client is gone — re-entering would silently reconstruct, which is a footgun). - -**AC10.** **And** the **request_method casing** deferred from Story 1.3 is resolved at this seam: `Httpx2Transport.__call__` uppercases `request.method` before passing it to `httpx2.Request(method=...)` AND before storing it on any raised `StatusError`'s `request_method` field. `httpware.Request.method` itself is **NOT** mutated — it remains whatever the caller stored (immutability of `Request` is a Story 1.2 invariant). The uppercase normalization lives only in the seam path. One unit test asserts that `request.method = "get"` produces `exc.request_method == "GET"` on a 404 response. - -**AC11.** **And** the **header case-folding contract** deferred from Story 1.3 is resolved at this seam: `httpx2.Response.headers` returns lowercased keys (`{"content-type": "...", "content-length": "..."}`); `dict(resp.headers)` preserves that lowercasing. The `Response.headers` and `StatusError.headers` produced by this transport therefore use **lowercase ASCII** keys. This story does **not** add a case-insensitive header type (deferred to Story 2.3 or whoever lands header-handling middleware) — `Mapping[str, str]` with lowercase keys is the v0 contract. Documented in the `Httpx2Transport` class docstring. - -**AC12.** **And** the **CRLF / log-injection** concern deferred from Story 1.3 is partially mitigated by httpx2's own URL validation: `httpx2.Request(url="http://x.com/\r\nInjected: yes")` raises `httpx2.InvalidURL` before reaching the wire. We rely on httpx2 here; we do **NOT** add validation in `transports/httpx2.py`. The `request.method` value is uppercased (AC10) and httpx2 validates it ("Invalid HTTP method" `LocalProtocolError`); no further mitigation in this story. This is added to `docs/deferred-work.md` as "method-/URL-validation centralization" for the future `Redactor`/validation story. - -**AC13.** **And** the CI-enforced invariant from the architecture is met: `grep -rE 'import httpx2|from httpx2' src/httpware/` returns matches **only** in `src/httpware/transports/httpx2.py`. No other module — including `transports/__init__.py` — imports `httpx2`. A test in `tests/test_no_httpx2_leakage.py` runs this grep against the source tree and asserts the only match path is `transports/httpx2.py`. - -**AC14.** **And** `src/httpware/__init__.py` re-exports `Transport`, `Httpx2Transport`, and `StreamResponse` via explicit absolute-path imports, and `__all__` is updated to the final 24-entry list (sorted by `ruff` RUF022 — ALL-CAPS first, then mixed-case classes alphabetically). The full expected list is in Dev Notes → Public API Surface. `from httpware import Transport, Httpx2Transport, StreamResponse` succeeds. - -**AC15.** **And** `uv run ty check`, `uv run ruff format --check`, and `uv run ruff check` all pass with zero diagnostics on the new modules and on the modified `__init__.py` / `response.py`. - -**AC16.** **And** unit tests under `tests/test_transports_httpx2.py` cover, at minimum (use `httpx2.MockTransport` to inject responses and exceptions — pass it via `Httpx2Transport(client=httpx2.AsyncClient(transport=httpx2.MockTransport(handler)))`): - -- (a) **Protocol membership:** `isinstance(Httpx2Transport(), Transport) is True` (runtime_checkable check). -- (b) **Success path 200:** issues a GET, returns a `httpware.Response` with the expected status, content, lowercased headers, url, and a non-negative `elapsed`. -- (c) **Status-code mapping — parametrized over 200, 400, 401, 403, 404, 409, 422, 429, 500, 503:** asserts that 200 returns a `Response`; the others raise the precise leaf class (e.g., 404 → `NotFoundError`, 503 → `ServiceUnavailableError`). For each error case, asserts `exc.status == code`, `exc.request_method == "GET"`, and `exc.request_url == `. -- (d) **Unknown-status fallback:** 418 raises `ClientStatusError` (not any leaf); 504 raises `ServerStatusError` (not any leaf). Confirms the architecture's fallback idiom. -- (e) **`_try_decode_json` hits all branches:** JSON content type with valid JSON sets `exc.json` to the decoded value; non-JSON content type sets `exc.json` to `None`; malformed JSON in a JSON-typed response sets `exc.json` to `None` (helper swallows `JSONDecodeError`). -- (f) **Exception mapping — `httpx2.TimeoutException` family:** parametrized over `ConnectTimeout`, `ReadTimeout`, `WriteTimeout`, `PoolTimeout` — each raises `httpware.TimeoutError` from the seam; assert `type(exc) is TimeoutError`, `exc.__cause__` is the original `httpx2.`. -- (g) **Exception mapping — `httpx2.HTTPError` family (representative):** parametrized over `ConnectError`, `NetworkError`, `ProxyError`, `UnsupportedProtocol`, `LocalProtocolError`, `RemoteProtocolError`, `DecodingError`, `TooManyRedirects` — each raises `httpware.TransportError` from the seam; assert `type(exc) is TransportError`, `exc.__cause__` is the original httpx2 exception. -- (h) **Exception mapping — `httpx2.InvalidURL`:** raises `httpware.TransportError`; assert chain via `__cause__`. (Verifies the orphan-class case that's not under `HTTPError`.) -- (i) **No `httpx2` exception escapes:** parametrized over the union of (f)+(g)+(h) — asserts `pytest.raises((TimeoutError, TransportError))` catches every case; no `pytest.raises(httpx2.HTTPError)` test ever fires because the seam catches them all. -- (j) **Method casing normalization:** `Httpx2Transport.__call__(Request(method="get", url=...))` against a 404 mock produces `exc.request_method == "GET"` (uppercased at the seam). -- (k) **`stream()` raises `NotImplementedError` synchronously:** `transport.stream(req)` (without `async with`) raises `NotImplementedError`. -- (l) **`aclose()` is idempotent:** `await transport.aclose(); await transport.aclose()` does not raise; `_client` is `None` afterwards. -- (m) **`aclose()` on a never-used transport** (lazy client never constructed): `await Httpx2Transport().aclose()` is a no-op (does not raise). -- (n) **Post-close call raises:** after `await transport.aclose()`, `await transport(req)` raises `TransportError`. -- (o) **Lazy event-loop binding:** `Httpx2Transport()` constructed outside an event loop does not raise; the underlying `httpx2.AsyncClient` is created on first `__call__` (assert via `transport._client is None` before, non-None after). -- (p) **Constructor argument conflict:** `Httpx2Transport(client=..., limits=Limits())` raises `ValueError`; `Httpx2Transport(client=..., timeout=Timeout())` raises `ValueError`. -- (q) **No-leakage CI grep test** (`tests/test_no_httpx2_leakage.py`): walks `src/httpware/`, asserts that `import httpx2` / `from httpx2` appears **only** in `src/httpware/transports/httpx2.py`. - -`uv run pytest` exits 0; coverage on `src/httpware/transports/httpx2.py` and `src/httpware/transports/__init__.py` is **100%** (the modules are small and every branch is reachable from these tests). - -## Tasks/Subtasks - -- [x] **Task 1: Add minimal `StreamResponse` stub to `src/httpware/response.py`** (AC2) - - [x] 1.1: Append a `@dataclass(frozen=True, slots=True)` class `StreamResponse` after the existing `Response` class. Fields: `status: int`, `headers: Mapping[str, str]`, `url: str`. No methods. One-line docstring: `"""Placeholder for the streaming response type — fleshed out in Story 4.1."""`. - - [x] 1.2: Do **not** modify `Response` or its helpers. Do **not** add `_stream` / `_release` private fields yet (they're Story 4.1's contract surface). - -- [x] **Task 2: Create `src/httpware/transports/__init__.py`** (AC1) - - [x] 2.1: Module docstring: `"""Transport protocol — the middleware ↔ transport seam (Seam 1)."""`. - - [x] 2.2: Imports: `from contextlib import AbstractAsyncContextManager` (deviation from spec — see Completion Notes; spec said `collections.abc` but the symbol is not exported there on Python 3.11/3.12), `from typing import Protocol, runtime_checkable`, `from httpware.request import Request`, `from httpware.response import Response, StreamResponse`. No `httpx2` import — this module is the protocol, not the implementation. - - [x] 2.3: Define `@runtime_checkable` `class Transport(Protocol):` with the three methods exactly as specified in AC1. Method bodies are `...` (Protocol convention). Each method has a one-line docstring. - - [x] 2.4: `__all__ = ["Transport"]`. - -- [x] **Task 3: Create `src/httpware/transports/httpx2.py` — module setup** (AC3) - - [x] 3.1: Module docstring: short line plus second paragraph documenting the lowercase-headers + uppercase-method contracts (AC10, AC11). - - [x] 3.2: Imports: `dataclasses`, `json`, `time`, `contextlib.AbstractAsyncContextManager` (same deviation as Task 2.2), `typing.Any`, `httpx2`, plus the `httpware` config/errors/request/response symbols. `Httpx2Transport` does not import `Transport` — structural subtyping handles it. - - [x] 3.3: `# noqa: A004` applied to the `TimeoutError` line in the multi-line `from httpware.errors import (...)` block. - - [x] 3.4: Only `import httpx2` is used — no `from httpx2 import ...` lines. - -- [x] **Task 4: Implement `Httpx2Transport.__init__` + lazy client construction** (AC3) - - [x] 4.1: Class signature: `class Httpx2Transport:` (no protocol inheritance). - - [x] 4.2: `__init__` kwargs-only via bare `*`. `_client`, `_owned`, `_limits`, `_timeout` set; `ValueError` raised on `client` + (`limits` or `timeout`). - - [x] 4.3: `_closed: bool = False` initialised. - - [x] 4.4: `_get_client()` raises `TransportError` when `_closed`; otherwise lazily constructs `httpx2.AsyncClient(limits=..., timeout=...)` from `Limits()` / `Timeout()` defaults. - -- [x] **Task 5: Implement `Httpx2Transport.__call__` — request translation + send + timing** (AC4, AC5, AC10, AC12) - - [x] 5.1: Signature: `async def __call__(self, request: Request) -> Response`. - - [x] 5.2: `method = request.method.upper()`; `request` is not mutated. - - [x] 5.3: `httpx2.Request(...)` built with defensive `dict(...)` copies; `content=request.body`; `extensions=` only set when non-empty. - - [x] 5.4: `time.monotonic()` measurement straddles `client.send`; passed to `Response`. - - [x] 5.5: Three-clause `except` (`TimeoutException` → `TimeoutError`, `HTTPError` → `TransportError`, `InvalidURL` → `TransportError`), all `from exc`. - - [x] 5.6: `400 <= status < 600` branch uses `STATUS_TO_EXCEPTION.get(status, ClientStatusError if status < 500 else ServerStatusError)`; uppercased `method` stored on `request_method`; `request.url` passed verbatim. - - [x] 5.7: Non-error statuses build and return `Response(status, headers=dict(...), content, url=str(resp.url), elapsed=elapsed)`. - -- [x] **Task 6: Implement `_try_decode_json` helper** (AC6) - - [x] 6.1: Module-level private function with `# noqa: ANN401` on `Any | None` return. - - [x] 6.2: Case-insensitive `content-type` lookup; `lstrip().lower().startswith("application/json")` gate. - - [x] 6.3: `try: json.loads(resp.content)` / `except json.JSONDecodeError: return None` — no broader catch. - - [x] 6.4: Empty-body short-circuit returns `None` before `json.loads`. - -- [x] **Task 7: Implement `Httpx2Transport.stream`** (AC8) - - [x] 7.1: Sync `def stream(self, request: Request) -> AbstractAsyncContextManager[StreamResponse]`. - - [x] 7.2: Body raises `NotImplementedError("Streaming arrives in Epic 4 (Story 4.1).")` synchronously (ruff did not flag the unused parameter so no `# noqa: ARG002` was needed). - -- [x] **Task 8: Implement `Httpx2Transport.aclose`** (AC9) - - [x] 8.1: `async def aclose(self) -> None`; returns immediately if already closed. - - [x] 8.2: Awaits `self._client.aclose()` when present, sets `_client = None`, sets `_closed = True`. - - [x] 8.3: No try/except wrapper around `aclose()`. - -- [x] **Task 9: Update `src/httpware/__init__.py`** (AC14) - - [x] 9.1: `from httpware.response import Response, StreamResponse` extended in place. - - [x] 9.2: `from httpware.transports import Transport` added. - - [x] 9.3: `from httpware.transports.httpx2 import Httpx2Transport` added. - - [x] 9.4: `__all__` expanded to 24 entries; `ruff check` reports zero diagnostics. - -- [x] **Task 10: Write `tests/test_transports_httpx2.py`** (AC16) - - [x] 10.1: Imports cover the public symbols under test; `TimeoutError` uses `# noqa: A004`. - - [x] 10.2: `_make_transport(handler)` helper builds an `Httpx2Transport` wrapping an `httpx2.AsyncClient(transport=httpx2.MockTransport(handler))`. - - [x] 10.3: Cases (a)–(p) implemented with `pytest.mark.parametrize` over the status leaves, 4 timeout classes, and 8 HTTPError descendants. - - [x] 10.4: Case (i) parametrizes over all mapped httpx2 exceptions and asserts `not isinstance(info.value, httpx2.HTTPError)`. - - [x] 10.5: Case (o) tests pre-call `_client is None`, post-call (with MockTransport) `_client is not None`, and a separate test invokes `_get_client()` directly with `Limits()/Timeout()` to exercise lazy construction without a network call. - - [x] 10.6: Case (q) lives in `tests/test_no_httpx2_leakage.py`, parametrized over every `.py` file under `src/httpware/`, asserts the only file that imports `httpx2` is `src/httpware/transports/httpx2.py`. - -- [x] **Task 11: Update `docs/deferred-work.md`** (AC12 — partial) - - [x] 11.1: New "Deferred from: code review of story-1-4 (2026-05-13)" section added with the 3 carryover items (URL CRLF / log-injection deferred to Story 5.3, `request.method` validation, case-insensitive header type). - - [x] 11.2: Story-1-3 section replaced with the one-line "Resolved by Story 1.4" summary. - -- [x] **Task 12: Validate locally + update CHANGELOG** (AC13, AC15, DoD) - - [x] 12.1: `just lint` passes (eof-fixer, ruff format, ruff check --fix, ty check) — zero diagnostics. - - [x] 12.2: `just test` passes (132 tests); coverage 100% on `src/httpware/transports/__init__.py` and `src/httpware/transports/httpx2.py`; no regressions on prior tests. - - [x] 12.3: `grep -rE 'import httpx2|from httpx2' src/httpware/` returns exactly `src/httpware//transports/httpx2.py:import httpx2`. - - [x] 12.4: `uv run python -c "from httpware import Transport, Httpx2Transport, StreamResponse; print('ok')"` → `ok`. - - [x] 12.5: `CHANGELOG.md` `Unreleased` → `Added` appended with one Story-1.4 bullet; prior entries untouched. - - [x] 12.6: Front-matter `status:` and trailing `## Status` set to `review`; File List updated below. - -### Review Findings - -Code review of 2026-05-14 (Blind Hunter + Edge Case Hunter + Acceptance Auditor). 16 findings retained after triage; 15+ dismissed as wrong, intentional, or out-of-scope. All 3 decision-needed items resolved 2026-05-14. - -#### Patch (11) — fix unambiguously - -- [x] [Review][Patch] **Wrap `httpx2.Request(...)` construction inside the try block** — `httpx2.InvalidURL`, `httpx2.CookieConflict`, and `httpx2.LocalProtocolError` raised at Request init escape uncaught, violating the "no `httpx2` exception escapes" invariant. Note: `InvalidURL` and `CookieConflict` are `Exception` (not `HTTPError`) subclasses, so they bypass the existing `except httpx2.HTTPError`. [`src/httpware/transports/httpx2.py:91-99`] -- [x] [Review][Patch] **Map closed-client `RuntimeError` to `TransportError`** — `client.send` on a closed `httpx2.AsyncClient` raises a bare `RuntimeError("Cannot send a request, as the client has been closed.")` which is not caught and escapes. Add `except RuntimeError as exc: raise TransportError(str(exc)) from exc` (or narrow check on message), reachable today via the user-supplied-client lifecycle. [`src/httpware/transports/httpx2.py:101-108`] -- [x] [Review][Patch] **Preserve original exception message when mapping** — `raise TimeoutError from exc` instantiates `TimeoutError()` with no args; `str(exc)` is empty, operators see `httpware.TimeoutError: ` in logs. Replace with `raise TimeoutError(str(exc)) from exc` (and same for `TransportError`). The `__cause__` chain is preserved either way. [`src/httpware/transports/httpx2.py:103-108`] -- [x] [Review][Patch] **Remove dead `_owned` field (or honor it in `aclose()`)** — set at `__init__` line 65, never read; `aclose()` unconditionally closes the underlying client. Spec line 51 incorrectly claims `_owned` is "used by `aclose()`". Story 1.4 deliberately chose the "close everything" policy (spec line 395), so remove `_owned` and reconcile spec line 51 with reality. Alternative: implement ownership-respecting `aclose` (Story 1.7 territory). [`src/httpware/transports/httpx2.py:65,138-145`] -- [x] [Review][Patch] **Tighten `_try_decode_json` content-type match** — `.startswith("application/json")` falsely matches `application/jsonpatch`. Split on `;` and check the bare media type equals `application/json` (deferring `+json` variants per spec Open Question (a)). [`src/httpware/transports/httpx2.py:40`] -- [x] [Review][Patch] **`extensions=dict(request.extensions)` unconditionally** — the `if request.extensions else None` branch silently drops falsy-but-non-empty mapping subclasses and is fragile to intent. Just pass a dict (default `{}` is fine). [`src/httpware/transports/httpx2.py:98`] -- [x] [Review][Patch] **Anchor leakage test to a repo-relative path and assert non-empty** — `_SOURCES = sorted(Path("src/httpware").rglob("*.py"))` resolves to cwd at module import; pytest invoked from a different directory yields zero parametrized cases and silently passes. Anchor to `Path(__file__).resolve().parents[1] / "src/httpware"` and add `assert _SOURCES, "leakage test discovered no source files"`. [`tests/test_no_httpx2_leakage.py:10-11`] -- [x] [Review][Patch] **Move lowercase-headers + uppercase-method contract to `Httpx2Transport` class docstring** — AC11 says "Documented in the `Httpx2Transport` class docstring"; currently lives in the module docstring only. Cosmetic but a literal AC gap. [`src/httpware/transports/httpx2.py:51`] -- [x] [Review][Patch] **Add `_closed` check to `stream()`** *(from decision 1b)* — currently raises `NotImplementedError` unconditionally; AC9 requires post-close calls to raise `TransportError`. Check `self._closed` first; raise `TransportError("Httpx2Transport is closed.")` if closed, else fall through to `NotImplementedError`. Add a test for the post-close case under section (n). [`src/httpware/transports/httpx2.py:133-136`, `tests/test_transports_httpx2.py:326-331`] -- [x] [Review][Patch] **Document multi-valued response header collapse** *(from decision 2a)* — `dict(resp.headers)` drops duplicate-key headers (Set-Cookie, Via, Link). Accept the loss for v0 (consistent with `Mapping[str, str]`); add a `# noqa`-style comment at the call site and extend the `case-insensitive header type` deferred-work entry to mention multi-value collapse alongside case-insensitivity. [`src/httpware/transports/httpx2.py:111`, `docs/deferred-work.md`] -- [x] [Review][Patch] **`asyncio.Lock` around lazy `_get_client()`** *(from decision 3a)* — guard the `self._client is None` check with a lazily-created `asyncio.Lock` (stored on the transport). Double-checked locking: cheap fast path when already initialized, lock only on first init. Update the lazy-init test to also assert no leaks under concurrent first-calls. [`src/httpware/transports/httpx2.py:70-85`, `tests/test_transports_httpx2.py:651-657`] - -#### Defer (5) — appended to `docs/deferred-work.md` - -- [x] [Review][Defer] **Unbounded error body size on `StatusError.body`** — `resp.content` materializes the full body; large 5xx pages stay pinned in memory through exception lifetimes. No size cap. Revisit alongside retry/observability middleware. [`src/httpware/transports/httpx2.py:117-124`] -- [x] [Review][Defer] **`httpx2.StreamError` / `StreamConsumed` family escape uncaught** — these are `RuntimeError` subclasses, not `HTTPError`, so `except httpx2.HTTPError` misses them. Not triggered by default httpx2 config (no redirects, no retries) but reachable via user-supplied clients with retry middleware. Revisit when retry middleware lands. [`src/httpware/transports/httpx2.py:103-108`] -- [x] [Review][Defer] **Header CRLF injection** — `dict(request.headers)` forwards header values verbatim, including embedded `\r\n`. Spec already defers URL CRLF to the Redactor (Story 5.3); extend the deferral to cover headers, which are a strictly larger attack surface. [`src/httpware/transports/httpx2.py:94`] -- [x] [Review][Defer] **Userinfo preserved on `StatusError.request_url` field** — `__repr__` and `Exception.__init__` summary strip `user:pass@`, but the raw field retains credentials. Defense-in-depth strip at construction is the Redactor's job (Story 5.3) per existing errors.py docstring. [`src/httpware/transports/httpx2.py:123`] -- [x] [Review][Defer] **Concurrent `aclose()` ↔ `__call__` races** — no synchronization between in-flight `client.send` and `aclose`. Best case `RuntimeError`; worst case partly-disposed pool. Broader concurrency/lifecycle design; defer to Story 1.7 or retry stories. [`src/httpware/transports/httpx2.py:87-145`] - -## Dev Notes - -### Architecture references (authoritative — read these before coding) - -- `docs/architecture.md` § **Decision 2 — Transport protocol shape** (lines 200–213). Authoritative signature for the protocol; ship this verbatim. -- `docs/architecture.md` § **Decision 3 — Exception mapping at the seam** (lines 214–223). The mapping table the AC7 except clauses implement. **Caveat:** the table lists `InvalidURL` under `TransportError` but doesn't flag that `httpx2.InvalidURL` is NOT a subclass of `httpx2.HTTPError` — this story catches it separately (AC7). Flag for reviewer. -- `docs/architecture.md` § **Pattern Examples — exception construction with keyword args** (lines 577–597). The fallback idiom `STATUS_TO_EXCEPTION.get(status, ClientStatusError if status < 500 else ServerStatusError)` shipped verbatim in AC6. -- `docs/architecture.md` § **Async Naming** (lines 474–478). `aclose()` is the sole `a`-prefix exception; `__call__` is plain async. `stream` is sync `def` returning an async CM (not `async def`). -- `docs/architecture.md` § **Exception Construction** (lines 480–499). Kwargs-only construction with 6 fields — already enforced by Story 1.3's `StatusError.__init__`. -- `docs/architecture.md` § **Cross-cutting concerns** (line 749): "**Event-loop binding** — `transports/httpx2.py` (lazy `httpx2.AsyncClient` creation on first `__call__`)" — AC3 codifies this. -- `docs/architecture.md` § **Seam 4 — Httpx2Transport ↔ httpx2** (lines 709–714). The CI grep invariant from AC13. -- `docs/prd.md` **FR12–FR16** (Transport Layer). FR13 is "Custom `Transport` is pluggable"; FR14–FR16 are signature shape; this story implements them. -- `docs/prd.md` **FR36** ("framework raises `httpware`-owned exceptions only"). AC7's no-leakage invariant. -- `docs/epics.md` **Story 1.4** (lines 321–337) — authoritative AC source; this file expands it. -- `docs/stories/1-3-exception-hierarchy-with-plain-fields.md` — exception classes this story constructs. **Read the Review Findings section** — the seam contract for headers/method-casing/URL-userinfo is partly inherited from there. -- `docs/deferred-work.md` lines 7–9 — the three Story 1.3 deferrals that this story resolves (AC10, AC11, AC12). -- `CLAUDE.md` § Code conventions — kwargs-only exceptions, `Http` is two letters (`Httpx2Transport`, not `HTTPX2Transport`), `# ty: ignore[…]` only. - -### Key design points - -**`Transport` is a structural Protocol, not an ABC.** `@runtime_checkable` enables `isinstance(x, Transport)` for the few places we want to gate on conformance (notably `AsyncClient(transport=...)` in Story 1.7). Cost is acceptable for a 3-method protocol per architecture line 212. **Do not** make `Httpx2Transport` inherit from `Transport` — structural typing handles it, and inheritance creates needless coupling. - -**Lazy `httpx2.AsyncClient` construction.** Architecture line 749 is explicit: the underlying client is built on first `__call__`, not in `__init__`. Reason: an `AsyncClient` constructed before the event loop exists binds to the *wrong* loop and explodes at first use. Tests that construct `Httpx2Transport` in sync `setup` fixtures must not require an event loop. The `_get_client()` indirection isolates this concern. - -**`_owned` is informational, not consequential — yet.** In Story 1.4, `aclose()` closes whatever it has (owned or not). The architecture's Decision 9 ref-counting (Story 1.7) will use `_owned` to decide whether `AsyncClient.__aexit__` may safely call this — i.e., a user-supplied client should NOT be closed by `AsyncClient.__aexit__` if the user might still hold a reference. We carry `_owned` now so Story 1.7 doesn't have to refactor the constructor. - -**Method casing normalization at the seam.** Story 1.3's deferred-work `request_method casing` is resolved by uppercasing in `__call__` immediately before passing to httpx2 AND storing on `StatusError`. The httpware `Request` itself is immutable and unchanged — callers can build `Request(method="get", ...)` and the on-the-wire request will be `GET` while `request.method` (the immutable value) remains `"get"`. This matches HTTP/1.1 RFC 7230 (case-insensitive, but conventionally uppercase) and httpx2's own behavior (httpx2 forwards whatever you give it; it does not uppercase). - -**Header case-folding contract: lowercase wins (v0).** `httpx2.Response.headers` is a case-insensitive multimap; `dict(resp.headers)` returns lowercased keys. The httpware `Response.headers` is `Mapping[str, str]` — the contract is "lowercase ASCII keys" in v0. If/when a real header-handling middleware needs case-preservation, Story 2.3 (or whenever it lands) will introduce a `Headers` type. **Do not** add `Headers` in this story. - -**`request_url` is stored raw on `StatusError`; `errors.py` redacts on `__repr__` / `str()`.** Story 1.3 added `_strip_userinfo` and `__reduce__` machinery; the seam's job is to hand the raw URL to `StatusError(...)` and trust the error type to redact at emission points. **Do not** call `_strip_userinfo` from `transports/httpx2.py` — that would be double-redaction and would break the `pickle` round-trip (the reconstructor expects the raw URL). - -**No `auth`, `follow_redirects`, `stream=` overrides on `client.send`.** Per the architecture's middleware-execution model (Decision 4), auth normalization is a middleware (Story 2.4); redirects are out of scope for v0.1.0 (the architecture treats 3xx as responses that callers must handle); `stream=` is Story 4.1's province. The seam passes only `request` to `send`. - -**`_try_decode_json` is best-effort, lossy, and never raises.** If a response declares `application/json` but the bytes are garbage, `exc.json` is `None`. If a 4xx response declares `text/html` with HTML content, `exc.json` is `None`. The user can always inspect `exc.body` directly for the raw bytes. This matches the architecture's pattern example (line 586): `json=_try_json(resp)` is documented as best-effort. - -**`__call__` has the architecture's "exception-mapping seam" responsibility — and only that.** No retry, no logging, no observability hooks here. Those layers live in `middleware/*` (Epic 2+) and wrap the transport from above. Resist the urge to "add a try/except for a clean error message" — clean error messages are the `StatusError.__repr__` story (Story 1.3), not this one. - -**`stream()` returns a typed AbstractAsyncContextManager but raises NotImplementedError when called.** The protocol contract requires the signature; the implementation deferral is documented in the architecture (Decision 10 → Story 4.1). Raising synchronously (not from inside `__aenter__`) catches the misuse at the call site rather than inside an `async with`, which is friendlier for users debugging an "I tried to stream and got nothing" report. - -### Public API Surface (canonical `__all__` after this story) - -After Story 1.4, `httpware/__init__.py` must re-export exactly these **24** symbols in RUF022 order (ALL-CAPS first, then mixed-case classes alphabetically): - -```python -__all__ = [ - "STATUS_TO_EXCEPTION", - "BadRequestError", - "ClientConfig", - "ClientError", - "ClientStatusError", - "ConflictError", - "ForbiddenError", - "Httpx2Transport", - "InternalServerError", - "Limits", - "NotFoundError", - "RateLimitedError", - "Request", - "Response", - "ServerStatusError", - "ServiceUnavailableError", - "StatusError", - "StreamResponse", - "Timeout", - "TimeoutError", - "Transport", - "TransportError", - "UnauthorizedError", - "UnprocessableEntityError", -] -``` - -That's 24 entries: 21 from Story 1.3 + 3 new (`Httpx2Transport`, `StreamResponse`, `Transport`). The CI gate is `ruff check --select RUF022`; trust ruff's order. (`Transport` sorts between `Timeout`/`TimeoutError` and `TransportError`; `StreamResponse` between `StatusError` and `Timeout`.) - -### What lives where after this story - -| File | New / modified | Contents | -|---|---|---| -| `src/httpware/transports/__init__.py` | **new** | `@runtime_checkable` `Transport(Protocol)` with `__call__`, `stream`, `aclose`. | -| `src/httpware/transports/httpx2.py` | **new** | `Httpx2Transport`: kwargs-only `__init__`, lazy `_get_client`, `__call__` with mapping/timing/status-check, `stream` raising `NotImplementedError`, idempotent `aclose`, `_try_decode_json` helper. | -| `src/httpware/response.py` | **modify** | Append minimal `StreamResponse` (3 public fields, no methods). | -| `src/httpware/__init__.py` | **modify** | Add 3 new re-exports; expand `__all__` from 21 to 24. | -| `tests/test_transports_httpx2.py` | **new** | (a)–(p) from AC16. | -| `tests/test_no_httpx2_leakage.py` | **new** | (q) — the CI-invariant grep test (architecture Seam 4). | -| `docs/deferred-work.md` | **modify** | Mark 3 Story-1-3 items resolved; add 3 new deferrals from this story's review surface. | -| `CHANGELOG.md` | **modify** | One new bullet under `Unreleased` → `Added`. | - -### Read-before-edit (per architect's guidance) - -Files this story modifies (read current state before editing): - -- `src/httpware/response.py` — currently 51 lines: module docstring, `_get_content_type` helper, `_parse_charset` helper, `_CHARSET_PREFIX` constant, `Response` dataclass with `text` property and `json()` method. Append `StreamResponse` after the `Response` class definition; do not reorder or rename anything that exists. -- `src/httpware/__init__.py` — currently 48 lines: module docstring, four `from httpware. import (...)` blocks (config, errors, request, response), one `__all__` with 21 entries. Add 2 new imports (`from httpware.transports import Transport`, `from httpware.transports.httpx2 import Httpx2Transport`), extend the existing `from httpware.response import Response` to also import `StreamResponse`, and replace `__all__` with the 24-entry list. `ruff check --fix` will re-sort everything. -- `docs/deferred-work.md` — has a section "Deferred from: code review of story-1-3 (2026-05-13)" with 3 items at lines 5–9. Mark those 3 items as resolved (replace section content with a one-line "Resolved by Story 1.4" note) and append a new section for this story's review surface. -- `CHANGELOG.md` — `Unreleased` → `Added` has bullets for Stories 1.1, 1.2, 1.3. Append at end; do not reformat existing entries. - -Files this story creates (no prior state to preserve): `src/httpware/transports/__init__.py`, `src/httpware/transports/httpx2.py`, `tests/test_transports_httpx2.py`, `tests/test_no_httpx2_leakage.py`. - -### Carryover from Story 1.3 - -- The 15 exception classes from Story 1.3 are the **destination** of this story's mapping. `StatusError.__init__` is kwargs-only; construct via the architecture's idiom (line 581). -- `TransportError` and `TimeoutError` are bare in Story 1.3 — construct them in this story with **zero arguments**: `raise TransportError() from exc`. No `status` / `body` / etc. on these two classes. -- `_strip_userinfo` redaction lives in `errors.py` — invoked automatically on `__repr__` / `str(exc)` / `Exception.__init__`. **Do not** call it from `transports/httpx2.py`. -- `StatusError.__reduce__` (pickling) round-trips `headers` through `dict(self.headers)` — so the seam can hand in any `Mapping[str, str]` and pickle will work. We pass `dict(resp.headers)`, which is already a plain dict; no concern. -- `from __future__ import annotations` is forbidden; use native PEP 604 unions. -- `asyncio_mode = "auto"` is set — async test functions don't need `@pytest.mark.asyncio`. Heavy use of async tests in this story; pytest-asyncio handles it. -- `pythonpath = ["src"]` is set; `from httpware.transports import ...` works. - -### Anti-patterns to reject (will fail review or CI) - -- ❌ `import httpx2` anywhere outside `src/httpware/transports/httpx2.py`. CI grep test (AC13/AC16-q) is the gate. -- ❌ `from __future__ import annotations` anywhere. -- ❌ Making `Httpx2Transport` inherit from `Transport`. Structural subtyping via `@runtime_checkable` is the design. -- ❌ Catching `httpx2.HTTPError` first (before `TimeoutException`) — would misroute timeouts to `TransportError`. -- ❌ Omitting the `httpx2.InvalidURL` except clause — it's not under `HTTPError` and would escape. -- ❌ Letting a `httpx2.*` exception escape `__call__` via any code path (validated by AC16-i). -- ❌ Calling `_strip_userinfo` from the transport — that's the error type's contract, not the seam's. -- ❌ Calling `resp.aclose()` after `await client.send(...)` — `send` with default `stream=False` already buffers `resp.content`, and calling `aclose()` would discard timing information for nothing. -- ❌ Reading `resp.elapsed` — measure it ourselves with `time.monotonic()` (httpx2's `elapsed` is set lazily by the response-close lifecycle and is unreliable in `MockTransport` test paths). -- ❌ Raising `Exception` / `RuntimeError` / `ValueError` for transport-level failures (except the `ValueError` for constructor argument conflicts, which is a programming error and not a runtime transport failure). -- ❌ Adding retry / observability / logging logic inside `Httpx2Transport`. Those are middleware concerns (Epic 2+). -- ❌ Mutating `request` (it's a frozen dataclass; would raise). Use locals (`method = request.method.upper()`). -- ❌ Passing `body=request.body` to `httpx2.Request` — the parameter is `content=`, not `body=`. `body=` does not exist on `httpx2.Request`. -- ❌ Importing `Transport` from `httpware.transports` inside `httpx2.py` — unnecessary; structural typing handles the conformance check. -- ❌ `Optional[X]` / `Union[X, Y]` — use `X | None` / `X | Y`. -- ❌ `typing.Mapping`, `typing.AbstractContextManager` — use `collections.abc` versions. -- ❌ Adding `__init__.py` files to `tests/` subdirectories beyond what already exists (`tests/__init__.py` is fine — keep it). -- ❌ Using `respx` for transport-level mocking. The architecture forbids it in httpware's own tests (line 553); use `httpx2.MockTransport` instead. - -### Testing standards summary - -- `pytest-asyncio` auto mode — async test functions don't need `@pytest.mark.asyncio`. -- `httpx2.MockTransport` for response and exception injection; **no `respx`** (architecture line 553). -- Parametrize aggressively over the 10 status codes (AC16-c), the 4 timeout classes (AC16-f), the 8 HTTPError descendants (AC16-g). Keeps the test file dense. -- Use `pytest.raises(httpware.TimeoutError)` / `pytest.raises(httpware.TransportError)` — exact-type assertions (`type(exc) is TimeoutError`) to catch any future subclass drift. -- Assert `exc.__cause__` is the original httpx2 exception in each mapping test — locks the `raise ... from exc` invariant. -- Coverage target: **100%** on both new transport modules (`__init__.py` and `httpx2.py`). The modules are small (~80 LOC combined); every branch is reachable. -- No Hypothesis tests in this story (no concurrency, no property-based invariants — those land in retry-budget territory, Epic 3). -- No real-network tests in this story — every test uses `MockTransport`. Integration tests against `httpbingo.org` arrive in Story 1.7. -- `tests/test_no_httpx2_leakage.py` is a project-invariant test, not transport-specific — keep it short and explicit. - -### Definition of Done - -- All 16 ACs verified (each AC mapped to at least one test in `tests/test_transports_httpx2.py` / `tests/test_no_httpx2_leakage.py`, or to a check in `just lint`). -- All Task/Subtask checkboxes are `[x]`. -- `ruff format`, `ruff check`, `ty check`, `pytest` all pass locally with zero diagnostics. -- Coverage 100% on `src/httpware/transports/__init__.py` AND `src/httpware/transports/httpx2.py`; 100% retained on the 4 prior modules. -- `grep -rE 'import httpx2|from httpx2' src/httpware/` returns exactly one path: `src/httpware/transports/httpx2.py`. -- File List below is updated to reflect every changed and new file. -- `CHANGELOG.md` has a new dated bullet under `Unreleased` → `Added`. -- `docs/deferred-work.md` reflects the 3 resolved items (from story-1-3 review) and the 3 new deferrals (from this story's anticipated review). -- `httpware/__init__.py`'s `__all__` and explicit imports are in lockstep (every entry in `__all__` is imported; every imported symbol is in `__all__`); RUF022 is clean. -- Front-matter `status:` and trailing `## Status` are both set to `review`. - -### Open questions / things to flag for reviewer - -- **3xx handling.** AC6 treats 3xx as successful responses (returned to the caller as a `Response`), not as errors. Rationale: `httpx2.AsyncClient` has `follow_redirects=False` by default; a 3xx coming through means the caller wants to inspect it. The architecture does not explicitly say "treat 3xx as success", but the fallback rule in `errors.py` ("Fallback assumes `400 <= status < 600`") implies it. If the reviewer prefers raising `ClientStatusError` for 3xx (or auto-following redirects via httpx2's `follow_redirects=True`), that's a one-line change to the status range check and a constructor change. Flag for review. -- **`Httpx2Transport.__init__` accepting `client + limits/timeout`.** AC3 raises `ValueError` if both are non-None (silent precedence is a footgun). The alternative — silently ignoring `limits`/`timeout` when `client` is supplied — is what httpx itself does in some cases, but the strict-error path is friendlier in debugging. If the reviewer prefers silent precedence, that's a one-line change. -- **`aclose()` closing a user-supplied client.** AC9 says we close whatever we have, owned or not. Architecture Decision 9 (Story 1.7) introduces ref-counting that may want a different policy — specifically, "don't close what we don't own". For Story 1.4, the simple policy is "close everything"; Story 1.7 can refine. Flag if reviewer wants the more conservative policy now. -- **`_try_decode_json` accepts `application/json` prefix only, not `+json` suffixes.** Real-world APIs sometimes return `application/vnd.api+json` or `application/problem+json` (RFC 7807). AC6 deliberately scopes to the strict prefix to avoid surprises; the architecture doesn't prescribe a richer match. If the reviewer wants `+json` support, it's a one-line widening. Flag. -- **`Httpx2Transport()` with no args and no `client=`** is constructible, but actually-issuing requests will try to make a real `httpx2.AsyncClient` and hit the network on a real `await transport(...)`. The "lazy" property is tested by checking `_client is None` pre-call; the post-call non-None case requires a `MockTransport`-backed instance to avoid real network. AC16-o has a slightly awkward shape because of this — flag if the reviewer wants a cleaner test arrangement (e.g., requiring `client=` explicitly in v0). -- **`httpx2.InvalidURL` is the architectural mapping table's hidden gotcha** (line 218 lists it but doesn't note the inheritance gap). The architecture document should be amended; flag during review whether to land an architecture patch in the same PR. - -## Change Log - -| Date | Change | Notes | -|---|---|---| -| 2026-05-13 | Story created | Drafted from `docs/epics.md` Story 1.4 + `docs/architecture.md` Decision 2/3 + `transports/httpx2.py` placement; expanded the epic's multi-clause AC into AC1–AC16 for traceability; verified `httpx2` exception hierarchy and `MockTransport` shape against installed `httpx2==2.0.0`; flagged `httpx2.InvalidURL` as not-under-HTTPError (closes a gap in the architecture's mapping table); resolved 3 Story-1-3 deferrals (request_method casing AC10, header case-folding AC11, CRLF mitigation strategy AC12); added open questions on 3xx handling, constructor argument conflict, aclose policy, `+json` suffix matching, and lazy-construction test ergonomics. | -| 2026-05-13 | Story implemented | All 12 tasks (32 subtasks) complete; 16 ACs verified. Two new modules under `src/httpware/transports/`, `StreamResponse` stub added, public `__all__` expanded 21 → 24. One deviation from spec: `AbstractAsyncContextManager` imported from `contextlib` rather than `collections.abc` because the symbol is only re-exported from `collections.abc` on Python ≥3.12 and the project floor is 3.11. 62 new tests added (`tests/test_transports_httpx2.py` 55 + `tests/test_no_httpx2_leakage.py` 7); full suite 132 passing; 100% coverage on every source module; `grep` invariant holds (single `httpx2` import in the library). 3 Story 1.3 deferrals resolved, 3 new Story 1.4 deferrals recorded. Status → `review`. | -| 2026-05-14 | Code review + patches | 3-layer adversarial review (Blind Hunter, Edge Case Hunter, Acceptance Auditor) → 16 retained findings (3 decision-needed, 8 patch, 5 defer), ~28 dismissed. Decisions resolved: (1b) `stream()` honors AC9 post-close → `TransportError`; (2a) accept multi-valued header collapse for v0, document; (3a) `asyncio.Lock` around lazy init. All 11 patches applied to `transports/httpx2.py`, `tests/test_transports_httpx2.py` (+8 new tests, 140 total), `tests/test_no_httpx2_leakage.py` (cwd-robust). 5 deferrals appended to `docs/deferred-work.md`. Full suite 140 passing, 100% coverage holds, lint green. Status → `done`. | - -## Dev Agent Record - -### Implementation Plan - -Implemented in the order specified by the story tasks: `StreamResponse` stub (Task 1) → `Transport` protocol (Task 2) → `Httpx2Transport` module + class (Tasks 3–8) → public `__init__.py` re-exports (Task 9) → tests (Task 10) → docs/changelog/story metadata (Tasks 11–12). Each task was finished and verified (lint + tests) before advancing. - -### Debug Log - -- `from collections.abc import AbstractAsyncContextManager` (specified in AC1, Task 2.2, and Task 3.2) raises `ImportError` on the project's Python 3.11 floor — the symbol was only added to `collections.abc` in Python 3.12. Switched to `from contextlib import AbstractAsyncContextManager` in both `transports/__init__.py` and `transports/httpx2.py`. This is a deviation from the story spec but is required for the project's stated Python floor; surface in the architecture review (the spec carried this assumption through from earlier drafts). Code behaves identically — `contextlib.AbstractAsyncContextManager` is the canonical home and the symbol is re-exported from `collections.abc` only on ≥3.12. -- Initial lint pass surfaced `ANN202` (missing return type annotation) on the inner-handler factory helpers in `tests/test_transports_httpx2.py` and `PLR2004` on a handful of literal status comparisons in test assertions; both addressed by typing the helpers as returning `Callable[[httpx2.Request], httpx2.Response]` and adding targeted `# noqa: PLR2004` on the assertion lines (test-only ergonomic noise). -- Ruff auto-fixed `raise TimeoutError() from exc` → `raise TimeoutError from exc` (TRY302/equivalent). Semantics unchanged for zero-arg construction; left as ruff produced. -- `httpx2.Response.elapsed` raises `RuntimeError` before the response is closed; confirms AC4's rationale for `time.monotonic()`-based timing at the seam. - -### Completion Notes - -- All 16 ACs satisfied. Each AC maps to at least one parametrized or explicit test in `tests/test_transports_httpx2.py` / `tests/test_no_httpx2_leakage.py`, or to a `just lint` / grep guard. -- Single deviation from the story spec: `AbstractAsyncContextManager` is imported from `contextlib` instead of `collections.abc` (see Debug Log). All other AC text honoured verbatim. -- 132 tests pass (70 pre-existing + 62 new). Coverage is **100% on every source module**, including the two new transport modules. -- `grep -rE 'import httpx2|from httpx2' src/httpware/` returns a single match in `src/httpware/transports/httpx2.py` — Seam 4 invariant holds. -- `from httpware import Transport, Httpx2Transport, StreamResponse` succeeds; `__all__` is now 24 entries (RUF022-clean). -- Resolved the 3 Story 1.3 deferrals (method casing AC10, header case-folding AC11, CRLF mitigation strategy AC12); recorded 3 new deferrals from this story (URL CRLF / log-injection → Story 5.3, `request.method` validation, case-insensitive header type). -- Open questions from the story remain open for the reviewer: 3xx handling, `client + limits/timeout` `ValueError` policy, `aclose()` closing a user-supplied client, `_try_decode_json` accepting `+json` suffixes, and `Httpx2Transport()` lazy-construct-then-real-network ergonomics — all noted under "Open questions" in Dev Notes. - -## File List - -- `src/httpware/transports/__init__.py` — new (Transport protocol; Seam 1) -- `src/httpware/transports/httpx2.py` — new (Httpx2Transport adapter; only file importing httpx2; Seam 4) -- `src/httpware/response.py` — modified (appended minimal `StreamResponse` stub) -- `src/httpware/__init__.py` — modified (added `Transport`, `Httpx2Transport`, `StreamResponse`; `__all__` expanded 21 → 24) -- `tests/test_transports_httpx2.py` — new (AC16 cases a–p, 55 parametrized + explicit tests) -- `tests/test_no_httpx2_leakage.py` — new (AC16-q project-invariant grep guard) -- `docs/deferred-work.md` — modified (Story 1.3 deferrals collapsed to "Resolved by Story 1.4"; 3 new Story 1.4 deferrals added) -- `CHANGELOG.md` — modified (one new bullet under `Unreleased` → `Added`) -- `docs/stories/1-4-transport-protocol-and-httpx2transport-adapter.md` — modified (status → `review`; checkboxes; Dev Agent Record / File List / Change Log filled) - -## Status - -`done` diff --git a/docs/archive/stories/1-5-responsedecoder-protocol-and-pydantic-adapter.md b/docs/archive/stories/1-5-responsedecoder-protocol-and-pydantic-adapter.md deleted file mode 100644 index dfb5549..0000000 --- a/docs/archive/stories/1-5-responsedecoder-protocol-and-pydantic-adapter.md +++ /dev/null @@ -1,407 +0,0 @@ ---- -story_key: 1-5-responsedecoder-protocol-and-pydantic-adapter -epic: 1 -story: 5 -title: ResponseDecoder protocol and pydantic adapter -status: done -created: 2026-05-14 -input_documents: - - docs/prd.md - - docs/architecture.md - - docs/epics.md - - docs/stories/1-2-core-data-types.md - - docs/stories/1-3-exception-hierarchy-with-plain-fields.md - - docs/stories/1-4-transport-protocol-and-httpx2transport-adapter.md - - docs/deferred-work.md ---- - -# Story 1.5: ResponseDecoder protocol and pydantic adapter - -## Status - -`done` - -## Story - -**As a** consumer developer, -**I want** to decode response bodies into pydantic models in a single parse pass, -**So that** `response_model=User` returns a typed `User` with minimal overhead. - -## Acceptance Criteria - -**AC1.** **Given** the `Response` type from Story 1.2 and the project scaffold, **When** I implement `src/httpware/decoders/__init__.py`, **Then** the module defines a `ResponseDecoder` class decorated with `@runtime_checkable` and inheriting `Protocol` (from `typing`), with **exactly** this one method signature and no other public attributes: - -```python -def decode(self, content: bytes, model: type[T]) -> T: ... -``` - -`T` is a module-level `TypeVar("T")`. The protocol is NOT parameterized at the class level (`Protocol`, not `Protocol[T]`) — `T` is scoped per-call so a single decoder instance decodes into many model types. `Protocol` and `runtime_checkable` are imported from `typing`. The module's `__all__` is `["ResponseDecoder"]`. The module docstring is one short line identifying the protocol as Seam 3 (`AsyncClient ↔ ResponseDecoder`) per architecture lines 703–708. - -**AC2.** **And** I implement `src/httpware/decoders/pydantic.py` with a class `PydanticDecoder` whose `__init__` takes no arguments (defining `__init__` explicitly is optional — the implicit `object.__init__` is acceptable). Instances are stateless: all callable state is in the module-level cache (AC3). `PydanticDecoder` **does not** inherit from `ResponseDecoder` — structural subtyping via `@runtime_checkable` handles conformance (same pattern Story 1.4 used for `Httpx2Transport`; architecture line 282). The module's `__all__` is `["PydanticDecoder"]`. The module docstring is one short line. - -**AC3.** **And** `src/httpware/decoders/pydantic.py` defines a **module-level** function `_get_adapter` with this signature and decoration: - -```python -@functools.lru_cache(maxsize=1024) -def _get_adapter(model: type[T]) -> pydantic.TypeAdapter[T]: - return pydantic.TypeAdapter(model) -``` - -The cache is module-scope (shared across all `PydanticDecoder` instances in the process). `model` is the cache key; per pydantic's reuse guidance ("create a `TypeAdapter` for a given type just once and reuse it"; https://docs.pydantic.dev/latest/concepts/performance/#avoid-creating-instances-of-typeadapter-in-loops). `maxsize=1024` is a soft upper bound that protects long-running services from unbounded growth if a consumer dynamically generates models (e.g., `pydantic.create_model(...)` per request — see Open Questions item 3). The bound is far above any realistic static schema universe. `_get_adapter` is private (leading underscore) and NOT in `__all__`. - -**Review-driven amendment (2026-05-14):** AC3 originally specified `maxsize=None`. Relaxed to `maxsize=1024` per review Decision 1, resolving Open Questions item 3. - -**AC4.** **And** `PydanticDecoder.decode(self, content, model)` has exactly this body: - -```python -try: - adapter = _get_adapter(model) -except TypeError: - adapter = TypeAdapter(model) -return adapter.validate_json(content) -``` - -The `try/except TypeError` fallback handles unhashable `model` arguments (e.g., `Annotated[int, unhashable_metadata]`) — without it, `functools.lru_cache`'s hash step would leak a `TypeError` from a `functools`-internal frame, breaking the "validation errors surface as `pydantic.ValidationError`" contract. The fallback bypasses the cache for that single call only; cached-path performance for the common hashable case is unaffected. No intermediate `json.loads`, no `validate_python`, no `BaseModel.model_validate_json` — a single `validate_json(bytes)` call (NFR3: single parse pass). The method does not catch `pydantic.ValidationError` — it surfaces unchanged to the caller. The method does not validate `content` is `bytes` at runtime. - -**Review-driven amendment (2026-05-14):** AC4 originally required `return _get_adapter(model).validate_json(content)` as the only body. The `try/except TypeError` fallback was added per review Decision 5. - -**AC5.** **And** `pydantic` is imported at the top of `src/httpware/decoders/pydantic.py` as a plain top-level import — **no `try/except ImportError`** is needed because `pydantic>=2.0,<3.0` is a **base** dependency (`pyproject.toml` line 32), not an optional extra. The architecture's optional-extra import pattern (lines 509–530) applies only to extras-gated modules (msgspec → Story 1.6, otel → Epic 5). Importing the third-party `pydantic` package from a module named `pydantic.py` is unambiguous in Python 3 (absolute imports walk `sys.path`, never the current package); use `from pydantic import TypeAdapter` for readability. - -**AC6.** **And** I extend `src/httpware/__init__.py` to re-export `ResponseDecoder` and `PydanticDecoder` from their respective submodules, growing `__all__` from 24 entries (Story 1.4 close-out) to **26** entries. The RUF022-sorted order places `PydanticDecoder` between `NotFoundError` and `RateLimitedError`, and `ResponseDecoder` between `Response` and `ServerStatusError`. Trust `ruff check --select RUF022 --fix` to produce the canonical order. - -**AC7.** **And** unit tests in `tests/test_decoders_pydantic.py` verify successful decode for the five type categories the epic enumerates (each a separate test or a parametrized case): - -| Target type | Sample input bytes | Expected return | -|---|---|---| -| `pydantic.BaseModel` subclass | `b'{"id": 1, "name": "Ada"}'` | instance of the model with `.id == 1`, `.name == "Ada"` | -| stdlib `@dataclasses.dataclass` | `b'{"id": 1, "name": "Ada"}'` | dataclass instance with `id == 1`, `name == "Ada"` | -| `list[User]` (User = BaseModel) | `b'[{"id": 1, "name": "Ada"}, {"id": 2, "name": "Bo"}]'` | `list` of length 2, both `User` instances | -| `dict[str, User]` | `b'{"u1": {"id": 1, "name": "Ada"}}'` | `dict` with one `User` value | -| primitive `int` | `b'42'` | `42` (`type(result) is int`) | - -For each, assert exact type identity and field equality. Use plain stdlib `@dataclasses.dataclass` (frozen=False is fine; pydantic accepts both `dataclasses.dataclass` and `pydantic.dataclasses.dataclass`). - -**AC8.** **And** a cache-invariance test verifies the NFR2 contract: `_get_adapter` constructs **exactly one** `pydantic.TypeAdapter` instance per `model` argument across 1000 calls with the same `model`. Implementation guidance: monkeypatch (or `unittest.mock.patch`) `pydantic.TypeAdapter` (or `httpware.decoders.pydantic.pydantic.TypeAdapter` — whichever import shape was used) inside the test, clear the cache with `_get_adapter.cache_clear()` in test setup, perform 1000 `PydanticDecoder().decode(...)` calls (or 1000 direct `_get_adapter(SameModel)` calls), and assert the patched constructor was called exactly once. Also assert that two distinct model types in the same loop trigger exactly two constructor calls. - -**AC9.** **And** a benchmark test in `tests/test_decoders_pydantic_bench.py` confirms the NFR3 contract: `PydanticDecoder().decode(content, User)` is **≥1.5× faster** than `pydantic.TypeAdapter(User).validate_python(json.loads(content))` on a 5KB JSON payload, measured via a median-of-60-rounds `time.perf_counter_ns` loop with GC disabled. The benchmark fixture builds a 5KB payload (`4500 <= len(payload) <= 5500`). The ratio is asserted in the test (`SPEEDUP_FLOOR = 1.5`) so a regression fails CI. The assertive timing test carries `@pytest.mark.perf` and is skipped by default `pytest`; run it with `pytest -m perf` (the `-m 'not perf'` exclusion is in `tool.pytest.ini_options.addopts`). Two complementary `@pytest.mark.benchmark` cases (`test_bench_single_pass_*`, `test_bench_two_pass_*`) remain in the default suite via pytest-benchmark. - -**Review-driven amendment (2026-05-14):** AC9 originally required ≥2×. Relaxed to ≥1.5× per review Decision 2, resolving Open Questions item 5 (observed median ~1.63× on Apple Silicon / pydantic-core 2.46.4; ≥2× not hardware-portable). The assertive timing test was gated behind the `perf` marker per Decision 3 to keep `just test` runs CI-noise-resilient. - -**AC10.** **And** invalid input surfaces `pydantic.ValidationError` unchanged: a test passes malformed JSON (e.g., `b'{"id": "not-a-number"}'` for a `User(id: int, name: str)` model) and asserts `pytest.raises(pydantic.ValidationError)`. **No** httpware-owned exception wraps this — the decoder's contract is "the caller chose the model; the caller catches validation errors". (See Open Questions for the framework-policy tension with FR36.) - -**AC11.** **And** the cache survives concurrent first-calls without constructing extra `TypeAdapter` instances. `functools.lru_cache` is thread-safe (the underlying hashmap is protected by the GIL for ops the cache uses); document this in the test rationale rather than adding an explicit lock. A single async test that schedules 50 concurrent `PydanticDecoder().decode(...)` coroutines against a freshly-cleared cache and asserts the patched `TypeAdapter` constructor is called exactly once suffices. - -**AC12.** **And** `ty check` passes with zero diagnostics. The protocol's generic typing is achievable on Python 3.11 (the project floor): `T = TypeVar("T")` at module scope, `def decode(self, content: bytes, model: type[T]) -> T: ...` on the protocol, and the same signature on `PydanticDecoder.decode`. If `ty` rejects the `Protocol` + per-method `TypeVar` combination, fall back to declaring `class ResponseDecoder(Protocol)` with `decode` as a `Generic[T]`-style method; do **not** make the protocol class-level generic (`Protocol[T]`) — that breaks "one decoder, many models". Confirmation: `ty` on the existing `Transport` protocol (Story 1.4) accepts the structurally-identical signature shape; this AC is a continuation of that pattern. - -**AC13.** **And** `ruff format`, `ruff check`, and `pytest --cov` all pass locally. The decoder modules are explicitly **excluded from the 90% coverage threshold** (NFR23 line 690: "transports and decoders excluded, since both are largely adapter code"), but the new tests should still drive **100% line + branch coverage** on `src/httpware/decoders/__init__.py` and `src/httpware/decoders/pydantic.py` — the surface area is small (~25 LOC combined), so full coverage is cheap and prevents regressions. The CI invariant `grep -rE 'import httpx2|from httpx2' src/httpware/` (Story 1.4) is unaffected — this story adds no `httpx2` references. - -**AC14.** **And** `CHANGELOG.md`'s `Unreleased` → `Added` section gains one new bullet for this story (after Story 1.4's bullet), e.g. "ResponseDecoder protocol and PydanticDecoder adapter (Story 1.5)." Do not reformat existing entries; append only. - -## Tasks/Subtasks - -- [x] **Task 1 — `ResponseDecoder` protocol module** (AC1, AC12) - - [x] Create `src/httpware/decoders/__init__.py` - - [x] Add module docstring (one line, mentions Seam 3) - - [x] Define `T = TypeVar("T")` at module scope - - [x] Define `@runtime_checkable` `class ResponseDecoder(Protocol):` with the single `decode` method - - [x] Set `__all__ = ["ResponseDecoder"]` - - [x] Verify `ty check src/httpware/decoders/__init__.py` is clean - -- [x] **Task 2 — `PydanticDecoder` class** (AC2, AC4, AC5) - - [x] Create `src/httpware/decoders/pydantic.py` - - [x] Add module docstring (one line) - - [x] Top-of-file imports: `import functools`, `from pydantic import TypeAdapter`, `from typing import TypeVar` - - [x] Define module-level `T = TypeVar("T")` - - [x] Define `class PydanticDecoder:` with `decode(self, content: bytes, model: type[T]) -> T` returning `_get_adapter(model).validate_json(content)` - - [x] Set `__all__ = ["PydanticDecoder"]` - -- [x] **Task 3 — `_get_adapter` cached factory** (AC3, AC4) - - [x] In `src/httpware/decoders/pydantic.py`, define `@functools.lru_cache(maxsize=None)` decorated `_get_adapter(model: type[T]) -> TypeAdapter[T]` - - [x] Body: `return TypeAdapter(model)` (single line) - - [x] Place `_get_adapter` above `PydanticDecoder` so the class can reference it - - [x] Leading underscore — NOT in `__all__` - -- [x] **Task 4 — Public re-exports** (AC6) - - [x] Add `from httpware.decoders import ResponseDecoder` to `src/httpware/__init__.py` - - [x] Add `from httpware.decoders.pydantic import PydanticDecoder` to `src/httpware/__init__.py` - - [x] Extend `__all__` to 26 entries (let `ruff check --fix` handle RUF022 sort) - - [x] Verify `from httpware import ResponseDecoder, PydanticDecoder` resolves at the REPL - -- [x] **Task 5 — Unit tests for type categories** (AC7, AC10) - - [x] Create `tests/test_decoders_pydantic.py` - - [x] Define fixtures: a `pydantic.BaseModel` subclass `User`, a stdlib `@dataclasses.dataclass` `UserDC` - - [x] Parametrized success test over five (type, bytes, assertion) tuples — AC7 table - - [x] Negative test: `pydantic.ValidationError` surfaces unchanged on malformed input (AC10) - -- [x] **Task 6 — Cache invariance tests** (AC8, AC11) - - [x] In the same test file, add a synchronous test that monkeypatches the `TypeAdapter` constructor, calls `decode(content, User)` 1000 times, and asserts exactly one construction - - [x] Add an async test that schedules 50 concurrent `decode` coroutines after `_get_adapter.cache_clear()`, asserts exactly one construction - - [x] Add a third test asserting two distinct model types in the same loop trigger exactly two constructions - -- [x] **Task 7 — Benchmark** (AC9) - - [x] Create `tests/test_decoders_pydantic_bench.py` - - [x] Fixture: deterministic 5KB JSON payload (list of `User` dicts; pad to ~5120 bytes) plus `User` BaseModel definition - - [x] Two `pytest-benchmark` cases: single-pass (`PydanticDecoder().decode(content, list[User])`) vs two-pass (`TypeAdapter(list[User]).validate_python(json.loads(content))`) - - [x] Assert `single_pass.stats.mean * 2 <= two_pass.stats.mean` (or equivalent ratio assertion) - - [x] Mark benchmark to run under `pytest --benchmark-only` invocation as well as regular `pytest` - -- [x] **Task 8 — Changelog + status flip** (AC14) - - [x] Append a bullet under `Unreleased` → `Added` in `CHANGELOG.md` - - [x] Update front-matter `status:` and trailing `## Status` in this story file to `review` - - [x] Fill in `Dev Agent Record` → `Implementation Plan`, `Debug Log`, `Completion Notes`, and `File List` - - [x] Append a new row to `## Change Log` - -- [x] **Task 9 — Final verification** - - [x] `just lint` clean - - [x] `just test` passes; full suite count grows by 5 unit tests (AC7) + 3 cache tests (AC8/AC11) + 1 negative test (AC10) + 2 benchmark cases (AC9) ≈ 11 new tests (one parametrized case still counts as several test IDs) - - [x] 100% line + branch coverage on `src/httpware/decoders/__init__.py` and `src/httpware/decoders/pydantic.py` - - [x] Full suite coverage remains ≥90% on the threshold-tracked modules - - [x] `from httpware import ResponseDecoder, PydanticDecoder` succeeds; both appear in `httpware.__all__` - - [x] `grep -rE 'import httpx2|from httpx2' src/httpware/` still returns exactly one match (`transports/httpx2.py` — unchanged from Story 1.4) - -## Dev Notes - -### Architecture references (authoritative — read these before coding) - -- `docs/architecture.md` § **Decision 8 — ResponseDecoder protocol** (lines 270–283). Authoritative shape; ship verbatim. -- `docs/architecture.md` § **Seam 3 — AsyncClient ↔ ResponseDecoder** (lines 703–708). Documents the v1.x public-contract status of the protocol. -- `docs/architecture.md` § **Structure Patterns** (lines 426–429). `decoders/__init__.py` holds the protocol; `decoders/pydantic.py` holds the cached TypeAdapter adapter; `decoders/msgspec.py` is extras-gated (Story 1.6). -- `docs/architecture.md` § **Type-Hint Style** (lines 463–472). No `from __future__ import annotations`; PEP 604 unions; `@runtime_checkable` only where isinstance gating is real. -- `docs/architecture.md` § **Async Naming** (lines 474–478). `decode` is **sync** `def` — CPU-bound; no `a` prefix. -- `docs/architecture.md` § **Optional-Extra Import Pattern** (lines 509–530). **Does NOT apply** to `decoders/pydantic.py` (pydantic is a base dep). Will apply to `decoders/msgspec.py` in Story 1.6. -- `docs/architecture.md` § **Public API Export Discipline** (lines 531–536). `__all__` is the single source of truth; CI snapshot test asserts the set. -- `docs/architecture.md` § **Cross-cutting concerns → NFR2** (line 43): "Module-level cache keyed by `response_model` — explicit memoization layer". -- `docs/prd.md` **FR31–FR35** (lines 622–626). The functional contract for the decoder family. -- `docs/prd.md` **NFR1** (line 653; ≤15% overhead), **NFR2** (line 654; zero `TypeAdapter` constructions per request after warm-up), **NFR3** (line 655; single parse pass), **NFR17** (line 678; ty + py.typed), **NFR20** (line 684; pydantic v2 compatibility), **NFR23** (line 690; coverage exclusion for adapters). -- `docs/epics.md` **Story 1.5** (lines 338–352) — authoritative AC source; this file expands it. -- `CLAUDE.md` § Code conventions — kwargs-only exception construction (unchanged here), `# ty: ignore[…]` only, no `print()`, absolute imports inside `src/httpware/`. -- `docs/stories/1-4-transport-protocol-and-httpx2transport-adapter.md` — the protocol+adapter shape pattern to mirror (especially: structural-only conformance, `@runtime_checkable`, module-level `__all__`, RUF022 expansion of public `__all__`). - -### Key design points - -**`ResponseDecoder` is a structural Protocol, not an ABC.** Same rationale as `Transport` (Story 1.4): `@runtime_checkable` enables `isinstance` gating at `AsyncClient(decoder=...)` (Story 1.7), and structural typing means `PydanticDecoder` does not need to inherit. **Do not** make `PydanticDecoder` inherit from `ResponseDecoder` — inheritance would be redundant with the structural check and creates needless coupling. - -**Per-method `TypeVar`, not class-level generic.** The protocol's `T` is bound per `decode` call, not per decoder instance. One decoder handles many model types. Express this by placing `T` at module scope (not in `Protocol[T]`): - -```python -T = TypeVar("T") - -@runtime_checkable -class ResponseDecoder(Protocol): - def decode(self, content: bytes, model: type[T]) -> T: ... -``` - -This matches the architecture's snippet at lines 275–277 verbatim. Class-level `Protocol[T]` would force `decoder: ResponseDecoder[User]`, breaking the "single decoder, many models" use case (and the AsyncClient method overloads in Story 1.7). - -**Module-level cache, not instance-level.** `_get_adapter` is a free function (not a `PydanticDecoder` method) decorated with `@functools.lru_cache(maxsize=None)`. Rationale: (1) pydantic's official guidance is "instantiate `TypeAdapter` once and reuse"; (2) the cache key is `model`, which is identical regardless of which `PydanticDecoder` instance asked; (3) module-level cache means multiple `PydanticDecoder()` instances (e.g., from `client.with_options(decoder=...)` later) share the cache, which is what users want. Trade-off: the cache lives for the process lifetime — but the bound is the consumer's schema universe, not request volume, so `maxsize=None` is safe (matches the architecture line 280). - -**`functools.lru_cache` thread-safety is sufficient.** CPython's `functools.lru_cache` is thread-safe for the operations it performs (the cache dict is protected by GIL atomicity for the lookup/insert path). The narrow race window — two threads call `_get_adapter(SameModel)` simultaneously when the cache is empty, both construct a `TypeAdapter`, one wins the cache slot, the other's instance is GC'd — is acceptable per pydantic's reuse semantics (`TypeAdapter` construction is idempotent; two instances for the same model decode identically). The AC8/AC11 tests verify the post-warmup invariant, not absolute serialization of cache writes. **Do not** add an `asyncio.Lock` or `threading.Lock` around `_get_adapter` — it would only protect a degenerate race that doesn't affect correctness, at the cost of contention on every cached lookup. - -**Single parse pass is the entire performance story.** `pydantic.TypeAdapter(...).validate_json(bytes)` parses JSON and validates against the schema in one Rust-side traversal. The two-pass alternative — `json.loads(bytes)` then `validate_python(parsed)` — parses JSON to a Python dict (allocations everywhere) and then re-walks it inside pydantic-core. NFR3 codifies this; AC9's benchmark proves the 2× minimum. Architecture line 279: "Operates on raw `bytes` (NFR3 — single parse pass)". - -**`pydantic` is a base dependency, not extras.** `pyproject.toml` line 32: `"pydantic>=2.0,<3.0"`. The Optional-Extra Import Pattern (architecture lines 509–530) is for `msgspec`, `otel`, `niquests` — not pydantic. Top-of-file `from pydantic import TypeAdapter` is correct; no `try/except ImportError` shim. - -**The `decoders/pydantic.py` module name shadows nothing.** Python 3 absolute imports always resolve to top-level `sys.path` packages first; `from pydantic import ...` from inside `httpware/decoders/pydantic.py` resolves to the third-party `pydantic`, not the current module. Relative-import syntax (`from . import pydantic`) would be needed to reach the local module, which we never do. This is a deliberate choice in the architecture (line 428: `pydantic.py # PydanticDecoder + TypeAdapter cache`) — name the module by what it adapts, not what it adapts *to*. - -**`ValidationError` surfaces unchanged.** The architecture and PRD do not specify wrapping `pydantic.ValidationError` in an httpware exception. FR36 ("framework raises `httpware`-owned exceptions only") refers to the transport seam — translating httpx2 exceptions at `Httpx2Transport`. The decoder's failure mode is the consumer's contract violation with their own `response_model`; wrapping `ValidationError` would hide pydantic's structured error location data behind a thinner shim. Flag for reviewer (Open Questions). - -**`decode` is sync, not async.** Decoding is CPU-bound; making it async would invite event-loop blocking with no benefit. The Story 1.7 `AsyncClient.get(..., response_model=User)` path will simply call `decoder.decode(response.content, User)` synchronously after the awaitable transport call returns — the architecture confirms this at line 705 ("the client invokes `decoder.decode(response.content, response_model)`"). - -**Single-decoder-instance is the default in Story 1.7.** `AsyncClient.__init__` will use `decoder=PydanticDecoder()` if none is supplied (Story 1.7 AC, epic line 382). Therefore `PydanticDecoder` must be cheaply instantiable (no expensive `__init__` work). The lru-cache lives at module scope, so default construction is `O(1)`. - -### Public API Surface (canonical `__all__` after this story) - -After Story 1.5, `httpware/__init__.py` must re-export exactly these **26** symbols in RUF022 order (ALL-CAPS first, then mixed-case alphabetically): - -```python -__all__ = [ - "STATUS_TO_EXCEPTION", - "BadRequestError", - "ClientConfig", - "ClientError", - "ClientStatusError", - "ConflictError", - "ForbiddenError", - "Httpx2Transport", - "InternalServerError", - "Limits", - "NotFoundError", - "PydanticDecoder", - "RateLimitedError", - "Request", - "Response", - "ResponseDecoder", - "ServerStatusError", - "ServiceUnavailableError", - "StatusError", - "StreamResponse", - "Timeout", - "TimeoutError", - "Transport", - "TransportError", - "UnauthorizedError", - "UnprocessableEntityError", -] -``` - -That's 26 entries: 24 from Story 1.4 + 2 new (`PydanticDecoder`, `ResponseDecoder`). The CI gate is `ruff check --select RUF022`. (`PydanticDecoder` sorts between `NotFoundError` and `RateLimitedError`; `ResponseDecoder` between `Response` and `ServerStatusError`. Trust ruff.) - -### What lives where after this story - -| File | New / modified | Contents | -|---|---|---| -| `src/httpware/decoders/__init__.py` | **new** | `@runtime_checkable` `ResponseDecoder(Protocol)` with `decode`. Module-level `T = TypeVar("T")`. | -| `src/httpware/decoders/pydantic.py` | **new** | `PydanticDecoder`: no-arg `__init__`, `decode(content, model)` calling `_get_adapter(model).validate_json(content)`. Module-level `_get_adapter` with `@functools.lru_cache(maxsize=None)`. | -| `src/httpware/__init__.py` | **modify** | Add 2 new re-exports; expand `__all__` from 24 to 26. | -| `tests/test_decoders_pydantic.py` | **new** | AC7 (5 type categories), AC8 + AC11 (cache invariance, sync + async + multi-model), AC10 (`ValidationError` surface). | -| `tests/test_decoders_pydantic_bench.py` | **new** | AC9 (≥2× single-pass vs two-pass on a 5KB payload). | -| `CHANGELOG.md` | **modify** | One new bullet under `Unreleased` → `Added`. | - -### Read-before-edit (per architect's guidance) - -Files this story modifies (read current state before editing): - -- `src/httpware/__init__.py` — currently 54 lines (Story 1.4 close-out): module docstring, six `from httpware. import (...)` blocks (config, errors, request, response, transports, transports.httpx2), one `__all__` with 24 entries. Add two new imports (`from httpware.decoders import ResponseDecoder`, `from httpware.decoders.pydantic import PydanticDecoder`), and extend `__all__` to 26 entries. `ruff check --fix` will re-sort imports and `__all__`. -- `CHANGELOG.md` — `Unreleased` → `Added` has bullets for Stories 1.1, 1.2, 1.3, 1.4. Append at the end of that section; do not reformat existing entries. - -Files this story creates (no prior state to preserve): `src/httpware/decoders/__init__.py`, `src/httpware/decoders/pydantic.py`, `tests/test_decoders_pydantic.py`, `tests/test_decoders_pydantic_bench.py`. - -### Carryover from Story 1.4 - -- The `Response` type from Story 1.2 has a `.content: bytes` field — the decoder consumes this field directly (the AsyncClient call site in Story 1.7 will pass `response.content` as the first argument to `decode`). -- `Httpx2Transport.__call__` (Story 1.4) already returns `Response(content=resp.content, ...)` where `resp.content` is `bytes`; no transport-side changes are needed. -- The `Transport` protocol pattern (Story 1.4) — `@runtime_checkable` `Protocol` with structural conformance, no inheritance for the default adapter — is the template this story mirrors. Differences: the decoder is sync; the protocol is non-generic at class level but generic at method level. -- The `__all__` expansion convention (Story 1.4: 21 → 24) — append, run `ruff check --fix`, trust RUF022 — is the same pattern here (24 → 26). -- The "lazy `httpx2.AsyncClient`" pattern from Story 1.4 (event-loop binding) **does not apply** to this story: `TypeAdapter` construction is loop-independent, and the cache is lazy by virtue of `lru_cache`'s on-demand population. -- `pytest-asyncio` is in `auto` mode — async tests don't need `@pytest.mark.asyncio` (architecture line 548). -- `from __future__ import annotations` is forbidden anywhere (architecture line 465, CLAUDE.md line 27). -- `# ty: ignore[]` is the only allowed type-suppression form (CLAUDE.md line 11). - -### Anti-patterns to reject (will fail review or CI) - -- ❌ `from __future__ import annotations` anywhere. -- ❌ `Protocol[T]` (class-level generic) instead of per-method `T` — would break "one decoder, many models" downstream in Story 1.7's overload typing. -- ❌ Making `PydanticDecoder` inherit from `ResponseDecoder`. Structural subtyping is the design (architecture line 282 applied to decoders). -- ❌ Adding `try/except ImportError` around `from pydantic import TypeAdapter`. Pydantic is a base dep — the Optional-Extra Import Pattern doesn't apply here. (Story 1.6 will apply it to msgspec.) -- ❌ Instance-level `lru_cache` (e.g., `@functools.lru_cache` on `self.decode`). Methods bound on `self` interact badly with `lru_cache` (the cache becomes per-call-bound-method, not per-class) and prevent multi-decoder cache sharing. Use a module-level free function. -- ❌ `lru_cache(maxsize=128)` or any bounded size. The model space is bounded by the consumer's schemas; eviction would force re-construction at the worst possible time (warm production). Architecture line 280 explicitly says `maxsize=None`. -- ❌ Two-pass parse: `validate_python(json.loads(content))`. Violates NFR3 explicitly; AC9's 2× benchmark would fail. -- ❌ Catching `pydantic.ValidationError` inside `decode` and re-raising as a different type. AC10 forbids it; FR36 does not require it for the decoder seam (see Open Questions). -- ❌ `decode` as `async def`. Decoding is CPU-bound; async would invite event-loop blocking and would not match the architecture's signature (line 276). -- ❌ Calling `pydantic.TypeAdapter(model).validate_json(content)` directly inline (skipping `_get_adapter`). Bypasses the cache; violates NFR2; AC8 would fail. -- ❌ Importing the local module via relative syntax (`from .pydantic import PydanticDecoder` from outside `decoders/`). The architecture's import rule (line 559) permits relative imports only within the same subpackage; `httpware/__init__.py` uses absolute `from httpware.decoders.pydantic import PydanticDecoder`. -- ❌ Putting `_get_adapter` in `__all__`. It's private (underscore prefix). -- ❌ Wrapping `decode` in a `try/except` to attach extra context. The decoder is one of the few "raise-through" surfaces in the library. -- ❌ Adding a `pydantic.BaseModel` subclass to `httpware/_internal/` or anywhere in the library code. The decoder is the only place pydantic is referenced; consumers bring their own models. -- ❌ `Optional[X]` / `Union[X, Y]` — use `X | None` / `X | Y` (PEP 604). -- ❌ `typing.List`, `typing.Dict` — use `list`, `dict` (PEP 585). -- ❌ Adding extra `__init__.py` files to `tests/` subdirectories. - -### Testing standards summary - -- `pytest-asyncio` auto mode — async test functions don't need `@pytest.mark.asyncio` (architecture line 548). -- `pytest-benchmark` for AC9 (already in dev deps; `pyproject.toml` line 62). -- Parametrize the AC7 type-category test for density. -- For AC8/AC11 cache tests: use `unittest.mock.patch` on `httpware.decoders.pydantic.TypeAdapter` (patching at the call site, per pytest mock conventions). Clear the lru-cache between tests with `_get_adapter.cache_clear()` (importable via `from httpware.decoders.pydantic import _get_adapter`). -- Coverage target: **100% line + branch** on both new decoder modules. NFR23 excludes decoders from the project's ≥90% gate, but the modules are tiny (~25 LOC combined) and full coverage is cheap. -- No Hypothesis tests in this story (no concurrency-sensitive primitives; AC11 is a single async test, not a property test). -- No real-network tests in this story; all tests are pure-function or pure-coroutine (no transport involved). -- No `respx` (architecture line 553) — not relevant here since the decoder is transport-independent. -- The benchmark file (`tests/test_decoders_pydantic_bench.py`) is part of the regular `pytest` suite; pytest-benchmark hooks pick it up automatically. The `--benchmark-only` invocation runs only benchmark tests; the default `pytest` invocation runs benchmarks alongside unit tests (with the benchmark assertion verifying the ratio). - -### Definition of Done - -- All 14 ACs verified (each AC maps to at least one test in `tests/test_decoders_pydantic.py` / `tests/test_decoders_pydantic_bench.py`, or to a check in `just lint`, or to a `__all__`/import assertion). -- All Task/Subtask checkboxes are `[x]`. -- `ruff format`, `ruff check`, `ty check`, `pytest` all pass locally with zero diagnostics. -- Coverage 100% (line + branch) on `src/httpware/decoders/__init__.py` AND `src/httpware/decoders/pydantic.py`. Threshold-tracked coverage on the prior modules holds. -- `from httpware import ResponseDecoder, PydanticDecoder` succeeds; `__all__` contains both; RUF022 is clean; total `__all__` length is 26. -- `grep -rE 'import httpx2|from httpx2' src/httpware/` still returns exactly one path: `src/httpware/transports/httpx2.py` — unchanged from Story 1.4. -- File List below is updated to reflect every changed and new file. -- `CHANGELOG.md` has a new dated bullet under `Unreleased` → `Added`. -- Front-matter `status:` and trailing `## Status` are both set to `review`. - -### Open questions / things to flag for reviewer - -- **`pydantic.ValidationError` policy (FR36 tension).** AC10 surfaces `pydantic.ValidationError` unchanged. FR36 says "framework raises `httpware`-owned exceptions only". The tension: FR36 is about *transport* exception leakage (httpx2 types); the decoder seam is on the *consumer's* side of the model contract. The architecture (Decision 8) does not specify wrapping. Defaults: do not wrap. Alternatives: (a) define a thin `DecodingError(ClientError)` wrapper in `errors.py` and translate at the seam, preserving `__cause__`; (b) leave as pydantic. Flag. -- **`+json` content types and friends.** The decoder accepts `bytes` regardless of the response's content-type. `application/json`, `application/vnd.api+json`, `application/problem+json` all parse identically because the bytes are JSON. The decoder does not inspect content-type — that's the caller's responsibility (or a future middleware's). Confirm reviewer is OK with the "decoder trusts the bytes" contract. -- **`maxsize=None` cache eviction policy.** Unbounded by design (architecture line 280, prd NFR2). The risk surface: a consumer that builds many short-lived ad-hoc model types in a loop (e.g., dynamic pydantic class generation) could leak memory. Real-world consumers don't do this; flag if reviewer wants a `maxsize=1024` safety cap. -- **Multiple `PydanticDecoder` instances share the module cache.** A consumer who builds two `AsyncClient` instances with different `PydanticDecoder()` instances will see the *same* `TypeAdapter` per model across both clients. This is correct (TypeAdapter has no per-client state), but worth confirming — flag if reviewer prefers per-instance cache for isolation (would require attaching the cache to `self` and breaking the module-level free function — see "Anti-patterns" for why we reject that). -- **Concurrent first-call race window.** AC11 documents that two concurrent first-calls may both construct a `TypeAdapter` before one wins the cache slot, and that this is acceptable. Story 1.4 used an explicit `asyncio.Lock` for the same shape of race (lazy `httpx2.AsyncClient`). The difference: there, the client was a heavyweight resource with side effects (event-loop binding); here, `TypeAdapter` construction is pure and idempotent, so a loser instance is just a GC microcost. Flag if reviewer wants symmetry with Story 1.4's pattern. -- **Benchmark stability under CI.** AC9 asserts ≥2× speedup. Real-world ratios on `validate_json(bytes)` vs `validate_python(json.loads(bytes))` are typically 3–5×, so 2× is conservative. But CI runners are noisy. Possible mitigations: `pytest.mark.benchmark(group="decoder")` to allow `--benchmark-compare`; `pytest-benchmark` already uses statistical means. If CI flakes, the fallback is to widen to ≥1.5× and re-flag the NFR3 phrasing. -- **`Protocol` + per-method `TypeVar` for `ty`.** AC12 assumes `ty` accepts this shape (same shape as `Transport`'s signature in Story 1.4 — which did pass `ty`). If `ty` regresses or the decoder shape triggers a new diagnostic, the workaround is `class ResponseDecoder(Protocol):` with `decode` annotated using a `@typing.overload`-free `TypeVar` re-binding. Don't introduce `ty: ignore` unless you've exhausted other shapes. - -## Change Log - -| Date | Change | Notes | -|---|---|---| -| 2026-05-14 | Story created | Drafted from `docs/epics.md` Story 1.5 + `docs/architecture.md` Decision 8 + Story 1.4's adapter pattern; expanded the epic's 5-clause AC into AC1–AC14 for traceability; verified pydantic v2 `TypeAdapter` API surface and reuse guidance via context7; confirmed `pydantic` is a base dep (not extras-gated, unlike Story 1.6's msgspec); noted Python 3.11 floor implications for protocol generic syntax; flagged FR36 tension over `ValidationError` wrapping, `+json` content-type policy, cache eviction, and benchmark CI stability for reviewer. | -| 2026-05-14 | Story implemented | Shipped `ResponseDecoder` protocol + `PydanticDecoder` adapter + module-level `_get_adapter` cache; added 11 unit/cache tests + 3 benchmark tests; 100% line+branch coverage on both new decoder modules; `__all__` 24 → 26 entries; `just lint` + `just test` clean. AC9 benchmark threshold relaxed to ≥1.5× per Open Questions item 5 (≥2× not portable on this hardware/pydantic-core version); reviewer to confirm the documented fallback. | -| 2026-05-14 | Code review applied | Parallel-layer review (Blind Hunter + Edge Case Hunter + Acceptance Auditor): 5 decision-needed → resolved, 2 patches applied, 4 deferred (in `docs/deferred-work.md`), 15 dismissed. Amendments: AC3 `maxsize=None` → `maxsize=1024` (Decision 1, Open Q 3); AC4 added `try/except TypeError` fallback for unhashable `model` (Decision 5); AC9 ≥2× → ≥1.5× and gated behind `pytest -m perf` (Decisions 2 + 3, Open Q 5); added `ThreadPoolExecutor`-based concurrent cache-invariance test (Decision 4); added autouse `_clear_adapter_cache` fixture to unit tests; `_warm_cache` bench fixture moved to `scope="module"`. | - -## Dev Agent Record - -### Implementation Plan - -1. Create `decoders/__init__.py` with `ResponseDecoder(Protocol)` carrying a single per-call-generic `decode` method, gated by `@runtime_checkable`. `T = TypeVar("T")` at module scope per AC1/AC12. -2. Create `decoders/pydantic.py` with module-level `@functools.lru_cache(maxsize=None)` factory `_get_adapter(model) -> TypeAdapter[T]` and a stateless `PydanticDecoder` whose `decode` is `return _get_adapter(model).validate_json(content)`. No inheritance from the protocol. -3. Wire two new symbols into `src/httpware/__init__.py`; let `ruff check --fix` resolve RUF022 ordering (24 → 26 entries). -4. Tests: parametrize-free unit tests covering AC7's 5 type categories, AC10 `ValidationError` surfacing, AC8 single-model and two-model cache invariance via `unittest.mock.patch(..., wraps=pydantic.TypeAdapter)`, AC11 50-coroutine concurrent first-call test against a freshly-cleared cache. -5. Benchmark file: spec-aligned `list[User]` shape; two `pytest-benchmark` cases (single-pass + two-pass) plus a deterministic ratio assertion using `time.perf_counter_ns` with GC disabled. -6. CHANGELOG bullet, story bookkeeping. - -### Debug Log - -- Initial `_get_adapter` decoration: ruff rewrote `@functools.lru_cache(maxsize=None)` → `@functools.cache` (UP033). AC3 specifies the literal form, so the rewrite was reverted with `# noqa: UP033` to keep the AC-verbatim signature. -- AC9 benchmark threshold (≥2×) was not achievable on this dev machine (Darwin 24.3.0 / Apple Silicon / pydantic 2.13.4 + pydantic_core 2.46.4): the measured `validate_json(bytes)` vs `validate_python(json.loads(bytes))` ratio for `list[User]` shapes lands at 1.18×–1.65× depending on payload composition. The Open Questions section of this story (item 5) preemptively documented a ≥1.5× fallback for this exact CI/portability scenario; the benchmark file uses `SPEEDUP_FLOOR = 1.5` with a comment citing the open question. The decoder still uses `validate_json` (single parse pass), so the structural NFR3 invariant is satisfied — only the numerical proxy was relaxed. Flagged in Completion Notes for reviewer. -- Pyright IDE diagnostics about unresolved `httpware.decoders.pydantic` imports were stale (new module not yet indexed); `ty check` is the project's actual gate and reported zero diagnostics. No code changes were warranted. - -### Completion Notes - -- All 14 ACs satisfied with one documented deviation: AC9 uses `SPEEDUP_FLOOR = 1.5` (per the story's own Open Questions item 5) instead of the literal `≥2×`. Local median ratio across runs: ~1.63×. Reviewer to confirm whether to (a) keep the documented fallback, (b) re-tune the benchmark payload (e.g., a `dict[str, list[int]]` shape consistently exceeds 2×) at the cost of drifting from the AC's "User-like dicts" wording, or (c) tighten when CI hardware/pydantic-core versions move. -- Decoder modules reach 100% line + branch coverage; full suite jumps from 142 → 156 tests (+14). All 156 pass, full-suite line+branch coverage = 99% (the remaining 1% is pre-existing partial branches in `transports/httpx2.py` and `response.py`). -- `from httpware import ResponseDecoder, PydanticDecoder` resolves; `httpware.__all__` has 26 entries and is RUF022-sorted. -- `grep -rE 'import httpx2|from httpx2' src/httpware/` still returns exactly one path (`src/httpware/transports/httpx2.py`) — Story 1.4 invariant preserved. -- The cache-invariance tests patch `httpware.decoders.pydantic.TypeAdapter` with `wraps=pydantic.TypeAdapter` (real construction still happens, so the cache holds a working adapter), `_get_adapter.cache_clear()` runs in test setup to neutralize prior fills, and `spy.call_count` is asserted exactly once for the same-model 1000-call loop and exactly twice for the two-distinct-model loop. The async concurrent-first-call test schedules 50 coroutines through `asyncio.gather`; since `decode` is sync there is no true preemption point, but the test still verifies the post-warmup invariant the story requires. - -## File List - -**New** -- `src/httpware/decoders/__init__.py` -- `src/httpware/decoders/pydantic.py` -- `tests/test_decoders_pydantic.py` -- `tests/test_decoders_pydantic_bench.py` - -**Modified** -- `src/httpware/__init__.py` — added two re-exports; `__all__` 24 → 26 entries (RUF022-sorted). -- `CHANGELOG.md` — one new bullet under `Unreleased` → `Added`. -- `docs/stories/sprint-status.yaml` — `1-5-responsedecoder-protocol-and-pydantic-adapter`: `ready-for-dev` → `in-progress` → `review`. -- `docs/stories/1-5-responsedecoder-protocol-and-pydantic-adapter.md` — Tasks/Subtasks checked, status flipped, Dev Agent Record and File List populated, Change Log row appended. - -## Review Findings - -Code review performed 2026-05-14 (parallel: Blind Hunter + Edge Case Hunter + Acceptance Auditor). 26 raw findings → 5 decision-needed, 2 patch, 4 deferred, 15 dismissed. - -### Decision-needed (resolved 2026-05-14) - -- [x] [Review][Decision] **Unbounded `lru_cache` cache leak with dynamic `pydantic.create_model` types** — Resolved: cap at `maxsize=1024`; AC3 amended. (Decision 1 / Open Q 3) -- [x] [Review][Decision] **AC9 benchmark threshold relaxed from ≥2× to ≥1.5×** — Resolved: keep `SPEEDUP_FLOOR = 1.5`; AC9 text amended to ≥1.5× citing Open Q 5. (Decision 2) -- [x] [Review][Decision] **Benchmark timing test runs in default `pytest` invocation** — Resolved: timing test gated behind `@pytest.mark.perf`; `pyproject.toml` excludes via `addopts = "... -m 'not perf'"`; run with `pytest -m perf`. (Decision 3) -- [x] [Review][Decision] **AC11 async test does not exercise real concurrency** — Resolved: kept async test (post-warmup invariant), added `test_cache_invariance_concurrent_first_calls_threadpool` with `ThreadPoolExecutor` to exercise the real GIL-level race. (Decision 4) -- [x] [Review][Decision] **Decoder contract for unhashable `model` arguments** — Resolved: added `try/except TypeError` fallback in `decode`; AC4 amended; new test `test_unhashable_model_falls_back_to_uncached_adapter` pins the contract. (Decision 5) - -### Patch (applied 2026-05-14) - -- [x] [Review][Patch] Added module-level autouse `_clear_adapter_cache` fixture to `tests/test_decoders_pydantic.py`. -- [x] [Review][Patch] Changed `_warm_cache` in `tests/test_decoders_pydantic_bench.py` to `scope="module"`. - -### Deferred - -- [x] [Review][Defer] Empty/malformed payload tests (`b""`, `b"null"`, `b"{}"`, invalid UTF-8) — current pydantic-core behavior is correct but unpinned; a future pydantic upgrade could change the error type undetected. Defer to a dedicated decoder-contract test pass. [`tests/test_decoders_pydantic.py`] -- [x] [Review][Defer] Move `# noqa: PLR2004` to `tool.ruff.lint.per-file-ignores` for `tests/*` — repeated four times in this file; the per-file-ignores config is the idiomatic fix and would also clean up prior test modules. Defer to a project-wide lint-config tidy. [`tests/test_decoders_pydantic.py:48,57,68,82`] -- [x] [Review][Defer] CHANGELOG bullet leaks implementation detail (`_get_adapter`, "zero adapter-construction cost") into a user-facing log — style only; AC14 sets no wording constraint. Defer to a CHANGELOG tone pass before v1 release. [`CHANGELOG.md:19`] -- [x] [Review][Defer] `@runtime_checkable` `isinstance` cost when Story 1.7's `AsyncClient(decoder=...)` gate is added (~µs per check; matters only if the gate runs per-request rather than per-construction). Defer to Story 1.7 design. [N/A — future story concern] - -### Dismissed - -15 findings dismissed as noise or spec-permitted: `# noqa: UP033` is correct (verified via `ruff rule UP033`); `@runtime_checkable` attribute-only check is the spec's chosen design (mirrors Story 1.4's Transport); `type(result) is X` is AC7-mandated; AC6's alphabetic `__all__` is spec; `_get_adapter` private import in tests is spec-authorized; `pydantic.py` module-name shadowing was pre-verified; `PydanticDecoder` instance ceremony is intentional (module-level cache is the design); CHANGELOG content choice is spec-permitted; etc. diff --git a/docs/archive/stories/sprint-status.yaml b/docs/archive/stories/sprint-status.yaml deleted file mode 100644 index 4be355d..0000000 --- a/docs/archive/stories/sprint-status.yaml +++ /dev/null @@ -1,93 +0,0 @@ -# generated: 2026-05-14 -# last_updated: 2026-05-14 # story 1-5 → done (code review applied) -# project: httpware -# project_key: NOKEY -# tracking_system: file-system -# story_location: docs/stories - -# STATUS DEFINITIONS: -# ================== -# Epic Status: -# - backlog: Epic not yet started -# - in-progress: Epic actively being worked on -# - done: All stories in epic completed -# -# Epic Status Transitions: -# - backlog → in-progress: Automatically when first story is created (via create-story) -# - in-progress → done: Manually when all stories reach 'done' status -# -# Story Status: -# - backlog: Story only exists in epic file -# - ready-for-dev: Story file created in stories folder -# - in-progress: Developer actively working on implementation -# - review: Ready for code review (via Dev's code-review workflow) -# - done: Story completed -# -# Retrospective Status: -# - optional: Can be completed but not required -# - done: Retrospective has been completed -# -# WORKFLOW NOTES: -# =============== -# - Epic transitions to 'in-progress' automatically when first story is created -# - Stories can be worked in parallel if team capacity allows -# - Developer typically creates next story after previous one is 'done' to incorporate learnings -# - Dev moves story to 'review', then runs code-review (fresh context, different LLM recommended) - -generated: 2026-05-14 -last_updated: 2026-05-14 # story 1-5 → done (code review applied) -project: httpware -project_key: NOKEY -tracking_system: file-system -story_location: docs/stories - -development_status: - epic-1: in-progress - 1-1-project-scaffold-and-tooling: done - 1-2-core-data-types: done - 1-3-exception-hierarchy-with-plain-fields: done - 1-4-transport-protocol-and-httpx2transport-adapter: done - 1-5-responsedecoder-protocol-and-pydantic-adapter: done - 1-6-msgspec-decoder-via-extras: backlog - 1-7-asyncclient-with-http-methods-response-model-with-options-lifecycle: backlog - 1-8-recordedtransport-for-testing: backlog - epic-1-retrospective: optional - - epic-2: backlog - 2-1-middleware-protocol-next-type-and-chain-composition: backlog - 2-2-phase-shortcut-decorators: backlog - 2-3-request-immutability-helpers: backlog - 2-4-auth-coercion-as-middleware: backlog - 2-5-wire-middleware-into-asyncclient: backlog - epic-2-retrospective: optional - - epic-3: backlog - 3-1-timeout-middleware-per-attempt: backlog - 3-2-retry-middleware: backlog - 3-3-retrybudget-data-structure: backlog - 3-4-retrybudget-middleware-integration: backlog - 3-5-bulkhead-middleware: backlog - 3-6-document-the-extension-slot: backlog - epic-3-retrospective: optional - - epic-4: backlog - 4-1-streamresponse-type: backlog - 4-2-transport-stream-implementation-in-httpx2transport: backlog - 4-3-asyncclient-stream-context-manager: backlog - epic-4-retrospective: optional - - epic-5: backlog - 5-1-layer-1-observability-middleware-lifecycle-hooks: backlog - 5-2-wire-emission-into-resilience-middlewares: backlog - 5-3-redactor-class-and-integration: backlog - 5-4-opentelemetry-middleware: backlog - 5-5-logging-policy-enforcement: backlog - epic-5-retrospective: optional - - epic-6: backlog - 6-1-migration-guide-from-base-client: backlog - 6-2-documentation-site-mkdocs: backlog - 6-3-public-benchmark-suite: backlog - 6-4-ci-enforcement-gates: backlog - 6-5-release-flow-with-trusted-publishers-and-sigstore: backlog - epic-6-retrospective: optional diff --git a/docs/dev/contributing.md b/docs/dev/contributing.md new file mode 100644 index 0000000..af4029e --- /dev/null +++ b/docs/dev/contributing.md @@ -0,0 +1,46 @@ +# Contributing to httpware + +Thank you for your interest in contributing. `httpware` is an open-source resilience-first async HTTP client framework for Python, maintained under the [`modern-python`](https://github.com/modern-python) org. + +## Quick start + +```bash +git clone https://github.com/modern-python/httpware.git +cd httpware +just install # uv lock --upgrade && uv sync --all-extras --frozen --group lint +just lint # ruff format + ruff check + ty check +just test # pytest with coverage +``` + +## Development workflow + +1. **Open an issue first** for non-trivial changes — design discussion catches issues earlier than code review. +2. **Branch from `main`**, use a descriptive name (`feat/retry-budget-jitter`, `fix/transport-cancel-leak`). +3. **Run `just lint` and `just test`** locally before pushing. CI will reject changes that fail either. +4. **Add tests** for any code change. Property-based tests (via Hypothesis) are required for concurrency-sensitive code (retry budget, bulkhead, retry interleaving). +5. **Open a pull request** against `main`. PR titles use conventional-commits style (`feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`). + +## Code style + +- `ruff format` enforces formatting; do not hand-format. +- Type-check with `ty` (Astral). Use `# ty: ignore[]` for suppressions, not `# type: ignore`. +- Do NOT use `from __future__ import annotations`. Python 3.11+ is the floor. +- Module docstrings are required; per-method docstrings only when types alone are insufficient. + +## Architecture invariants + +These are enforced by CI grep gates. Do not break them in pull requests: + +- No `import httpx2` outside `src/httpware/transports/httpx2.py`. +- No `httpx2._*` (private API) usage anywhere in the library. +- No `from __future__ import annotations`. +- No `print()` calls. +- No `logging.basicConfig()` or bare `logging.getLogger()`. + +## Code of Conduct + +By participating in this project, you agree to abide by its Code of Conduct. Be excellent to one another. + +## License + +By contributing, you agree that your contributions will be licensed under the project's MIT license. diff --git a/docs/engineering.md b/docs/dev/engineering.md similarity index 89% rename from docs/engineering.md rename to docs/dev/engineering.md index 309d42a..616390c 100644 --- a/docs/engineering.md +++ b/docs/dev/engineering.md @@ -1,6 +1,6 @@ # `httpware` engineering notes -This doc is the single distilled reference for `httpware` design rationale, protocol seams, and remaining roadmap. It complements [`../CLAUDE.md`](../CLAUDE.md): `CLAUDE.md` holds AI-enforced invariants and operational commands; this file holds the reasoning and the structural map. Historical planning artifacts live in [`archive/`](./archive/) and are cited only for original rationale. +This doc is the single distilled reference for `httpware` design rationale, protocol seams, and remaining roadmap. It complements `CLAUDE.md` (at the repo root): `CLAUDE.md` holds AI-enforced invariants and operational commands; this file holds the reasoning and the structural map. ## 1. Project intent @@ -37,7 +37,7 @@ A protocol seam is a documented internal boundary. AI agents and contributors mu - **Where:** `src/httpware/client.py` ↔ `src/httpware/decoders/`. - **Contract:** the decoder is invoked when the caller passes `response_model=`. The protocol is `decode(content: bytes, model: type[T]) -> T`. -- **Rule:** the decoder must operate on raw bytes in a single parse pass. Two-pass decoding (`json.loads` then `validate_python`) is rejected — see `archive/architecture.md` Validation & Decoding for rationale. +- **Rule:** the decoder must operate on raw bytes in a single parse pass. Two-pass decoding (`json.loads` then `validate_python`) is rejected: a single bytes-in / typed-object-out pass avoids the redundant intermediate `dict` allocation and parses faster. The Pydantic adapter implements this as `TypeAdapter(model).validate_json(content)`, with the `TypeAdapter` itself memoized via `@functools.lru_cache(maxsize=1024)` on a module-level `_get_adapter(model)` factory (the adapter is the expensive part to build). The msgspec adapter implements it as `msgspec.json.decode(content, type=model)`. ### Seam 4: `Httpx2Transport ↔ httpx2` @@ -131,7 +131,7 @@ Caller-facing pattern: consumers select the implementation by passing it explici ## 8. Remaining roadmap -Twenty-seven stories remain. Topic slugs in `docs/superpowers/specs/` and `docs/superpowers/plans/` use kebab-case descriptions, not the story IDs — these IDs are kept here only as a stable mapping to the archived epic specs (`archive/epics.md`). +Twenty-seven stories remain. Topic slugs in `planning/specs/` and `planning/plans/` use kebab-case descriptions, not the story IDs — these IDs are retained as a stable identifier convention from the original epic structure. ### Epic 1 — Make typed HTTP requests with sensible defaults @@ -178,8 +178,8 @@ Twenty-seven stories remain. Topic slugs in `docs/superpowers/specs/` and `docs/ - **6-4** CI enforcement gates (codify the invariants in Section 2 as CI jobs). - **6-5** Release flow with Trusted Publishers + Sigstore. -When work starts on a roadmap item, it gets a superpowers spec at `docs/superpowers/specs/YYYY-MM-DD--design.md` and a plan at `docs/superpowers/plans/YYYY-MM-DD--plan.md`. The bmad-era 40KB story specs in `archive/stories/` cover 1-1 through 1-5 and are retired going forward. +When work starts on a roadmap item, it gets a spec at `planning/specs/YYYY-MM-DD--design.md` and a plan at `planning/plans/YYYY-MM-DD--plan.md`. ## 9. Deferred work -Review-surfaced items that are real but not actionable now live in [`./deferred-work.md`](./deferred-work.md). Each entry cites the originating story and the file/line, and explains why the fix is deferred (cross-story dependency, scope, performance/security tradeoff, etc.). When a deferred item becomes actionable, it migrates into the spec for the story that resolves it. +Review-surfaced items that are real but not actionable now live in `planning/deferred-work.md` at the repository root. Each entry cites the originating story and the file/line, and explains why the fix is deferred (cross-story dependency, scope, performance/security tradeoff, etc.). When a deferred item becomes actionable, it migrates into the spec for the story that resolves it. diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..2e0a51b --- /dev/null +++ b/docs/index.md @@ -0,0 +1,49 @@ +# httpware + +A Python async HTTP client framework for building resilient service clients. `httpware` owns the abstraction layer above the underlying HTTP client (`httpx2` by default); consumers never import the transport directly. + +> **Status:** Pre-1.0 (0.1.0 alpha). Public API is subject to change between minor releases until v1.0. + +## Install + +```bash +pip install httpware +``` + +Optional extras: + +```bash +pip install httpware[msgspec] # MsgspecDecoder +``` + +## First request + +```python +import asyncio + +from httpware import AsyncClient +from pydantic import BaseModel + + +class User(BaseModel): + id: int + name: str + + +async def main() -> None: + async with AsyncClient(base_url="https://api.example.com") as client: + user = await client.get("/users/1", response_model=User) + print(user.name) + + +asyncio.run(main()) +``` + +## Where to go next + +- **[Engineering Notes](dev/engineering.md)** — design invariants, the five protocol seams, exception contract, module layout, testing patterns, optional-extras pattern. +- **[Contributing](dev/contributing.md)** — setup, conventions, workflow. + +## Part of `modern-python` + +`httpware` ships under the [`modern-python`](https://github.com/modern-python) org. See the org profile for the categorized index of related templates and libraries. diff --git a/docs/requirements.txt b/docs/requirements.txt new file mode 100644 index 0000000..9a8a4ca --- /dev/null +++ b/docs/requirements.txt @@ -0,0 +1,2 @@ +mkdocs +mkdocs-material diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 0000000..0cc2299 --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,46 @@ +site_name: httpware +site_url: https://httpware.readthedocs.io/ +repo_url: https://github.com/modern-python/httpware +docs_dir: docs +edit_uri: edit/main/docs/ + +nav: + - Quick-Start: index.md + - Development: + - Engineering Notes: dev/engineering.md + - Contributing: dev/contributing.md + +theme: + name: material + features: + - content.code.copy + - content.action.edit + - navigation.footer + - navigation.sections + - navigation.top + - header.autohide + palette: + - media: "(prefers-color-scheme: light)" + scheme: default + primary: black + accent: pink + toggle: + icon: material/brightness-7 + name: Switch to dark mode + - media: "(prefers-color-scheme: dark)" + scheme: slate + primary: black + accent: pink + toggle: + icon: material/brightness-4 + name: Switch to system preference + +markdown_extensions: + - toc: + permalink: true + - pymdownx.highlight: + anchor_linenums: true + - pymdownx.inlinehilite + - pymdownx.superfences + - admonition + - attr_list diff --git a/docs/deferred-work.md b/planning/deferred-work.md similarity index 100% rename from docs/deferred-work.md rename to planning/deferred-work.md diff --git a/docs/superpowers/plans/2026-05-31-asyncclient-plan.md b/planning/plans/2026-05-31-asyncclient-plan.md similarity index 99% rename from docs/superpowers/plans/2026-05-31-asyncclient-plan.md rename to planning/plans/2026-05-31-asyncclient-plan.md index c8d732f..c10d715 100644 --- a/docs/superpowers/plans/2026-05-31-asyncclient-plan.md +++ b/planning/plans/2026-05-31-asyncclient-plan.md @@ -10,7 +10,7 @@ **Branch:** `story/1-7-asyncclient` (already created; spec commit `bebb1dd` is on it). -**Spec:** `docs/superpowers/specs/2026-05-31-asyncclient-design.md`. +**Spec:** `planning/specs/2026-05-31-asyncclient-design.md`. --- @@ -1805,7 +1805,7 @@ Expected: `eof-fixer`, `ruff format --check`, `ruff check --no-fix`, `ty check` - [ ] **Step 3: Confirm the working tree is clean** Run: `git status --short` -Expected: only the untracked plan file `docs/superpowers/plans/2026-05-31-asyncclient-plan.md`. +Expected: only the untracked plan file `planning/plans/2026-05-31-asyncclient-plan.md`. - [ ] **Step 4: Review the branch diff** @@ -1818,7 +1818,7 @@ Expected: changes to `CHANGELOG.md`, `src/httpware/__init__.py`, `src/httpware/c - [ ] **Step 5: Stage and commit the plan file** ```bash -git add docs/superpowers/plans/2026-05-31-asyncclient-plan.md +git add planning/plans/2026-05-31-asyncclient-plan.md git commit -m "docs(story-1.7): implementation plan for AsyncClient Co-Authored-By: Claude Opus 4.7 (1M context) " @@ -1845,7 +1845,7 @@ gh pr create --title "feat(story-1.7): AsyncClient — the v0.1.0 public surface **Out of scope and deferred:** `auth=` (Story 2-4), `data=`/`files=` body params, transport reference-counting, streaming (Epic 4), observability (Epic 5). -Spec + plan: `docs/superpowers/specs/2026-05-31-asyncclient-design.md`, `docs/superpowers/plans/2026-05-31-asyncclient-plan.md`. +Spec + plan: `planning/specs/2026-05-31-asyncclient-design.md`, `planning/plans/2026-05-31-asyncclient-plan.md`. ## Test plan diff --git a/docs/superpowers/plans/2026-05-31-bmad-to-superpowers-transition-plan.md b/planning/plans/2026-05-31-bmad-to-superpowers-transition-plan.md similarity index 87% rename from docs/superpowers/plans/2026-05-31-bmad-to-superpowers-transition-plan.md rename to planning/plans/2026-05-31-bmad-to-superpowers-transition-plan.md index 661d0d6..12ff9bb 100644 --- a/docs/superpowers/plans/2026-05-31-bmad-to-superpowers-transition-plan.md +++ b/planning/plans/2026-05-31-bmad-to-superpowers-transition-plan.md @@ -4,7 +4,7 @@ **Goal:** Cutover the `httpware` project from bmad to superpowers in a single PR. After merge, all future work runs through the superpowers flow (brainstorming → writing-plans → executing-plans → requesting-code-review). -**Architecture:** Docs-only refactor. Move five large bmad planning files plus the `docs/stories/` directory into `docs/archive/`. Write one new distilled `docs/engineering.md` (~250–350 lines) sourcing content from the archived docs and `CLAUDE.md`. Update `CLAUDE.md` to point at the new structure. Delete `.review-tmp/`. No source code changes. +**Architecture:** Docs-only refactor. Move five large bmad planning files plus the `docs/stories/` directory into `docs/archive/`. Write one new distilled `docs/dev/engineering.md` (~250–350 lines) sourcing content from the archived docs and `CLAUDE.md`. Update `CLAUDE.md` to point at the new structure. Delete `.review-tmp/`. No source code changes. **Tech Stack:** Markdown only. Git for renames. `just lint` and `just test` for regression verification (they should remain green; the cutover touches no source). @@ -13,17 +13,17 @@ **Prereqs already satisfied:** - Story 1-5 merged to `main` (PR #5). - Branch rebased onto post-1-5 `main`. -- Spec at `docs/superpowers/specs/2026-05-31-bmad-to-superpowers-transition-design.md` (committed). -- This plan file exists at `docs/superpowers/plans/2026-05-31-bmad-to-superpowers-transition-plan.md` (untracked until Task 6 stages it). +- Spec at `planning/specs/2026-05-31-bmad-to-superpowers-transition-design.md` (committed). +- This plan file exists at `planning/plans/2026-05-31-bmad-to-superpowers-transition-plan.md` (untracked until Task 6 stages it). --- ## File Structure **Files created:** -- `docs/engineering.md` — the distilled doc (~250–350 lines). Single source for design rationale + roadmap. +- `docs/dev/engineering.md` — the distilled doc (~250–350 lines). Single source for design rationale + roadmap. - `docs/archive/README.md` — ~1 paragraph framing the archive as historical reference. -- `docs/superpowers/plans/2026-05-31-bmad-to-superpowers-transition-plan.md` — this file. +- `planning/plans/2026-05-31-bmad-to-superpowers-transition-plan.md` — this file. **Files moved (git mv, preserves history):** - `docs/prd.md` → `docs/archive/prd.md` @@ -40,7 +40,7 @@ - `.review-tmp/` (entire directory) — bmad code-review artifact dump. **Files untouched (deliberate):** -- `docs/deferred-work.md` — kept at repo root as the active deferral log. +- `planning/deferred-work.md` — kept at repo root as the active deferral log. - All `src/httpware/**`, `tests/**`, `pyproject.toml`, `Justfile`, `.github/**`, `CHANGELOG.md`, `README.md`, `CONTRIBUTING.md`, `SECURITY.md`, `LICENSE` — no source/CI changes. --- @@ -58,7 +58,7 @@ - [ ] **Step 1: Verify branch and working-tree state** Run: `git branch --show-current && git status --short` -Expected: branch is `chore/bmad-to-superpowers-transition`. Working tree has untracked `docs/superpowers/plans/2026-05-31-bmad-to-superpowers-transition-plan.md` (this file) and nothing else. +Expected: branch is `chore/bmad-to-superpowers-transition`. Working tree has untracked `planning/plans/2026-05-31-bmad-to-superpowers-transition-plan.md` (this file) and nothing else. If anything else is dirty, stop and resolve before proceeding. @@ -134,12 +134,12 @@ Expected: ~18 lines. --- -## Task 3: Write `docs/engineering.md` +## Task 3: Write `docs/dev/engineering.md` This is the largest task. The doc has nine sections (per the spec). Build it section by section. Final length target: 250–350 lines. **Files:** -- Create: `docs/engineering.md` +- Create: `docs/dev/engineering.md` **Source material:** - `CLAUDE.md` — invariants and conventions (current file at repo root). @@ -150,7 +150,7 @@ This is the largest task. The doc has nine sections (per the spec). Build it sec - [ ] **Step 1: Write Section 1 — Project intent (3–4 sentences)** -Append to `docs/engineering.md`: +Append to `docs/dev/engineering.md`: ```markdown # `httpware` engineering notes @@ -332,7 +332,7 @@ Append: ```markdown ## 8. Remaining roadmap -Twenty-seven stories remain. Topic slugs in `docs/superpowers/specs/` and `docs/superpowers/plans/` use kebab-case descriptions, not the story IDs — these IDs are kept here only as a stable mapping to the archived epic specs (`archive/epics.md`). +Twenty-seven stories remain. Topic slugs in `planning/specs/` and `planning/plans/` use kebab-case descriptions, not the story IDs — these IDs are kept here only as a stable mapping to the archived epic specs (`archive/epics.md`). ### Epic 1 — Make typed HTTP requests with sensible defaults @@ -379,7 +379,7 @@ Twenty-seven stories remain. Topic slugs in `docs/superpowers/specs/` and `docs/ - **6-4** CI enforcement gates (codify the invariants in Section 2 as CI jobs). - **6-5** Release flow with Trusted Publishers + Sigstore. -When work starts on a roadmap item, it gets a superpowers spec at `docs/superpowers/specs/YYYY-MM-DD--design.md` and a plan at `docs/superpowers/plans/YYYY-MM-DD--plan.md`. The bmad-era 40KB story specs in `archive/stories/` cover 1-1 through 1-5 and are retired going forward. +When work starts on a roadmap item, it gets a superpowers spec at `planning/specs/YYYY-MM-DD--design.md` and a plan at `planning/plans/YYYY-MM-DD--plan.md`. The bmad-era 40KB story specs in `archive/stories/` cover 1-1 through 1-5 and are retired going forward. ``` - [ ] **Step 9: Write Section 9 — Deferred work pointer** @@ -394,23 +394,23 @@ Review-surfaced items that are real but not actionable now live in [`./deferred- - [ ] **Step 10: Verify the engineering doc** -Run: `wc -l docs/engineering.md` +Run: `wc -l docs/dev/engineering.md` Expected: between 250 and 350 lines. If significantly shorter, sections were skipped; if longer, tighten the prose. -Run: `grep -c "^## " docs/engineering.md` +Run: `grep -c "^## " docs/dev/engineering.md` Expected: `9` (nine top-level sections, one per spec requirement). -Run: `grep -n "TBD\|TODO\|XXX" docs/engineering.md` +Run: `grep -n "TBD\|TODO\|XXX" docs/dev/engineering.md` Expected: no matches. Run (sanity-check internal links): ```bash -grep -oE '\]\([^)]+\)' docs/engineering.md | sort -u +grep -oE '\]\([^)]+\)' docs/dev/engineering.md | sort -u ``` Expected: every relative link points to a file that exists. Specifically `../CLAUDE.md`, `./archive/`, `./deferred-work.md`, `archive/architecture.md` should all be reachable from `docs/`. Verify each: ```bash -ls ../CLAUDE.md docs/archive docs/deferred-work.md docs/archive/architecture.md +ls ../CLAUDE.md docs/archive planning/deferred-work.md docs/archive/architecture.md ``` Each should exist. @@ -450,19 +450,19 @@ Replace the entire `## Project Overview` section (between the heading and the st **Where to find what:** -- [`docs/engineering.md`](docs/engineering.md) — the distilled design reference: invariants and *why*, the five protocol seams, exception contract, module layout, testing patterns, optional-extras pattern, remaining roadmap. Read this before adding any new module or extension point. -- [`docs/deferred-work.md`](docs/deferred-work.md) — review-surfaced items that are real but not actionable now. -- [`docs/superpowers/specs/`](docs/superpowers/specs/) and [`docs/superpowers/plans/`](docs/superpowers/plans/) — per-feature design specs and implementation plans (active work). +- [`docs/dev/engineering.md`](docs/dev/engineering.md) — the distilled design reference: invariants and *why*, the five protocol seams, exception contract, module layout, testing patterns, optional-extras pattern, remaining roadmap. Read this before adding any new module or extension point. +- [`planning/deferred-work.md`](planning/deferred-work.md) — review-surfaced items that are real but not actionable now. +- [`planning/specs/`](planning/specs/) and [`planning/plans/`](planning/plans/) — per-feature design specs and implementation plans (active work). - [`docs/archive/`](docs/archive/) — historical bmad-era planning bundle (PRD, architecture, epics, product briefs, per-story specs for 1-1 through 1-5). Consult only for original rationale or specific FR/NFR citations. -**Per-feature workflow:** brainstorming → spec in `docs/superpowers/specs/` → writing-plans → plan in `docs/superpowers/plans/` → executing-plans (or subagent-driven-development) → requesting-code-review → finishing-a-development-branch. Topic slugs are kebab-case descriptions (`msgspec-decoder-adapter`), not story IDs. +**Per-feature workflow:** brainstorming → spec in `planning/specs/` → writing-plans → plan in `planning/plans/` → executing-plans (or subagent-driven-development) → requesting-code-review → finishing-a-development-branch. Topic slugs are kebab-case descriptions (`msgspec-decoder-adapter`), not story IDs. ``` The "Architecture invariants (CI-enforced)", "Code conventions", "Module layout", "Protocol seams", "Testing", "Commands", and "When in doubt" sections stay **verbatim**. These are operational rules and quick references that the AI applies directly; they are not duplicated in `engineering.md` (which holds the rationale, not the rules). - [ ] **Step 3: Update the "When in doubt" section** -The existing "When in doubt" section may reference `base-client/docs/architecture.md`. Update any such reference to `docs/engineering.md` (primary) and `docs/archive/architecture.md` (historical detail). +The existing "When in doubt" section may reference `base-client/docs/architecture.md`. Update any such reference to `docs/dev/engineering.md` (primary) and `docs/archive/architecture.md` (historical detail). Run: `grep -n "base-client/docs\|docs/prd.md\|docs/architecture.md\|docs/epics.md\|docs/stories\|product-brief" CLAUDE.md` @@ -476,7 +476,7 @@ Expected: roughly 105–120 lines (the original was 104; the new Project Overvie Run: `grep -c "^## " CLAUDE.md` Expected: matches the original section count (no sections lost). The original had Project Overview, Commands, Architecture invariants (CI-enforced), Code conventions, Module layout, Protocol seams, Testing, When in doubt — count `8`. -Run: `grep -n "docs/engineering.md\|docs/archive" CLAUDE.md` +Run: `grep -n "docs/dev/engineering.md\|docs/archive" CLAUDE.md` Expected: multiple matches in the new Project Overview section. --- @@ -538,8 +538,8 @@ R docs/stories/1-4-transport-protocol-and-httpx2transport-adapter.md -> docs/ar R docs/stories/1-5-responsedecoder-protocol-and-pydantic-adapter.md -> docs/archive/stories/1-5-responsedecoder-protocol-and-pydantic-adapter.md R docs/stories/sprint-status.yaml -> docs/archive/stories/sprint-status.yaml ?? docs/archive/README.md # untracked -?? docs/engineering.md -?? docs/superpowers/plans/2026-05-31-bmad-to-superpowers-transition-plan.md +?? docs/dev/engineering.md +?? planning/plans/2026-05-31-bmad-to-superpowers-transition-plan.md ``` No `.review-tmp/` anywhere. No source/`pyproject.toml`/CI changes. @@ -550,7 +550,7 @@ If any line begins with `??` for a file not in the list above, or any source fil Run: `git add CLAUDE.md docs/` -(`git add docs/` picks up the new `docs/archive/README.md`, `docs/engineering.md`, and `docs/superpowers/plans/2026-05-31-...-plan.md` in one step.) +(`git add docs/` picks up the new `docs/archive/README.md`, `docs/dev/engineering.md`, and `planning/plans/2026-05-31-...-plan.md` in one step.) Re-run: `git status --short` Expected: every line begins with a capital letter (`A`, `M`, or `R`) in column 1 and a space in column 2 (everything staged, nothing left in the working tree). @@ -563,7 +563,7 @@ git commit -m "$(cat <<'EOF' chore: cutover from bmad to superpowers workflow Distills the bmad-era PRD / architecture / epics into a single -docs/engineering.md (~300 lines) covering project intent, the six +docs/dev/engineering.md (~300 lines) covering project intent, the six CI-enforced invariants with rationale, the five protocol seams, the exception contract, current + planned module layout, testing patterns, the optional-extras pattern, the remaining 27-story @@ -606,21 +606,21 @@ Run: gh pr create --title "chore: cutover from bmad to superpowers workflow" --body "$(cat <<'EOF' ## Summary -- Distills the bmad-era PRD / architecture / epics into a single `docs/engineering.md` (~300 lines): project intent, six CI-enforced invariants with rationale, the five protocol seams, exception contract, module layout, testing patterns, optional-extras pattern, remaining 27-story roadmap, deferred-work pointer. +- Distills the bmad-era PRD / architecture / epics into a single `docs/dev/engineering.md` (~300 lines): project intent, six CI-enforced invariants with rationale, the five protocol seams, exception contract, module layout, testing patterns, optional-extras pattern, remaining 27-story roadmap, deferred-work pointer. - Moves `prd.md`, `architecture.md`, `epics.md`, both product briefs, and `docs/stories/` (1-1 through 1-5 + `sprint-status.yaml`) into `docs/archive/` with a framing README. - Rewrites `CLAUDE.md` "Project Overview" to point at `engineering.md` and the new `docs/superpowers/{specs,plans}/` flow. Operational sections (invariants, conventions, module layout, seams, testing, commands, when-in-doubt) are kept verbatim. - Deletes the `.review-tmp/` bmad code-review artifact dump. -After this lands, future per-feature work uses the superpowers flow: brainstorming → spec in `docs/superpowers/specs/` → writing-plans → plan in `docs/superpowers/plans/` → executing-plans (or subagent-driven-development) → requesting-code-review → finishing-a-development-branch. Story IDs are retired as topic slugs; the bmad backlog mapping lives in `engineering.md`'s roadmap section. +After this lands, future per-feature work uses the superpowers flow: brainstorming → spec in `planning/specs/` → writing-plans → plan in `planning/plans/` → executing-plans (or subagent-driven-development) → requesting-code-review → finishing-a-development-branch. Story IDs are retired as topic slugs; the bmad backlog mapping lives in `engineering.md`'s roadmap section. -The driving design + plan for this PR are at `docs/superpowers/specs/2026-05-31-bmad-to-superpowers-transition-design.md` and `docs/superpowers/plans/2026-05-31-bmad-to-superpowers-transition-plan.md`. +The driving design + plan for this PR are at `planning/specs/2026-05-31-bmad-to-superpowers-transition-design.md` and `planning/plans/2026-05-31-bmad-to-superpowers-transition-plan.md`. ## Test plan - [x] No source or CI changes; the 157-test suite continues to pass at 100% coverage on the branch. - [x] `just lint-ci` clean. - [x] `git status --short` shows only renames, three new docs files, one modified `CLAUDE.md`, and the `.review-tmp/` deletion. -- [x] All internal links in `docs/engineering.md` and `CLAUDE.md` resolve to files that exist post-move. +- [x] All internal links in `docs/dev/engineering.md` and `CLAUDE.md` resolve to files that exist post-move. - [ ] CI green on the PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code) @@ -652,9 +652,9 @@ The cutover is complete. Task 1 (retrospective code review) is the next normal-f ## Definition of done -- `docs/engineering.md` exists (250–350 lines) and is referenced from `CLAUDE.md`. +- `docs/dev/engineering.md` exists (250–350 lines) and is referenced from `CLAUDE.md`. - `docs/archive/` contains the six top-level moves plus the `stories/` directory and a framing `README.md`. -- `docs/superpowers/specs/2026-05-31-bmad-to-superpowers-transition-design.md` and `docs/superpowers/plans/2026-05-31-bmad-to-superpowers-transition-plan.md` are committed. +- `planning/specs/2026-05-31-bmad-to-superpowers-transition-design.md` and `planning/plans/2026-05-31-bmad-to-superpowers-transition-plan.md` are committed. - `.review-tmp/` is gone. - `CLAUDE.md` no longer references `docs/prd.md`, `docs/architecture.md`, `docs/epics.md`, `docs/stories/`, or the product briefs at their original paths. - The cutover commit is a single commit on `chore/bmad-to-superpowers-transition`, merged to `main` via one PR. diff --git a/docs/superpowers/plans/2026-05-31-middleware-protocol-and-chain-plan.md b/planning/plans/2026-05-31-middleware-protocol-and-chain-plan.md similarity index 96% rename from docs/superpowers/plans/2026-05-31-middleware-protocol-and-chain-plan.md rename to planning/plans/2026-05-31-middleware-protocol-and-chain-plan.md index 51870e8..6330762 100644 --- a/docs/superpowers/plans/2026-05-31-middleware-protocol-and-chain-plan.md +++ b/planning/plans/2026-05-31-middleware-protocol-and-chain-plan.md @@ -10,7 +10,7 @@ **Branch:** `story/2-1-middleware-protocol-and-chain` (already created; the spec commit is on it). -**Spec:** `docs/superpowers/specs/2026-05-31-middleware-protocol-and-chain-design.md`. +**Spec:** `planning/specs/2026-05-31-middleware-protocol-and-chain-design.md`. --- @@ -748,15 +748,15 @@ Run: `git log --oneline main..HEAD` Expected: five or six commits — the spec commit (`docs(story-2.1): design...`), Task 1, Task 2, Task 3, Task 4, Task 5. Run: `git diff --stat main..HEAD` -Expected: changes to `CHANGELOG.md`, `docs/superpowers/specs/2026-05-31-middleware-protocol-and-chain-design.md`, `docs/superpowers/plans/2026-05-31-middleware-protocol-and-chain-plan.md`, `src/httpware/__init__.py`, two new files under `src/httpware/_internal/`, one new file under `src/httpware/middleware/`, and `tests/test_middleware.py`. No source files outside this scope should be touched. +Expected: changes to `CHANGELOG.md`, `planning/specs/2026-05-31-middleware-protocol-and-chain-design.md`, `planning/plans/2026-05-31-middleware-protocol-and-chain-plan.md`, `src/httpware/__init__.py`, two new files under `src/httpware/_internal/`, one new file under `src/httpware/middleware/`, and `tests/test_middleware.py`. No source files outside this scope should be touched. - [ ] **Step 5: Stage and commit the plan file** -The plan file at `docs/superpowers/plans/2026-05-31-middleware-protocol-and-chain-plan.md` is still untracked (it was created during the writing-plans step but not yet committed). Stage and commit it on this branch so the merge captures the plan alongside the spec. +The plan file at `planning/plans/2026-05-31-middleware-protocol-and-chain-plan.md` is still untracked (it was created during the writing-plans step but not yet committed). Stage and commit it on this branch so the merge captures the plan alongside the spec. Run: ```bash -git add docs/superpowers/plans/2026-05-31-middleware-protocol-and-chain-plan.md +git add planning/plans/2026-05-31-middleware-protocol-and-chain-plan.md git commit -m "docs(story-2.1): implementation plan for Middleware protocol and chain Co-Authored-By: Claude Opus 4.7 (1M context) " @@ -780,7 +780,7 @@ gh pr create --title "feat(story-2.1): Middleware protocol, Next type, and chain Out of scope (subsequent stories): phase decorators (2-2), Request immutability helpers beyond what already exists (2-3), auth coercion (2-4), AsyncClient wiring (2-5), streaming chain (4-3). -Spec + plan: `docs/superpowers/specs/2026-05-31-middleware-protocol-and-chain-design.md`, `docs/superpowers/plans/2026-05-31-middleware-protocol-and-chain-plan.md`. +Spec + plan: `planning/specs/2026-05-31-middleware-protocol-and-chain-design.md`, `planning/plans/2026-05-31-middleware-protocol-and-chain-plan.md`. ## Test plan diff --git a/docs/superpowers/plans/2026-05-31-msgspec-decoder-via-extras-plan.md b/planning/plans/2026-05-31-msgspec-decoder-via-extras-plan.md similarity index 95% rename from docs/superpowers/plans/2026-05-31-msgspec-decoder-via-extras-plan.md rename to planning/plans/2026-05-31-msgspec-decoder-via-extras-plan.md index 2330afe..b8542a1 100644 --- a/docs/superpowers/plans/2026-05-31-msgspec-decoder-via-extras-plan.md +++ b/planning/plans/2026-05-31-msgspec-decoder-via-extras-plan.md @@ -10,7 +10,7 @@ **Branch:** `story/1-6-msgspec-decoder-via-extras` (already created; spec commit `b12a989` is on it). -**Spec:** `docs/superpowers/specs/2026-05-31-msgspec-decoder-via-extras-design.md`. +**Spec:** `planning/specs/2026-05-31-msgspec-decoder-via-extras-design.md`. --- @@ -386,7 +386,7 @@ Expected: `eof-fixer`, `ruff format --check`, `ruff check --no-fix`, `ty check` - [ ] **Step 3: Confirm the working tree is clean** Run: `git status --short` -Expected: only the untracked plan file `docs/superpowers/plans/2026-05-31-msgspec-decoder-via-extras-plan.md`. +Expected: only the untracked plan file `planning/plans/2026-05-31-msgspec-decoder-via-extras-plan.md`. - [ ] **Step 4: Review the branch diff** @@ -394,12 +394,12 @@ Run: `git log --oneline main..HEAD` Expected: five or six commits — spec (`docs(story-1.6): design...`), Task 1, Task 2, Task 3, Task 4. Run: `git diff --stat main..HEAD` -Expected: changes to `CHANGELOG.md`, plus the four new files: `docs/superpowers/specs/2026-05-31-msgspec-decoder-via-extras-design.md`, `src/httpware/_internal/import_checker.py`, `src/httpware/decoders/msgspec.py`, `tests/test_decoders_msgspec.py`, `tests/test_optional_extras_isolation.py`. No other source files touched. +Expected: changes to `CHANGELOG.md`, plus the four new files: `planning/specs/2026-05-31-msgspec-decoder-via-extras-design.md`, `src/httpware/_internal/import_checker.py`, `src/httpware/decoders/msgspec.py`, `tests/test_decoders_msgspec.py`, `tests/test_optional_extras_isolation.py`. No other source files touched. - [ ] **Step 5: Stage and commit the plan file** ```bash -git add docs/superpowers/plans/2026-05-31-msgspec-decoder-via-extras-plan.md +git add planning/plans/2026-05-31-msgspec-decoder-via-extras-plan.md git commit -m "docs(story-1.6): implementation plan for MsgspecDecoder via extras Co-Authored-By: Claude Opus 4.7 (1M context) " @@ -425,7 +425,7 @@ gh pr create --title "feat(story-1.6): MsgspecDecoder via the [msgspec] extra" - Out of scope (subsequent stories): `AsyncClient` wiring (Story 1-7), `RecordedTransport` (Story 1-8), follow-up cleanup of legacy `__all__` exports in existing submodules. -Spec + plan: `docs/superpowers/specs/2026-05-31-msgspec-decoder-via-extras-design.md`, `docs/superpowers/plans/2026-05-31-msgspec-decoder-via-extras-plan.md`. +Spec + plan: `planning/specs/2026-05-31-msgspec-decoder-via-extras-design.md`, `planning/plans/2026-05-31-msgspec-decoder-via-extras-plan.md`. ## Test plan diff --git a/docs/superpowers/plans/2026-05-31-phase-shortcut-decorators-plan.md b/planning/plans/2026-05-31-phase-shortcut-decorators-plan.md similarity index 94% rename from docs/superpowers/plans/2026-05-31-phase-shortcut-decorators-plan.md rename to planning/plans/2026-05-31-phase-shortcut-decorators-plan.md index 5c09a4d..5a0d92e 100644 --- a/docs/superpowers/plans/2026-05-31-phase-shortcut-decorators-plan.md +++ b/planning/plans/2026-05-31-phase-shortcut-decorators-plan.md @@ -10,7 +10,7 @@ **Branch:** `story/2-2-phase-shortcut-decorators` (already created; spec commit `6cfc9fa` is on it). -**Spec:** `docs/superpowers/specs/2026-05-31-phase-shortcut-decorators-design.md`. +**Spec:** `planning/specs/2026-05-31-phase-shortcut-decorators-design.md`. --- @@ -19,7 +19,7 @@ **Modified files:** - `src/httpware/middleware/__init__.py` — append three factory functions (~65 lines added; file grows from 30 to ~95 lines). Update `__all__`. - `src/httpware/__init__.py` — import and re-export `before_request`, `after_response`, `on_error`. Update `__all__`. -- `docs/engineering.md` — fix line 145 stale decorator names. +- `docs/dev/engineering.md` — fix line 145 stale decorator names. - `CHANGELOG.md` — append Story 2.2 bullet under `[Unreleased]` / `### Added`. - `tests/test_middleware.py` — append 10 new tests (file grows from 14 → 24 tests). @@ -519,7 +519,7 @@ Wire the three decorators into the package root, fix the stale naming in `engine **Files:** - Modify: `src/httpware/__init__.py` -- Modify: `docs/engineering.md` +- Modify: `docs/dev/engineering.md` - Modify: `CHANGELOG.md` - Modify: `tests/test_middleware.py` (append 1 re-export test) @@ -573,7 +573,7 @@ Expected: PASS. - [ ] **Step 5: Fix the engineering.md roadmap line** -Edit `docs/engineering.md` line 145. The current text reads: +Edit `docs/dev/engineering.md` line 145. The current text reads: ``` - **2-2** Phase shortcut decorators (`@on_request`, `@on_response`, `@on_error`). @@ -604,7 +604,7 @@ Expected: All checks passed. - [ ] **Step 8: Commit** ```bash -git add src/httpware/__init__.py docs/engineering.md CHANGELOG.md tests/test_middleware.py +git add src/httpware/__init__.py docs/dev/engineering.md CHANGELOG.md tests/test_middleware.py git commit -m "$(cat <<'EOF' feat(story-2.2): re-export decorators; fix engineering.md naming; CHANGELOG @@ -612,7 +612,7 @@ Adds before_request, after_response, on_error to httpware/__init__.py imports and __all__ so consumers can `from httpware import …` in addition to the subpackage path. -Fixes docs/engineering.md §8 line 145 to reflect the canonical +Fixes docs/dev/engineering.md §8 line 145 to reflect the canonical @before_request / @after_response / @on_error names (it had stale @on_request / @on_response from the distillation). @@ -652,14 +652,14 @@ Run: `git log --oneline main..HEAD` Expected: six or seven commits — spec (`docs(story-2.2): design...`), Task 1, Task 2, Task 3, Task 4, Task 5. Run: `git diff --stat main..HEAD` -Expected: changes to `CHANGELOG.md`, `docs/engineering.md`, `docs/superpowers/specs/2026-05-31-phase-shortcut-decorators-design.md`, `docs/superpowers/plans/2026-05-31-phase-shortcut-decorators-plan.md`, `src/httpware/__init__.py`, `src/httpware/middleware/__init__.py`, `tests/test_middleware.py`. No other source files touched. +Expected: changes to `CHANGELOG.md`, `docs/dev/engineering.md`, `planning/specs/2026-05-31-phase-shortcut-decorators-design.md`, `planning/plans/2026-05-31-phase-shortcut-decorators-plan.md`, `src/httpware/__init__.py`, `src/httpware/middleware/__init__.py`, `tests/test_middleware.py`. No other source files touched. - [ ] **Step 5: Stage and commit the plan file** -The plan file at `docs/superpowers/plans/2026-05-31-phase-shortcut-decorators-plan.md` is still untracked. Stage and commit it on this branch so the merge captures the plan alongside the spec. +The plan file at `planning/plans/2026-05-31-phase-shortcut-decorators-plan.md` is still untracked. Stage and commit it on this branch so the merge captures the plan alongside the spec. ```bash -git add docs/superpowers/plans/2026-05-31-phase-shortcut-decorators-plan.md +git add planning/plans/2026-05-31-phase-shortcut-decorators-plan.md git commit -m "docs(story-2.2): implementation plan for phase-shortcut decorators Co-Authored-By: Claude Opus 4.7 (1M context) " @@ -684,11 +684,11 @@ gh pr create --title "feat(story-2.2): phase-shortcut decorators @before_request - All three exported at both `httpware.middleware.*` and `httpware.*`. - 10 new tests in `tests/test_middleware.py` (24 total): request and response transformations, on_error swallow/re-raise paths, `CancelledError` non-capture, exception identity, Protocol satisfaction, mixed-chain composition, `repr()` content, package-root re-export. -Bundled-in doc fix: `docs/engineering.md` §8 line 145 had stale `@on_request`/`@on_response` names from the distillation — corrected to the canonical `@before_request`/`@after_response`. +Bundled-in doc fix: `docs/dev/engineering.md` §8 line 145 had stale `@on_request`/`@on_response` names from the distillation — corrected to the canonical `@before_request`/`@after_response`. Out of scope (subsequent stories): `Request.with_*` helper expansion (2-3), auth coercion (2-4), AsyncClient wiring (2-5). -Spec + plan: `docs/superpowers/specs/2026-05-31-phase-shortcut-decorators-design.md`, `docs/superpowers/plans/2026-05-31-phase-shortcut-decorators-plan.md`. +Spec + plan: `planning/specs/2026-05-31-phase-shortcut-decorators-design.md`, `planning/plans/2026-05-31-phase-shortcut-decorators-plan.md`. ## Test plan @@ -730,7 +730,7 @@ Story 2-2 is complete. Story 2-3 (`Request` immutability helper expansion) is th - `src/httpware/middleware/__init__.py` exports `before_request`, `after_response`, `on_error` in addition to `Middleware` and `Next`. - `src/httpware/__init__.py` re-exports the three new names and adds them to `__all__` in alphabetic position (after `"UnprocessableEntityError"`). -- `docs/engineering.md` §8 line 145 reads `@before_request`, `@after_response`, `@on_error` — the stale `@on_request`/`@on_response` is gone. +- `docs/dev/engineering.md` §8 line 145 reads `@before_request`, `@after_response`, `@on_error` — the stale `@on_request`/`@on_response` is gone. - `CHANGELOG.md` has a Story 2.2 bullet under `[Unreleased]` / `### Added`. - `tests/test_middleware.py` contains 24 tests (14 carried forward from Story 2-1 + 10 new); all pass. - `just test` shows 184 passed, 1 deselected, 100% line coverage. diff --git a/docs/superpowers/plans/2026-05-31-recordedtransport-plan.md b/planning/plans/2026-05-31-recordedtransport-plan.md similarity index 98% rename from docs/superpowers/plans/2026-05-31-recordedtransport-plan.md rename to planning/plans/2026-05-31-recordedtransport-plan.md index f9221b1..3c76339 100644 --- a/docs/superpowers/plans/2026-05-31-recordedtransport-plan.md +++ b/planning/plans/2026-05-31-recordedtransport-plan.md @@ -10,7 +10,7 @@ **Branch:** `story/1-8-recordedtransport` (already created; spec commit `60d2e0c` is on it). -**Spec:** `docs/superpowers/specs/2026-05-31-recordedtransport-design.md`. +**Spec:** `planning/specs/2026-05-31-recordedtransport-design.md`. --- @@ -956,7 +956,7 @@ Expected: `eof-fixer`, `ruff format --check`, `ruff check --no-fix`, `ty check` - [ ] **Step 3: Confirm the working tree is clean** Run: `git status --short` -Expected: only the untracked plan file `docs/superpowers/plans/2026-05-31-recordedtransport-plan.md`. +Expected: only the untracked plan file `planning/plans/2026-05-31-recordedtransport-plan.md`. - [ ] **Step 4: Review the branch diff** @@ -969,7 +969,7 @@ Expected: new files `src/httpware/transports/recorded.py`, `tests/test_transport - [ ] **Step 5: Stage and commit the plan file** ```bash -git add docs/superpowers/plans/2026-05-31-recordedtransport-plan.md +git add planning/plans/2026-05-31-recordedtransport-plan.md git commit -m "docs(story-1.8): implementation plan for RecordedTransport Co-Authored-By: Claude Opus 4.7 (1M context) " @@ -996,7 +996,7 @@ This closes Epic 1. Out of scope (subsequent stories): URL pattern matching / globs, cassette files loaded from JSON, streaming responses (Epic 4). -Spec + plan: `docs/superpowers/specs/2026-05-31-recordedtransport-design.md`, `docs/superpowers/plans/2026-05-31-recordedtransport-plan.md`. +Spec + plan: `planning/specs/2026-05-31-recordedtransport-design.md`, `planning/plans/2026-05-31-recordedtransport-plan.md`. ## Test plan diff --git a/docs/superpowers/plans/2026-05-31-release-0.1.0-prep-plan.md b/planning/plans/2026-05-31-release-0.1.0-prep-plan.md similarity index 96% rename from docs/superpowers/plans/2026-05-31-release-0.1.0-prep-plan.md rename to planning/plans/2026-05-31-release-0.1.0-prep-plan.md index b79621d..c64745e 100644 --- a/docs/superpowers/plans/2026-05-31-release-0.1.0-prep-plan.md +++ b/planning/plans/2026-05-31-release-0.1.0-prep-plan.md @@ -10,7 +10,7 @@ **Branch:** `chore/release-0.1.0-prep` (already created; spec commit `eecd041` is on it). -**Spec:** `docs/superpowers/specs/2026-05-31-release-0.1.0-prep-design.md`. +**Spec:** `planning/specs/2026-05-31-release-0.1.0-prep-design.md`. --- @@ -32,7 +32,7 @@ - `.github/workflows/ci.yml` — test/lint matrix unchanged. - `src/httpware/**`, `tests/**` — no code or test changes. - `LICENSE`, `SECURITY.md`, `CLAUDE.md` — unchanged (CLAUDE.md verified to have no CHANGELOG references). -- `docs/engineering.md` — internal roadmap stays; "Released" markers are a follow-up. +- `docs/dev/engineering.md` — internal roadmap stays; "Released" markers are a follow-up. - `docs/archive/**` — bmad archive untouched. --- @@ -48,7 +48,7 @@ Remove the changelog file and the one CONTRIBUTING.md bullet that referenced it. - [ ] **Step 1: Verify the current state** Run: `git status --short && git branch --show-current` -Expected: branch is `chore/release-0.1.0-prep`. Working tree has the untracked plan file `docs/superpowers/plans/2026-05-31-release-0.1.0-prep-plan.md` (this file) and nothing else. +Expected: branch is `chore/release-0.1.0-prep`. Working tree has the untracked plan file `planning/plans/2026-05-31-release-0.1.0-prep-plan.md` (this file) and nothing else. Run: `/usr/bin/grep -in "changelog" CONTRIBUTING.md CLAUDE.md README.md` Expected output: @@ -76,7 +76,7 @@ Inspect the file first (`uv run head -50 CONTRIBUTING.md` or just read it), then - [ ] **Step 4: Verify no CHANGELOG references remain anywhere except docs/archive/** Run: `/usr/bin/grep -rIn "changelog" --include="*.md" --include="*.toml" --include="*.yml" --include="*.yaml" . --exclude-dir=docs/archive --exclude-dir=.venv --exclude-dir=.git --exclude-dir=docs/superpowers` -Expected: no matches (or only matches inside `docs/superpowers/specs/2026-05-31-release-0.1.0-prep-design.md` describing this very deletion, which is fine). +Expected: no matches (or only matches inside `planning/specs/2026-05-31-release-0.1.0-prep-design.md` describing this very deletion, which is fine). If `README.md` still has a CHANGELOG reference, leave it — Task 3 rewrites the README entirely. @@ -294,7 +294,7 @@ Expected: clean. - [ ] **Step 3: Confirm the working tree state** Run: `git status --short` -Expected: only the untracked plan file `docs/superpowers/plans/2026-05-31-release-0.1.0-prep-plan.md`. +Expected: only the untracked plan file `planning/plans/2026-05-31-release-0.1.0-prep-plan.md`. Run: `git log --oneline main..HEAD` Expected: four commits — spec (`docs(release): design for 0.1.0 release prep`), Task 1 (`chore(release): delete CHANGELOG.md…`), Task 2 (`ci(release): add publish.yml…`), Task 3 (`docs(release): trim README…`). @@ -305,7 +305,7 @@ Expected: `CHANGELOG.md` deleted, `CONTRIBUTING.md` modified (~1 line), `.github - [ ] **Step 4: Stage and commit the plan file** ```bash -git add docs/superpowers/plans/2026-05-31-release-0.1.0-prep-plan.md +git add planning/plans/2026-05-31-release-0.1.0-prep-plan.md git commit -m "docs(release): implementation plan for 0.1.0 release prep Co-Authored-By: Claude Opus 4.7 (1M context) " @@ -336,7 +336,7 @@ After this PR merges, the maintainer: 2. Verifies `PYPI_TOKEN` is present in repo secrets. 3. Creates a GitHub Release tagged `0.1.0`, which triggers `publish.yml` → `just publish` → PyPI. -Spec + plan: `docs/superpowers/specs/2026-05-31-release-0.1.0-prep-design.md`, `docs/superpowers/plans/2026-05-31-release-0.1.0-prep-plan.md`. +Spec + plan: `planning/specs/2026-05-31-release-0.1.0-prep-design.md`, `planning/plans/2026-05-31-release-0.1.0-prep-plan.md`. ## Test plan diff --git a/docs/superpowers/plans/2026-05-31-request-immutability-helpers-plan.md b/planning/plans/2026-05-31-request-immutability-helpers-plan.md similarity index 96% rename from docs/superpowers/plans/2026-05-31-request-immutability-helpers-plan.md rename to planning/plans/2026-05-31-request-immutability-helpers-plan.md index 3ab1141..6eb95a3 100644 --- a/docs/superpowers/plans/2026-05-31-request-immutability-helpers-plan.md +++ b/planning/plans/2026-05-31-request-immutability-helpers-plan.md @@ -10,7 +10,7 @@ **Branch:** `story/2-3-request-immutability-helpers` (already created; spec commit `5bcf9a4` is on it). -**Spec:** `docs/superpowers/specs/2026-05-31-request-immutability-helpers-design.md`. +**Spec:** `planning/specs/2026-05-31-request-immutability-helpers-design.md`. --- @@ -460,7 +460,7 @@ Expected: `eof-fixer`, `ruff format --check`, `ruff check --no-fix`, `ty check` - [ ] **Step 3: Confirm the working tree is clean** Run: `git status --short` -Expected: only the untracked plan file `docs/superpowers/plans/2026-05-31-request-immutability-helpers-plan.md`. +Expected: only the untracked plan file `planning/plans/2026-05-31-request-immutability-helpers-plan.md`. - [ ] **Step 4: Review the branch diff** @@ -468,12 +468,12 @@ Run: `git log --oneline main..HEAD` Expected: six or seven commits — the spec commit (`docs(story-2.3): design...`), Task 1, Task 2, Task 3, Task 4, Task 5. Run: `git diff --stat main..HEAD` -Expected: changes to `CHANGELOG.md`, `docs/superpowers/specs/2026-05-31-request-immutability-helpers-design.md`, `src/httpware/request.py`, `src/httpware/response.py`, `tests/test_request.py`, `tests/test_response.py`. No other files touched. +Expected: changes to `CHANGELOG.md`, `planning/specs/2026-05-31-request-immutability-helpers-design.md`, `src/httpware/request.py`, `src/httpware/response.py`, `tests/test_request.py`, `tests/test_response.py`. No other files touched. - [ ] **Step 5: Stage and commit the plan file** ```bash -git add docs/superpowers/plans/2026-05-31-request-immutability-helpers-plan.md +git add planning/plans/2026-05-31-request-immutability-helpers-plan.md git commit -m "docs(story-2.3): implementation plan for immutability helpers Co-Authored-By: Claude Opus 4.7 (1M context) " @@ -498,7 +498,7 @@ gh pr create --title "feat(story-2.3): Request/Response immutability helper expa Out of scope (subsequent stories): auth coercion (2-4), AsyncClient wiring (2-5), \`StreamResponse.with_*\` (Story 4-1), case-insensitive header keys (existing deferred-work entry). -Spec + plan: \`docs/superpowers/specs/2026-05-31-request-immutability-helpers-design.md\`, \`docs/superpowers/plans/2026-05-31-request-immutability-helpers-plan.md\`. +Spec + plan: \`planning/specs/2026-05-31-request-immutability-helpers-design.md\`, \`planning/plans/2026-05-31-request-immutability-helpers-plan.md\`. ## Test plan diff --git a/docs/superpowers/plans/2026-06-01-auth-coercion-plan.md b/planning/plans/2026-06-01-auth-coercion-plan.md similarity index 98% rename from docs/superpowers/plans/2026-06-01-auth-coercion-plan.md rename to planning/plans/2026-06-01-auth-coercion-plan.md index d7683fd..d41b281 100644 --- a/docs/superpowers/plans/2026-06-01-auth-coercion-plan.md +++ b/planning/plans/2026-06-01-auth-coercion-plan.md @@ -10,7 +10,7 @@ **Branch:** `story/2-4-auth-coercion` (already created; spec commit `5906d22` is on it). -**Spec:** `docs/superpowers/specs/2026-06-01-auth-coercion-design.md`. +**Spec:** `planning/specs/2026-06-01-auth-coercion-design.md`. --- @@ -924,7 +924,7 @@ Expected: `eof-fixer`, `ruff format --check`, `ruff check --no-fix`, `ty check` - [ ] **Step 3: Confirm the working tree** Run: `git status --short` -Expected: only the untracked plan file `docs/superpowers/plans/2026-06-01-auth-coercion-plan.md`. +Expected: only the untracked plan file `planning/plans/2026-06-01-auth-coercion-plan.md`. Run: `git log --oneline main..HEAD` Expected: six commits — spec, Task 1, Task 2, Task 3, Task 4, Task 5. @@ -932,7 +932,7 @@ Expected: six commits — spec, Task 1, Task 2, Task 3, Task 4, Task 5. - [ ] **Step 4: Stage and commit the plan** ```bash -git add docs/superpowers/plans/2026-06-01-auth-coercion-plan.md +git add planning/plans/2026-06-01-auth-coercion-plan.md git commit -m "docs(story-2.4): implementation plan for auth coercion Co-Authored-By: Claude Opus 4.7 (1M context) " @@ -960,7 +960,7 @@ gh pr create --title "feat(story-2.4): auth coercion as middleware" --body "$(ca Out of scope: OAuth2 / refresh tokens, mTLS, signature schemes (HMAC, AWS Sigv4), per-call \`auth=\` override on HTTP methods. -Spec + plan: \`docs/superpowers/specs/2026-06-01-auth-coercion-design.md\`, \`docs/superpowers/plans/2026-06-01-auth-coercion-plan.md\`. +Spec + plan: \`planning/specs/2026-06-01-auth-coercion-design.md\`, \`planning/plans/2026-06-01-auth-coercion-plan.md\`. ## Test plan diff --git a/planning/plans/2026-06-02-docs-reorg-and-mkdocs-plan.md b/planning/plans/2026-06-02-docs-reorg-and-mkdocs-plan.md new file mode 100644 index 0000000..f430dff --- /dev/null +++ b/planning/plans/2026-06-02-docs-reorg-and-mkdocs-plan.md @@ -0,0 +1,779 @@ +# Docs reorg + minimal mkdocs site implementation plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Delete `docs/archive/`, move agent/contributor workflow artifacts to a tool-neutral `planning/` directory at the repo root, and stand up a minimal mkdocs site published via Read the Docs — without breaking any internal markdown link, the CI lint/test pipeline, or git history for moved files. + +**Architecture:** Pure structural reorganization. Eleven atomic-commit tasks executed in dependency order: (1) preserve the one load-bearing archive citation by inlining it into engineering.md, (2) move workflow artifacts via `git mv` so history follows, (3) move engineering docs into a `docs/dev/` subtree that the mkdocs site will publish, (4) bulk-update path references across CLAUDE.md and every existing spec/plan, (5) refactor README.md to align with the modern-python org pattern, (6) add the mkdocs/RTD config files, (7) delete the archive, (8) verify everything builds and lints. No source code touched. No CI grep invariants change. The `src/` layout is unchanged. + +**Tech Stack:** `mkdocs` + `mkdocs-material` (Read the Docs published), `uv`, `just`, `git mv` for history preservation. + +--- + +## Pre-flight + +Plan assumes you are on a clean working tree at the spec's current commit (`a2abca4` or descendant). Verify before starting: + +```bash +git status # should be clean +git rev-parse HEAD # record starting commit for sanity +``` + +The spec lives at `docs/superpowers/specs/2026-06-02-docs-reorg-and-mkdocs-design.md` — read it once if you haven't. + +--- + +### Task 1: Inline the load-bearing decoder rationale into engineering.md + +**Goal:** Before `archive/` is deleted, port the one rationale that `engineering.md` externalizes — the "two-pass decoding is rejected" reasoning at Seam 3 — into `engineering.md` itself. This must happen first so nothing is lost when archive is removed in Task 10. + +**Files:** +- Modify: `docs/engineering.md` line 40 (Seam 3 "Rule" bullet) + +**Reference (source of inlined text):** `docs/archive/architecture.md` lines 270–283 ("Decision 8 — ResponseDecoder protocol"). + +- [ ] **Step 1: Read the archive section once** + +Run: `sed -n '270,283p' docs/archive/architecture.md` + +Confirm you see the "ResponseDecoder protocol" decision: single parse pass (NFR3), `TypeAdapter.validate_json` with `lru_cache` (NFR2), `msgspec.json.decode` for the extras adapter. + +- [ ] **Step 2: Edit engineering.md line 40** + +Replace this exact line in `docs/engineering.md`: + +```markdown +- **Rule:** the decoder must operate on raw bytes in a single parse pass. Two-pass decoding (`json.loads` then `validate_python`) is rejected — see `archive/architecture.md` Validation & Decoding for rationale. +``` + +With: + +```markdown +- **Rule:** the decoder must operate on raw bytes in a single parse pass. Two-pass decoding (`json.loads` then `validate_python`) is rejected: a single bytes-in / typed-object-out pass avoids the redundant intermediate `dict` allocation and parses faster. The Pydantic adapter implements this as `TypeAdapter(model).validate_json(content)` with `@functools.lru_cache(maxsize=None)` on `TypeAdapter` construction (the adapter object is the expensive part to build, keyed by `model`). The msgspec adapter implements it as `msgspec.json.decode(content, type=model)`. +``` + +- [ ] **Step 3: Confirm no other archive citation is load-bearing** + +Run: `grep -n "archive/" docs/engineering.md` + +Expected output (exactly three remaining lines): +``` +3:This doc is the single distilled reference for `httpware` design rationale, protocol seams, and remaining roadmap. ... +134:Twenty-seven stories remain. Topic slugs in `docs/superpowers/specs/` and `docs/superpowers/plans/` use kebab-case descriptions, not the story IDs — these IDs are kept here only as a stable mapping to the archived epic specs (`archive/epics.md`). +181:When work starts on a roadmap item, it gets a superpowers spec at `docs/superpowers/specs/YYYY-MM-DD--design.md` and a plan at `docs/superpowers/plans/YYYY-MM-DD--plan.md`. The bmad-era 40KB story specs in `archive/stories/` cover 1-1 through 1-5 and are retired going forward. +``` + +These three remaining references are *soft* — Task 2 strips them. Line 40 no longer mentions archive: verify with `grep -n "archive/architecture.md" docs/engineering.md` returning **no output**. + +- [ ] **Step 4: Commit** + +```bash +git add docs/engineering.md +git commit -m "docs: inline decoder rationale before archive deletion" +``` + +--- + +### Task 2: Strip remaining soft archive references from engineering.md + +**Goal:** Remove the three remaining `archive/` mentions (lines 3, 134, 181) and any pointer that will be broken when archive is deleted. These are informational, not load-bearing; they just get deleted/rephrased. + +**Files:** +- Modify: `docs/engineering.md` (three edits) + +- [ ] **Step 1: Edit line 3 — drop the archive pointer sentence** + +Replace this exact line in `docs/engineering.md`: + +```markdown +This doc is the single distilled reference for `httpware` design rationale, protocol seams, and remaining roadmap. It complements [`../CLAUDE.md`](../CLAUDE.md): `CLAUDE.md` holds AI-enforced invariants and operational commands; this file holds the reasoning and the structural map. Historical planning artifacts live in [`archive/`](./archive/) and are cited only for original rationale. +``` + +With: + +```markdown +This doc is the single distilled reference for `httpware` design rationale, protocol seams, and remaining roadmap. It complements [`../CLAUDE.md`](../CLAUDE.md): `CLAUDE.md` holds AI-enforced invariants and operational commands; this file holds the reasoning and the structural map. +``` + +- [ ] **Step 2: Edit line 134 — drop the archived-epics parenthetical** + +Replace this exact line in `docs/engineering.md`: + +```markdown +Twenty-seven stories remain. Topic slugs in `docs/superpowers/specs/` and `docs/superpowers/plans/` use kebab-case descriptions, not the story IDs — these IDs are kept here only as a stable mapping to the archived epic specs (`archive/epics.md`). +``` + +With: + +```markdown +Twenty-seven stories remain. Topic slugs in `docs/superpowers/specs/` and `docs/superpowers/plans/` use kebab-case descriptions, not the story IDs — these IDs are retained as a stable identifier convention from the original epic structure. +``` + +(Note: this line still references `docs/superpowers/...` — those references get rewritten in Task 7.) + +- [ ] **Step 3: Edit line 181 — drop the retired-stories sentence** + +Replace this exact line in `docs/engineering.md`: + +```markdown +When work starts on a roadmap item, it gets a superpowers spec at `docs/superpowers/specs/YYYY-MM-DD--design.md` and a plan at `docs/superpowers/plans/YYYY-MM-DD--plan.md`. The bmad-era 40KB story specs in `archive/stories/` cover 1-1 through 1-5 and are retired going forward. +``` + +With: + +```markdown +When work starts on a roadmap item, it gets a spec at `docs/superpowers/specs/YYYY-MM-DD--design.md` and a plan at `docs/superpowers/plans/YYYY-MM-DD--plan.md`. +``` + +(The `docs/superpowers/` paths here also get rewritten in Task 7.) + +- [ ] **Step 4: Verify no archive references remain** + +Run: `grep -n "archive/" docs/engineering.md` + +Expected output: **(nothing)** — all archive citations are gone from engineering.md. + +- [ ] **Step 5: Commit** + +```bash +git add docs/engineering.md +git commit -m "docs: drop soft archive references from engineering.md" +``` + +--- + +### Task 3: Move workflow artifacts to planning/ (preserves git history) + +**Goal:** Relocate the `docs/superpowers/` subtree to a tool-neutral `planning/` directory at the repository root, and move `docs/deferred-work.md` alongside. Use `git mv` so history follows. This is a pure rename — no content changes. + +**Files:** +- Move: `docs/superpowers/specs/` → `planning/specs/` +- Move: `docs/superpowers/plans/` → `planning/plans/` +- Move: `docs/deferred-work.md` → `planning/deferred-work.md` +- Delete: `docs/superpowers/` (empty after the moves) + +- [ ] **Step 1: Create the parent planning/ directory** + +```bash +mkdir -p planning +``` + +- [ ] **Step 2: Move the specs and plans subdirectories** + +```bash +git mv docs/superpowers/specs planning/specs +git mv docs/superpowers/plans planning/plans +``` + +- [ ] **Step 3: Move deferred-work.md** + +```bash +git mv docs/deferred-work.md planning/deferred-work.md +``` + +- [ ] **Step 4: Remove the now-empty docs/superpowers/ directory** + +```bash +rmdir docs/superpowers +``` + +If `rmdir` complains the directory is not empty, list the contents (`ls -la docs/superpowers/`) and resolve before proceeding. + +- [ ] **Step 5: Verify history is preserved** + +```bash +git log --follow --oneline -n 3 planning/specs/2026-06-01-auth-coercion-design.md +``` + +Expected: shows commits from before the rename. If git shows only one commit, the move did not preserve history — abort and investigate. + +- [ ] **Step 6: Commit** + +```bash +git add -A +git commit -m "docs: rename docs/superpowers/ to planning/ (history-preserving)" +``` + +--- + +### Task 4: Move engineering.md and CONTRIBUTING.md into docs/dev/ + +**Goal:** Establish the `docs/dev/` subtree that the mkdocs site will publish under "Development". Move `docs/engineering.md` and the root `CONTRIBUTING.md` into it. + +**Files:** +- Move: `docs/engineering.md` → `docs/dev/engineering.md` +- Move: `CONTRIBUTING.md` (root) → `docs/dev/contributing.md` +- Create directory: `docs/dev/` + +- [ ] **Step 1: Create docs/dev/** + +```bash +mkdir -p docs/dev +``` + +- [ ] **Step 2: Move engineering.md** + +```bash +git mv docs/engineering.md docs/dev/engineering.md +``` + +- [ ] **Step 3: Move root CONTRIBUTING.md** + +```bash +git mv CONTRIBUTING.md docs/dev/contributing.md +``` + +- [ ] **Step 4: Verify history is preserved** + +```bash +git log --follow --oneline -n 3 docs/dev/engineering.md +``` + +Expected: shows multiple historical commits (the file has been edited several times). + +- [ ] **Step 5: Commit** + +```bash +git add -A +git commit -m "docs: move engineering.md and CONTRIBUTING.md into docs/dev/" +``` + +--- + +### Task 5: Create thin root CONTRIBUTING.md stub + +**Goal:** Replace the moved-out root `CONTRIBUTING.md` with a tiny stub that points to the published docs URL and the in-repo source path. Preserves GitHub's "open a PR" UI integration (which surfaces a root `CONTRIBUTING.md` to PR authors). + +**Files:** +- Create: `CONTRIBUTING.md` (at repo root) + +- [ ] **Step 1: Write the stub** + +Create `CONTRIBUTING.md` with exactly this content: + +```markdown +# Contributing + +The contributing guide is published as part of the project documentation: +**https://httpware.readthedocs.io/en/latest/dev/contributing/** + +Source: [`docs/dev/contributing.md`](docs/dev/contributing.md). +``` + +- [ ] **Step 2: Commit** + +```bash +git add CONTRIBUTING.md +git commit -m "docs: add thin root CONTRIBUTING.md stub pointing to published guide" +``` + +--- + +### Task 6: Bulk-update path references across CLAUDE.md and the planning tree + +**Goal:** Every file that referenced `docs/superpowers/...`, `docs/engineering.md`, `docs/deferred-work.md`, or `docs/archive/...` is now pointing at paths that have moved (or will not exist post-Task 10). This task mechanically rewrites the safe-to-transform references and manually fixes the context-sensitive ones (archive removals). + +**Files to update (21 total):** +- `CLAUDE.md` (root) +- `docs/dev/engineering.md` +- `planning/specs/2026-05-31-*.md` (9 files) +- `planning/specs/2026-06-01-auth-coercion-design.md` +- `planning/specs/2026-06-02-docs-reorg-and-mkdocs-design.md` (self-reference — see Step 6 below) +- `planning/plans/2026-05-31-*.md` (8 files) +- `planning/plans/2026-06-01-auth-coercion-plan.md` + +Note: `README.md` does NOT contain these old paths; it's refactored separately in Task 7. + +- [ ] **Step 1: Mechanical replacements across CLAUDE.md and planning/** + +These four substitutions are safe to apply via `sed` because they always mean the same thing in every context: + +**Important:** this plan and the spec live inside `planning/` and *intentionally* contain old-path references (in `git mv` commands and narrative). They are excluded from the rewrite. + +```bash +# macOS sed uses -i '' (BSD); replace with -i (GNU) on Linux if needed. +SED_INPLACE=(-i '') + +find CLAUDE.md planning/ docs/dev/ -type f -name '*.md' \ + ! -name '2026-06-02-docs-reorg-and-mkdocs-design.md' \ + ! -name '2026-06-02-docs-reorg-and-mkdocs-plan.md' \ + -print0 | + xargs -0 sed "${SED_INPLACE[@]}" \ + -e 's|docs/superpowers/specs/|planning/specs/|g' \ + -e 's|docs/superpowers/plans/|planning/plans/|g' \ + -e 's|docs/engineering\.md|docs/dev/engineering.md|g' \ + -e 's|docs/deferred-work\.md|planning/deferred-work.md|g' +``` + +If you are on Linux: change `SED_INPLACE=(-i '')` to `SED_INPLACE=(-i)`. + +- [ ] **Step 2: Verify mechanical replacements landed** + +Run: +```bash +grep -rn "docs/superpowers/\|docs/engineering\.md\|docs/deferred-work\.md" CLAUDE.md planning/ docs/dev/ \ + | grep -v "2026-06-02-docs-reorg-and-mkdocs-design.md" \ + | grep -v "2026-06-02-docs-reorg-and-mkdocs-plan.md" +``` + +Expected output: **(nothing)**. The only files that legitimately still contain the old paths are this plan and its spec — both excluded from the find pass above and from this grep. + +If hits appear in any other file, investigate and fix manually before continuing. + +- [ ] **Step 3: Update CLAUDE.md archive references (context-sensitive — manual)** + +Edit `CLAUDE.md`. Two distinct edits. + +Edit A — delete the archive bullet from "Where to find what" section. Replace this exact line: + +```markdown +- [`docs/archive/`](docs/archive/) — historical bmad-era planning bundle (PRD, architecture, epics, product briefs, per-story specs for 1-1 through 1-5). Consult only for original rationale or specific FR/NFR citations. +``` + +With: **(delete entirely — no replacement)** + +Edit B — strip the archive trailer from the "When in doubt" bullet. Replace this exact line: + +```markdown +- Check [`docs/engineering.md`](docs/engineering.md) before adding a new module or extension point; `docs/archive/architecture.md` has the deeper historical rationale if needed. +``` + +With (note: the `docs/engineering.md` → `docs/dev/engineering.md` part was already handled by Step 1's sed, so the line currently reads `Check [\`docs/dev/engineering.md\`](docs/dev/engineering.md) before adding a new module or extension point; \`docs/archive/architecture.md\` has the deeper historical rationale if needed.` — the edit removes only the trailing semicolon-clause): + +```markdown +- Check [`docs/dev/engineering.md`](docs/dev/engineering.md) before adding a new module or extension point. +``` + +- [ ] **Step 4: Sweep for any remaining archive references** + +Run: `grep -rn "docs/archive\|archive/" CLAUDE.md planning/ docs/dev/` + +Expected hits are only inside `planning/specs/2026-06-02-docs-reorg-and-mkdocs-design.md` (the spec narrates the deletion). Anywhere else, investigate. + +- [ ] **Step 5: Sweep for any remaining references to the root CONTRIBUTING.md (which is now a stub)** + +References to `CONTRIBUTING.md` from within planning/ or docs/dev/ should point at `docs/dev/contributing.md`. Currently there are none (verify): + +Run: `grep -rn "CONTRIBUTING\.md" planning/ docs/dev/ CLAUDE.md` + +Expected: no hits (the contributing doc isn't cross-referenced from other files). If hits appear, replace each with `docs/dev/contributing.md` as appropriate to the link context. + +- [ ] **Step 6: Commit** + +```bash +git add -A +git commit -m "docs: update path references for docs reorg" +``` + +--- + +### Task 7: Refactor README.md + +**Goal:** Slim README to align with the modern-python org pattern (project intent, install, runnable snippet, links to docs site / PyPI / license, org positioning). The current README is good content but lacks the docs-site link and has a "What ships in 0.1.0" section that duplicates engineering notes. Trim it. + +**Files:** +- Modify: `README.md` + +- [ ] **Step 1: Replace README.md entirely** + +Overwrite `README.md` with this content: + +````markdown +# httpware + +[![Test](https://github.com/modern-python/httpware/actions/workflows/ci.yml/badge.svg)](https://github.com/modern-python/httpware/actions/workflows/ci.yml) +[![PyPI version](https://badge.fury.io/py/httpware.svg)](https://pypi.org/project/httpware/) +[![Python versions](https://img.shields.io/pypi/pyversions/httpware.svg)](https://pypi.org/project/httpware/) +[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) + +**Async HTTP client framework for Python.** + +`httpware` is a typed, async HTTP client library with a protocol-based seam so the transport is swappable (`httpx2` ships as the default). Middleware composes via an onion model. Pydantic and msgspec response decoding ship out of the box. `RecordedTransport` replaces `respx` for transport-level tests. + +> **Status:** Pre-1.0 (0.1.0 alpha). Public API is subject to change between minor releases until v1.0. Resilience middleware (retry / timeout / bulkhead), streaming, and observability are not yet shipped. + +## Install + +```bash +pip install httpware +``` + +Optional extras: + +```bash +pip install httpware[msgspec] # MsgspecDecoder +``` + +(`otel`, `niquests`, and `all` extras are declared; integrations have not shipped yet.) + +## Quickstart + +```python +from httpware import AsyncClient +from pydantic import BaseModel + + +class User(BaseModel): + id: int + name: str + + +async def main() -> None: + async with AsyncClient(base_url="https://api.example.com") as client: + user = await client.get("/users/1", response_model=User) + print(user.name) +``` + +## 📚 [Documentation](https://httpware.readthedocs.io) + +## 📦 [PyPI](https://pypi.org/project/httpware) + +## 📝 [License](./LICENSE) + +## Part of `modern-python` + +Browse the full list of templates and libraries in [`modern-python`](https://github.com/modern-python) — see the org profile for the categorized index. +```` + +(The "What ships in 0.1.0" section is removed — that level of detail belongs on the docs site, not the README. The badges and the runnable Quickstart stay.) + +- [ ] **Step 2: Commit** + +```bash +git add README.md +git commit -m "docs: refactor README to link to published docs site" +``` + +--- + +### Task 8: Add mkdocs.yml, .readthedocs.yaml, docs/requirements.txt, docs/index.md + +**Goal:** Stand up the minimal mkdocs site. Four new files. After this task, `mkdocs build --strict` should succeed against the new structure. + +**Files:** +- Create: `mkdocs.yml` +- Create: `.readthedocs.yaml` +- Create: `docs/requirements.txt` +- Create: `docs/index.md` + +- [ ] **Step 1: Create mkdocs.yml** + +Create `mkdocs.yml` with exactly this content: + +```yaml +site_name: httpware +site_url: https://httpware.readthedocs.io/ +repo_url: https://github.com/modern-python/httpware +docs_dir: docs +edit_uri: edit/main/docs/ + +nav: + - Quick-Start: index.md + - Development: + - Engineering Notes: dev/engineering.md + - Contributing: dev/contributing.md + +theme: + name: material + features: + - content.code.copy + - content.action.edit + - navigation.footer + - navigation.sections + - navigation.top + - header.autohide + palette: + - media: "(prefers-color-scheme: light)" + scheme: default + primary: black + accent: pink + toggle: + icon: material/brightness-7 + name: Switch to dark mode + - media: "(prefers-color-scheme: dark)" + scheme: slate + primary: black + accent: pink + toggle: + icon: material/brightness-4 + name: Switch to system preference + +markdown_extensions: + - toc: + permalink: true + - pymdownx.highlight: + anchor_linenums: true + - pymdownx.inlinehilite + - pymdownx.superfences + - admonition + - attr_list +``` + +- [ ] **Step 2: Create .readthedocs.yaml** + +Create `.readthedocs.yaml` with exactly this content: + +```yaml +version: 2 + +build: + os: "ubuntu-22.04" + tools: + python: "3.12" + +python: + install: + - requirements: docs/requirements.txt + +mkdocs: + configuration: mkdocs.yml +``` + +- [ ] **Step 3: Create docs/requirements.txt** + +Create `docs/requirements.txt` with exactly this content: + +``` +mkdocs +mkdocs-material +``` + +- [ ] **Step 4: Create docs/index.md** + +Create `docs/index.md` with exactly this content: + +````markdown +# httpware + +A Python async HTTP client framework for building resilient service clients. `httpware` owns the abstraction layer above the underlying HTTP client (`httpx2` by default); consumers never import the transport directly. + +> **Status:** Pre-1.0 (0.1.0 alpha). Public API is subject to change between minor releases until v1.0. + +## Install + +```bash +pip install httpware +``` + +Optional extras: + +```bash +pip install httpware[msgspec] # MsgspecDecoder +``` + +## First request + +```python +import asyncio + +from httpware import AsyncClient +from pydantic import BaseModel + + +class User(BaseModel): + id: int + name: str + + +async def main() -> None: + async with AsyncClient(base_url="https://api.example.com") as client: + user = await client.get("/users/1", response_model=User) + print(user.name) + + +asyncio.run(main()) +``` + +## Where to go next + +- **[Engineering Notes](dev/engineering.md)** — design invariants, the five protocol seams, exception contract, module layout, testing patterns, optional-extras pattern. +- **[Contributing](dev/contributing.md)** — setup, conventions, workflow. + +## Part of `modern-python` + +`httpware` ships under the [`modern-python`](https://github.com/modern-python) org. See the org profile for the categorized index of related templates and libraries. +```` + +- [ ] **Step 5: Build the site locally to catch broken links now** + +Run: `uv run --with mkdocs --with mkdocs-material mkdocs build --strict` + +Expected: exits 0 with no warnings. A `site/` directory is produced (ignore it; do not commit). + +If the build fails with a "doc file is not included in the 'nav'" warning for `dev/engineering.md` or `dev/contributing.md`, the nav block in `mkdocs.yml` is wrong — recheck Step 1. + +If the build fails with a broken-link warning, follow the link in the error to find the offending file and fix its reference. Common cases: + +- A link to `engineering.md` from a sibling file under `dev/` should be a relative `engineering.md`, not `dev/engineering.md`. +- A link from `index.md` to a file under `dev/` should be `dev/engineering.md`. + +- [ ] **Step 6: Confirm site/ is gitignored** + +Run: `grep -n "^site" .gitignore` + +If `site/` is not gitignored, add it: + +```bash +echo "site/" >> .gitignore +git add .gitignore +``` + +Then remove the local build artifact: + +```bash +rm -rf site/ +``` + +- [ ] **Step 7: Commit** + +```bash +git add mkdocs.yml .readthedocs.yaml docs/requirements.txt docs/index.md +# Also stage .gitignore if you modified it in Step 6 +git add .gitignore 2>/dev/null || true +git commit -m "docs: add minimal mkdocs site published via Read the Docs" +``` + +--- + +### Task 9: Delete docs/archive/ + +**Goal:** Remove the bmad-era archive. Everything load-bearing has been ported. After this commit, `git grep -n "docs/archive"` should return zero hits (outside the spec/plan files that narrate the deletion). + +**Files:** +- Delete: `docs/archive/` (entire subtree) + +- [ ] **Step 1: Confirm contents about to be deleted** + +```bash +ls docs/archive/ +ls docs/archive/stories/ +``` + +Expected files: +- `docs/archive/`: `README.md`, `architecture.md`, `epics.md`, `prd.md`, `product-brief-httpware.md`, `product-brief-httpware-distillate.md`, `stories/` +- `docs/archive/stories/`: `1-1-project-scaffold-and-tooling.md`, `1-2-core-data-types.md`, `1-3-exception-hierarchy-with-plain-fields.md`, `1-4-transport-protocol-and-httpx2transport-adapter.md`, `1-5-responsedecoder-protocol-and-pydantic-adapter.md`, `sprint-status.yaml` + +If the inventory differs, **stop** and investigate before deleting. + +- [ ] **Step 2: Delete the directory** + +```bash +git rm -rf docs/archive +``` + +- [ ] **Step 3: Verify nothing in the published tree references archive** + +```bash +grep -rn "docs/archive\|archive/architecture\.md\|archive/epics\.md\|archive/stories" docs/ CLAUDE.md README.md +``` + +Expected: **(no output)**. Files under `docs/` no longer mention archive at all. + +- [ ] **Step 4: Final repo-wide sweep, excluding the migration spec and plan** + +```bash +grep -rn "docs/archive" . \ + --exclude-dir=.git --exclude-dir=.venv --exclude-dir=site \ + | grep -v "planning/specs/2026-06-02-docs-reorg-and-mkdocs-design.md" \ + | grep -v "planning/plans/2026-06-02-docs-reorg-and-mkdocs-plan.md" +``` + +Expected: **(no output)**. The migration's own spec and plan narrate the deletion and are the only allowed mentions. + +- [ ] **Step 5: Commit** + +```bash +git add -A +git commit -m "docs: delete bmad-era archive (rationale inlined into engineering.md)" +``` + +--- + +### Task 10: Run lint, tests, and final mkdocs strict build + +**Goal:** Confirm nothing in the source tree, the test suite, or the docs build regressed. + +- [ ] **Step 1: Lint** + +Run: `just lint-ci` + +Expected: exits 0. If ruff or ty fail, the cause is unrelated to the docs reorg (no Python files were touched) — investigate before continuing. + +- [ ] **Step 2: Tests** + +Run: `just test` + +Expected: exits 0, all tests pass, coverage unchanged. + +- [ ] **Step 3: Docs build** + +Run: `uv run --with mkdocs --with mkdocs-material mkdocs build --strict` + +Expected: exits 0 with no warnings. Remove the build artifact afterward: `rm -rf site/`. + +- [ ] **Step 4: History-preservation spot check** + +```bash +git log --follow --oneline -n 3 docs/dev/engineering.md +git log --follow --oneline -n 3 planning/specs/2026-06-01-auth-coercion-design.md +git log --follow --oneline -n 3 docs/dev/contributing.md +``` + +Expected: each shows historical commits from before the rename. If any of them shows only the rename commit, history was not preserved — `git mv` was missed somewhere and the file was re-created instead of moved. Investigate and fix. + +- [ ] **Step 5: No commit needed** + +Task 10 only runs verifications. If everything passed, the working tree is clean and you are done with the implementation. + +--- + +### Task 11: Post-merge follow-up (out of band, manual, not part of the PR) + +**Goal:** Document the manual steps required after the PR merges. These are NOT part of the implementation — they're a checklist for the human reviewer/maintainer. + +**Out-of-band steps:** + +1. **Create the Read the Docs project.** Log in at https://readthedocs.org, add `modern-python/httpware` as a new project. The webhook is set up automatically when the project is added via the GitHub integration. +2. **Verify the build.** First build kicks off when the PR merges to `main`. Confirm at the RTD project dashboard that the build succeeds. +3. **Claim the subdomain.** Default subdomain will be `httpware.readthedocs.io` if available. If taken, update `site_url` in `mkdocs.yml` and the docs link in `README.md` accordingly. +4. **Set default branch / version.** Confirm RTD is tracking the `main` branch as the default version. +5. **Add the RTD badge to the README** (optional, follow-up): + ```markdown + [![Documentation Status](https://readthedocs.org/projects/httpware/badge/?version=latest)](https://httpware.readthedocs.io/en/latest/?badge=latest) + ``` + +This task does not require any commit. + +--- + +## Verification summary + +Every task above should leave the working tree in a state where: + +- `just lint-ci` exits 0 (Tasks 1–10) +- `just test` exits 0 (Tasks 1–10) +- `git status` is clean after each task's commit +- `git log --follow` shows preserved history for every moved file +- After Task 8 onwards, `mkdocs build --strict` exits 0 with zero warnings + +The final repository layout matches the spec's "File structure" section: + +``` +/ +├─ README.md, SECURITY.md, CLAUDE.md, LICENSE, Justfile, pyproject.toml, ... +├─ CONTRIBUTING.md ← thin stub +├─ .readthedocs.yaml ← NEW +├─ mkdocs.yml ← NEW +├─ docs/ +│ ├─ index.md ← NEW +│ ├─ requirements.txt ← NEW +│ └─ dev/ +│ ├─ engineering.md +│ └─ contributing.md +├─ planning/ +│ ├─ specs/ +│ ├─ plans/ +│ └─ deferred-work.md +├─ src/httpware/ ← unchanged +└─ tests/ ← unchanged +``` diff --git a/docs/superpowers/specs/2026-05-31-asyncclient-design.md b/planning/specs/2026-05-31-asyncclient-design.md similarity index 99% rename from docs/superpowers/specs/2026-05-31-asyncclient-design.md rename to planning/specs/2026-05-31-asyncclient-design.md index 02e7ef4..8942bfd 100644 --- a/docs/superpowers/specs/2026-05-31-asyncclient-design.md +++ b/planning/specs/2026-05-31-asyncclient-design.md @@ -3,7 +3,7 @@ - **Date:** 2026-05-31 - **Status:** approved, ready for plan - **Scope:** Story 1-7 (seventh story of Epic 1). Ships the main public surface: `AsyncClient` with HTTP method shortcuts, typed `response_model` overloads, per-call overrides, lifecycle management, and `with_options`. Wires middleware via `compose()` (Story 2-1) since that story already landed. Out of scope: `auth=` parameter (Story 2-4), `data=`/`files=` body params (follow-up), transport reference-counting (deferred), streaming (Epic 4), observability (Epic 5), `RecordedTransport` (Story 1-8). -- **Roadmap pointer:** `docs/engineering.md` §8 "Epic 1 — Make typed HTTP requests with sensible defaults". +- **Roadmap pointer:** `docs/dev/engineering.md` §8 "Epic 1 — Make typed HTTP requests with sensible defaults". ## Why diff --git a/docs/superpowers/specs/2026-05-31-bmad-to-superpowers-transition-design.md b/planning/specs/2026-05-31-bmad-to-superpowers-transition-design.md similarity index 90% rename from docs/superpowers/specs/2026-05-31-bmad-to-superpowers-transition-design.md rename to planning/specs/2026-05-31-bmad-to-superpowers-transition-design.md index 1540e87..ee3b4d9 100644 --- a/docs/superpowers/specs/2026-05-31-bmad-to-superpowers-transition-design.md +++ b/planning/specs/2026-05-31-bmad-to-superpowers-transition-design.md @@ -16,14 +16,14 @@ Five stories have shipped (1-1 through 1-5). Twenty-seven remain in the bmad bac | --- | --- | | Workflow | Pure superpowers: brainstorming → spec → writing-plans → plan → executing-plans/subagent-driven → requesting-code-review → finishing-a-development-branch | | Topic naming | Kebab-case descriptions (`msgspec-decoder-adapter`), not story IDs | -| Legacy planning docs | Distill load-bearing decisions into `docs/engineering.md`; archive the rest under `docs/archive/` | -| AI entrypoint | `CLAUDE.md` stays the AI entrypoint and points at `docs/engineering.md` | +| Legacy planning docs | Distill load-bearing decisions into `docs/dev/engineering.md`; archive the rest under `docs/archive/` | +| AI entrypoint | `CLAUDE.md` stays the AI entrypoint and points at `docs/dev/engineering.md` | | First task | Retrospective code review of shipped work (1-1 through 1-5) via `superpowers:requesting-code-review` | | Second task | Refactor based on review findings, one cohesive PR per finding-group | | Ordering | Transition (single PR) lands first; tasks 1–2 run on the new structure | | `deferred-work.md` | Kept at repo root as-is; remains the "real but not actionable now" log | -## Distilled doc: `docs/engineering.md` +## Distilled doc: `docs/dev/engineering.md` One focused ~250–350 line document, written for both human contributors and AI agents. Sections: @@ -46,7 +46,7 @@ One focused ~250–350 line document, written for both human contributors and AI 6. **Testing patterns** — `pytest-asyncio` auto mode (no `@pytest.mark.asyncio`); `RecordedTransport` for transport mocking, not `respx`; Hypothesis property-based tests for concurrency-sensitive code in files named `test_*_props.py`. 7. **Optional-extras pattern** — pydantic, msgspec, opentelemetry isolated to their own modules; import inside the module, never at package top-level. 8. **Remaining roadmap** — short bullets, one line per remaining story (1-6 through 6-5), grouped by epic. No 40KB specs. When work starts on a roadmap item, it gets a superpowers spec. -9. **Deferred work** — pointer to `docs/deferred-work.md`; not duplicated here. +9. **Deferred work** — pointer to `planning/deferred-work.md`; not duplicated here. Explicit omissions vs. the bmad planning bundle: the 47 numbered FRs, 25 NFRs, persona work, and long architecture-decision essays move to `docs/archive/` and are cited only when a future spec needs the original rationale (e.g., "we decided X because of NFR-12"). @@ -83,7 +83,7 @@ docs/ `CLAUDE.md` updates: - Replace the "Project Overview" bmad bullets with pointers to `engineering.md` (primary) and `docs/archive/` (history). -- Add a one-liner describing the per-feature flow: brainstorming → spec in `docs/superpowers/specs/` → writing-plans → plan in `docs/superpowers/plans/` → executing-plans → requesting-code-review. +- Add a one-liner describing the per-feature flow: brainstorming → spec in `planning/specs/` → writing-plans → plan in `planning/plans/` → executing-plans → requesting-code-review. - The "Architecture invariants (CI-enforced)" and "Code conventions" sections stay verbatim — these are AI-enforcement rules, not design rationale. ## New per-feature workflow @@ -91,8 +91,8 @@ docs/ For every future piece of work — story 1-6 onwards, the upcoming refactor, anything else: ``` -1. brainstorming → docs/superpowers/specs/YYYY-MM-DD--design.md -2. writing-plans → docs/superpowers/plans/YYYY-MM-DD--plan.md +1. brainstorming → planning/specs/YYYY-MM-DD--design.md +2. writing-plans → planning/plans/YYYY-MM-DD--plan.md 3. using-git-worktrees (isolate workspace) 4. executing-plans OR subagent-driven-development (depending on task shape) 5. test-driven-development (rigid skill, applied throughout step 4) @@ -120,10 +120,10 @@ Choice between executing-plans vs. subagent-driven-development: `executing-plans ### Cutover PR (transition, executed first) -1. Write `docs/engineering.md` distilled from PRD, architecture, the five merged stories. +1. Write `docs/dev/engineering.md` distilled from PRD, architecture, the five merged stories. 2. Create `docs/archive/`; move `prd.md`, `architecture.md`, `epics.md`, both product briefs, and `stories/` into it. 3. Write `docs/archive/README.md`. -4. `docs/superpowers/specs/` already contains this spec; the cutover commit adds the corresponding plan to `docs/superpowers/plans/`. +4. `planning/specs/` already contains this spec; the cutover commit adds the corresponding plan to `planning/plans/`. 5. Update `CLAUDE.md` per the rules above. 6. Delete `.review-tmp/`. 7. Single commit, single PR, merge to `main`. This is the cutover point. @@ -134,9 +134,9 @@ Choice between executing-plans vs. subagent-driven-development: `executing-plans - Scope: stories 1-1 through 1-5 (scaffold, core data types, exception hierarchy, transport + httpx2 adapter, decoder protocol + pydantic adapter). - Triage findings into three buckets: - **Refactor now** — feeds Task 2. - - **Defer** — added to `docs/deferred-work.md`. + - **Defer** — added to `planning/deferred-work.md`. - **Discard** — noise / disagree, with one-line rationale. -- Output: triaged review report committed at `docs/superpowers/specs/YYYY-MM-DD-shipped-work-review.md`. +- Output: triaged review report committed at `planning/specs/YYYY-MM-DD-shipped-work-review.md`. ### Task 2 — refactor based on review @@ -160,9 +160,9 @@ Choice between executing-plans vs. subagent-driven-development: `executing-plans ## Definition of done -- `docs/engineering.md` exists and replaces the per-document references in CLAUDE.md. +- `docs/dev/engineering.md` exists and replaces the per-document references in CLAUDE.md. - `docs/archive/` contains the listed files with a framing README. -- `docs/superpowers/specs/` and `docs/superpowers/plans/` exist; this spec is committed there. +- `planning/specs/` and `planning/plans/` exist; this spec is committed there. - `.review-tmp/` is removed. - `CLAUDE.md` no longer references bmad artifacts as authoritative. - The cutover lands as a single PR, merged before Task 1 begins. diff --git a/docs/superpowers/specs/2026-05-31-middleware-protocol-and-chain-design.md b/planning/specs/2026-05-31-middleware-protocol-and-chain-design.md similarity index 99% rename from docs/superpowers/specs/2026-05-31-middleware-protocol-and-chain-design.md rename to planning/specs/2026-05-31-middleware-protocol-and-chain-design.md index 2682ee8..84b7f0b 100644 --- a/docs/superpowers/specs/2026-05-31-middleware-protocol-and-chain-design.md +++ b/planning/specs/2026-05-31-middleware-protocol-and-chain-design.md @@ -3,7 +3,7 @@ - **Date:** 2026-05-31 - **Status:** approved, ready for plan - **Scope:** Story 2-1 (first story of Epic 2). Defines the `Middleware` Protocol, `Next` type alias, and the `compose()` chain composer. Out of scope: decorators (2-2), `Request` helpers (2-3), auth coercion (2-4), AsyncClient wiring (2-5), streaming middleware chain (Epic 4). -- **Roadmap pointer:** `docs/engineering.md` §8 "Epic 2 — Compose request-handling logic via middleware". +- **Roadmap pointer:** `docs/dev/engineering.md` §8 "Epic 2 — Compose request-handling logic via middleware". ## Why diff --git a/docs/superpowers/specs/2026-05-31-msgspec-decoder-via-extras-design.md b/planning/specs/2026-05-31-msgspec-decoder-via-extras-design.md similarity index 99% rename from docs/superpowers/specs/2026-05-31-msgspec-decoder-via-extras-design.md rename to planning/specs/2026-05-31-msgspec-decoder-via-extras-design.md index f16d177..fc233a4 100644 --- a/docs/superpowers/specs/2026-05-31-msgspec-decoder-via-extras-design.md +++ b/planning/specs/2026-05-31-msgspec-decoder-via-extras-design.md @@ -3,7 +3,7 @@ - **Date:** 2026-05-31 - **Status:** approved, ready for plan - **Scope:** Story 1-6 (sixth story of Epic 1). Adds the second `ResponseDecoder` adapter — `MsgspecDecoder` — gated behind the `msgspec` extra. Introduces a small private `import_checker` module that future opt-in extras (otel, etc.) will reuse. Out of scope: AsyncClient wiring (Story 1-7), RecordedTransport (Story 1-8). -- **Roadmap pointer:** `docs/engineering.md` §8 "Epic 1 — Make typed HTTP requests with sensible defaults". +- **Roadmap pointer:** `docs/dev/engineering.md` §8 "Epic 1 — Make typed HTTP requests with sensible defaults". ## Why diff --git a/docs/superpowers/specs/2026-05-31-phase-shortcut-decorators-design.md b/planning/specs/2026-05-31-phase-shortcut-decorators-design.md similarity index 96% rename from docs/superpowers/specs/2026-05-31-phase-shortcut-decorators-design.md rename to planning/specs/2026-05-31-phase-shortcut-decorators-design.md index afc4f92..fa5371d 100644 --- a/docs/superpowers/specs/2026-05-31-phase-shortcut-decorators-design.md +++ b/planning/specs/2026-05-31-phase-shortcut-decorators-design.md @@ -3,7 +3,7 @@ - **Date:** 2026-05-31 - **Status:** approved, ready for plan - **Scope:** Story 2-2 (second story of Epic 2). Defines `@before_request`, `@after_response`, and `@on_error` decorators that wrap simple async user functions into `Middleware`-conforming instances. Out of scope: AsyncClient wiring (2-5), `Request.with_*` helpers beyond what exists (2-3), auth coercion (2-4). -- **Roadmap pointer:** `docs/engineering.md` §8 "Epic 2 — Compose request-handling logic via middleware". +- **Roadmap pointer:** `docs/dev/engineering.md` §8 "Epic 2 — Compose request-handling logic via middleware". ## Why @@ -24,7 +24,7 @@ The shape is decided by the archived epic spec (`docs/archive/epics.md` Epic 2 | `@on_error` return contract | If the handler returns a `Response`, that Response is returned to the caller. If it returns `None`, the original exception is re-raised (bare `raise` to preserve traceback). | | `BaseExceptionGroup` carve-out | None. PEP 654 `ExceptionGroup` is a subclass of `Exception` and is caught like any other. Users carve out groups themselves if needed. | | Public exports | `before_request`, `after_response`, `on_error` exported from `httpware.middleware` and re-exported at `httpware`. Matches the existing `Middleware` / `Next` re-export pattern. | -| Roadmap doc fix | Bundled in: `docs/engineering.md` §8 says `(@on_request, @on_response, @on_error)`. Rewrite to `(@before_request, @after_response, @on_error)` to match the spec. | +| Roadmap doc fix | Bundled in: `docs/dev/engineering.md` §8 says `(@on_request, @on_response, @on_error)`. Rewrite to `(@before_request, @after_response, @on_error)` to match the spec. | | Scope | Strict — no AsyncClient wiring, no extra `Request` helpers, no auth coercion. Those land in stories 2-3 through 2-5. | ## File structure @@ -34,7 +34,7 @@ The shape is decided by the archived epic spec (`docs/archive/epics.md` Epic 2 ``` src/httpware/middleware/__init__.py # add 3 decorator factories + 3 private classes (~65 lines added) src/httpware/__init__.py # re-export 3 names; extend __all__ -docs/engineering.md # fix §8 line 142 decorator names +docs/dev/engineering.md # fix §8 line 142 decorator names CHANGELOG.md # add Story 2.2 bullet under [Unreleased] / ### Added tests/test_middleware.py # 10 new tests appended (14 → 24) ``` @@ -161,7 +161,7 @@ Update `src/httpware/__init__.py`: - Change the import line `from httpware.middleware import Middleware, Next` to `from httpware.middleware import Middleware, Next, after_response, before_request, on_error`. - Insert `"after_response"`, `"before_request"`, `"on_error"` into the `__all__` list. The existing `__all__` is sorted by ASCII order (uppercase < lowercase), so lowercase entries sort to the end of the list — append the three new names after the existing final entry `"UnprocessableEntityError"`, in alphabetic order: `"after_response"`, then `"before_request"`, then `"on_error"`. -Update `docs/engineering.md` §8 line ~142: +Update `docs/dev/engineering.md` §8 line ~142: - From: `**2-2** Phase shortcut decorators (\`@on_request\`, \`@on_response\`, \`@on_error\`).` - To: `**2-2** Phase shortcut decorators (\`@before_request\`, \`@after_response\`, \`@on_error\`).` @@ -226,7 +226,7 @@ Ten new tests. Total `tests/test_middleware.py` grows from 14 to 24. - `src/httpware/middleware/__init__.py` exports `before_request`, `after_response`, `on_error` in addition to the existing `Middleware` and `Next`. - `src/httpware/__init__.py` re-exports the three new names and adds them to `__all__`. -- `docs/engineering.md` §8 line 142 reflects the corrected decorator names. +- `docs/dev/engineering.md` §8 line 142 reflects the corrected decorator names. - `CHANGELOG.md` has a Story 2.2 bullet under `[Unreleased]` / `### Added`. - `tests/test_middleware.py` contains 10 new tests; all 24 tests pass. - `just test` shows the increment from baseline; 100% line coverage on the new code. diff --git a/docs/superpowers/specs/2026-05-31-recordedtransport-design.md b/planning/specs/2026-05-31-recordedtransport-design.md similarity index 99% rename from docs/superpowers/specs/2026-05-31-recordedtransport-design.md rename to planning/specs/2026-05-31-recordedtransport-design.md index 62cf102..22c2cea 100644 --- a/docs/superpowers/specs/2026-05-31-recordedtransport-design.md +++ b/planning/specs/2026-05-31-recordedtransport-design.md @@ -3,7 +3,7 @@ - **Date:** 2026-05-31 - **Status:** approved, ready for plan - **Scope:** Story 1-8 (eighth and final story of Epic 1). Ships a built-in `Transport` test double at `src/httpware/transports/recorded.py` plus follow-up commits that replace the five existing in-tree test stubs (`_OkTransport`, `_FailingTransport`, `_FakeTransport`, two distinct `_RecordingTransport` variants, `_TrackingTransport`) with this new class. Out of scope: URL pattern matching / globs, cassette files loaded from JSON, streaming (Epic 4). -- **Roadmap pointer:** `docs/engineering.md` §8 "Epic 1 — Make typed HTTP requests with sensible defaults". +- **Roadmap pointer:** `docs/dev/engineering.md` §8 "Epic 1 — Make typed HTTP requests with sensible defaults". ## Why diff --git a/docs/superpowers/specs/2026-05-31-release-0.1.0-prep-design.md b/planning/specs/2026-05-31-release-0.1.0-prep-design.md similarity index 99% rename from docs/superpowers/specs/2026-05-31-release-0.1.0-prep-design.md rename to planning/specs/2026-05-31-release-0.1.0-prep-design.md index bd95490..68518b1 100644 --- a/docs/superpowers/specs/2026-05-31-release-0.1.0-prep-design.md +++ b/planning/specs/2026-05-31-release-0.1.0-prep-design.md @@ -72,7 +72,7 @@ Behind the seam: `httpware._internal.chain.compose`, `httpware._internal.import_ - `src/httpware/**` — no code changes. - `tests/**` — no test changes. - `LICENSE`, `SECURITY.md` — unchanged. -- `docs/engineering.md` — internal roadmap stays; "Released" markers are a follow-up. +- `docs/dev/engineering.md` — internal roadmap stays; "Released" markers are a follow-up. - `docs/archive/**` — bmad archive untouched. ## `.github/workflows/publish.yml` content diff --git a/docs/superpowers/specs/2026-05-31-request-immutability-helpers-design.md b/planning/specs/2026-05-31-request-immutability-helpers-design.md similarity index 96% rename from docs/superpowers/specs/2026-05-31-request-immutability-helpers-design.md rename to planning/specs/2026-05-31-request-immutability-helpers-design.md index 4bbb59d..65ad975 100644 --- a/docs/superpowers/specs/2026-05-31-request-immutability-helpers-design.md +++ b/planning/specs/2026-05-31-request-immutability-helpers-design.md @@ -3,13 +3,13 @@ - **Date:** 2026-05-31 - **Status:** approved, ready for plan - **Scope:** Story 2-3 (third story of Epic 2). Extends the existing `with_*` helper grid on `Request` and adds the missing helpers on `Response`. Out of scope: auth coercion (2-4), AsyncClient wiring (2-5), streaming (Epic 4). -- **Roadmap pointer:** `docs/engineering.md` §8 "Epic 2 — Compose request-handling logic via middleware". +- **Roadmap pointer:** `docs/dev/engineering.md` §8 "Epic 2 — Compose request-handling logic via middleware". ## Why Middleware (Story 2-1) and the phase decorators (Story 2-2) now exist. The remaining gap before middleware-driven request rewriting is ergonomic: `Request` currently exposes `with_header`, `with_url`, `with_body`, `with_query` (Story 1-2), but no plural `with_headers`, no `with_cookie`/`with_cookies`, no `with_extension`/`with_extensions`. `Response` has no `with_*` helpers at all. Middleware authors can technically work around the gaps via `dataclasses.replace`, but the framework should ship the ergonomic API directly. -The archived epic spec (`docs/archive/epics.md` Story 2.3) calls for `with_headers` on `Request`, plus `with_headers` and `with_status` on `Response`. `docs/engineering.md` §8 broadens the scope to include `with_cookie` and `with_extension`. This spec adopts the broader scope. +The archived epic spec (`docs/archive/epics.md` Story 2.3) calls for `with_headers` on `Request`, plus `with_headers` and `with_status` on `Response`. `docs/dev/engineering.md` §8 broadens the scope to include `with_cookie` and `with_extension`. This spec adopts the broader scope. ## Decisions @@ -142,7 +142,7 @@ No new fixtures, no async tests, no transport interaction. | Risk | Mitigation | | --- | --- | | `ty` flags `dict[str, str]` (the literal merged dict) as not assignable to `Mapping[str, str]` field. | `dict` is a `Mapping`; ty should accept the assignment. If it doesn't, cast at the assignment site with `Mapping[str, str]` annotation — but no cast expected. Story 1-2's existing `with_header` uses the same pattern (`{**self.headers, name: value}`) and passes ty cleanly. | -| Caller mixes `"X-Trace"` and `"x-trace"` keys via `with_headers`. | Documented v0 limitation; the merged dict will have both. Same behavior as the existing `with_header` and tracked in `docs/deferred-work.md` under the broader case-insensitive Mapping work. Don't try to fix here. | +| Caller mixes `"X-Trace"` and `"x-trace"` keys via `with_headers`. | Documented v0 limitation; the merged dict will have both. Same behavior as the existing `with_header` and tracked in `planning/deferred-work.md` under the broader case-insensitive Mapping work. Don't try to fix here. | | Future call sites expect `with_headers` to REPLACE rather than MERGE. | Docstring is explicit ("merged in"). The phase decorators in Story 2-2 already follow this convention implicitly (e.g., `@after_response` rebuilds `Response(...)` rather than calling a non-existent helper). Anyone reading the docstring will see the semantics. | | `Self` import on `Response` triggers ruff `I001` (import-sorting). | The import line `from typing import Any, Self` is alphabetic. ruff format will resolve any ordering. | diff --git a/docs/superpowers/specs/2026-05-31-shipped-work-review.md b/planning/specs/2026-05-31-shipped-work-review.md similarity index 91% rename from docs/superpowers/specs/2026-05-31-shipped-work-review.md rename to planning/specs/2026-05-31-shipped-work-review.md index 53c0913..2190c07 100644 --- a/docs/superpowers/specs/2026-05-31-shipped-work-review.md +++ b/planning/specs/2026-05-31-shipped-work-review.md @@ -40,7 +40,7 @@ ### Discard -- **`response.headers` collapses multi-valued headers (`Set-Cookie`, `Via`, `Link`) to last value** — already in `docs/deferred-work.md` under story-1-4 (2026-05-13). Documented v0 contract on `Httpx2Transport` docstring lines 62-67. +- **`response.headers` collapses multi-valued headers (`Set-Cookie`, `Via`, `Link`) to last value** — already in `planning/deferred-work.md` under story-1-4 (2026-05-13). Documented v0 contract on `Httpx2Transport` docstring lines 62-67. - **`StatusError.body: bytes` retains full response body unboundedly** — already in deferred-work under story-1-4 (2026-05-14). - **`httpx2.StreamError` family escapes the seam (`RuntimeError` subclasses)** — already in deferred-work under story-1-4 (2026-05-14); verified `StreamError.__mro__ == [StreamError, RuntimeError, Exception, BaseException, object]`. - **Concurrent `aclose()` ↔ `__call__` race** — already in deferred-work under story-1-4 (2026-05-14). The `asyncio.Lock` only guards init, not lifecycle. @@ -73,7 +73,7 @@ ## Already-deferred items observed -All ten "Discard" entries marked "already in deferred-work" were verified present in `docs/deferred-work.md` and are still real on the current HEAD. No re-raising; they remain accurate. +All ten "Discard" entries marked "already in deferred-work" were verified present in `planning/deferred-work.md` and are still real on the current HEAD. No re-raising; they remain accurate. ## Assessment @@ -81,4 +81,4 @@ All ten "Discard" entries marked "already in deferred-work" were verified presen **Defer count:** 4 **Discard count:** 25 (24 duplicates of existing deferred items + 5 rejected on consideration; one — D-4 — overlaps with an existing entry but is restated because the call-path concern is fresh) -**Recommendation:** No refactor needed — go straight to Epic 2. The four D-items can be appended to `docs/deferred-work.md` under a new "Deferred from: retrospective review (2026-05-31)" section in a follow-up commit, but none of them block middleware work. The architectural invariants are intact, the seams are clean, the exception contract is locked, and tests exercise the documented behavior (not implementation detail). The only structurally noteworthy observation is that the `_get_adapter` `TypeError` fallback in `decoders/pydantic.py` is test-driven rather than path-driven — worth keeping in mind when Story 1.6 reviews the `ResponseDecoder` protocol surface. +**Recommendation:** No refactor needed — go straight to Epic 2. The four D-items can be appended to `planning/deferred-work.md` under a new "Deferred from: retrospective review (2026-05-31)" section in a follow-up commit, but none of them block middleware work. The architectural invariants are intact, the seams are clean, the exception contract is locked, and tests exercise the documented behavior (not implementation detail). The only structurally noteworthy observation is that the `_get_adapter` `TypeError` fallback in `decoders/pydantic.py` is test-driven rather than path-driven — worth keeping in mind when Story 1.6 reviews the `ResponseDecoder` protocol surface. diff --git a/docs/superpowers/specs/2026-06-01-auth-coercion-design.md b/planning/specs/2026-06-01-auth-coercion-design.md similarity index 99% rename from docs/superpowers/specs/2026-06-01-auth-coercion-design.md rename to planning/specs/2026-06-01-auth-coercion-design.md index 5612af6..c7e2dd0 100644 --- a/docs/superpowers/specs/2026-06-01-auth-coercion-design.md +++ b/planning/specs/2026-06-01-auth-coercion-design.md @@ -3,7 +3,7 @@ - **Date:** 2026-06-01 - **Status:** approved, ready for plan - **Scope:** Story 2-4 (final unshipped story of Epic 2). Adds an `auth=` parameter to `AsyncClient.__init__` and `with_options` that accepts `str | Callable[[], str | Awaitable[str]] | Middleware | None`. Implements `_normalize_auth` at `src/httpware/_internal/auth.py` and the helper bearer-middleware factories. Out of scope: OAuth flows, refresh tokens, mTLS, signature schemes (HMAC, AWS Sigv4), per-call `auth=` on HTTP methods. -- **Roadmap pointer:** `docs/engineering.md` §8 "Epic 2 — Compose request-handling logic via middleware". +- **Roadmap pointer:** `docs/dev/engineering.md` §8 "Epic 2 — Compose request-handling logic via middleware". ## Why diff --git a/docs/superpowers/specs/2026-06-02-docs-reorg-and-mkdocs-design.md b/planning/specs/2026-06-02-docs-reorg-and-mkdocs-design.md similarity index 100% rename from docs/superpowers/specs/2026-06-02-docs-reorg-and-mkdocs-design.md rename to planning/specs/2026-06-02-docs-reorg-and-mkdocs-design.md