From 4812b920c7984e25e9fa7289b6a580e0cbef82b4 Mon Sep 17 00:00:00 2001 From: Christian Helle Date: Thu, 30 Apr 2026 17:10:49 +0200 Subject: [PATCH 1/5] docs(.squad): merge smoke-test inbox into decisions and log session Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .squad/agents/fenster/history.md | 63 +++++++++++++ .squad/agents/juno/history.md | 58 ++++++++++++ .squad/agents/lando/history.md | 137 +++++++++++++++++++++------- .squad/agents/scribe/history.md | 5 + .squad/agents/starkiller/history.md | 92 +++++++++++++++++++ .squad/decisions.md | 14 +++ 6 files changed, 336 insertions(+), 33 deletions(-) diff --git a/.squad/agents/fenster/history.md b/.squad/agents/fenster/history.md index f119881..e84e1a4 100644 --- a/.squad/agents/fenster/history.md +++ b/.squad/agents/fenster/history.md @@ -52,6 +52,45 @@ - 10MB size limit reasonable for OpenAPI specs (typical: 10-500KB) +### Smoke Test Investigation — Generated File Compilation (2026-04-30) + +**Context:** Investigated how a PowerShell smoke-test script should invoke openapi2zig and verify each generated Zig file compiles. + +**CLI invocation (confirmed):** +- Build executable once: `zig build -Doptimize=ReleaseFast` → `zig-out/bin/openapi2zig` +- Per-spec generation: `./zig-out/bin/openapi2zig generate -i -o ` +- Default output if `-o` omitted: `generated.zig` + +**Generated file structure (critical):** +- Every generated file starts with `const std = @import("std");` and exports `pub const` structs + functions +- **NO `main()` function** — they are library modules, not executables +- Therefore: **`zig run ` does NOT work** (no entry point) — Juno's suggestion is incorrect +- **Correct compile verification:** `zig test ` — compiles the file, reports "All 0 tests passed" (valid!). No harness needed. +- Alternative: `zig build-obj ` works but creates `.o` files requiring cleanup + +**Spec inventory (corrected from Juno's count):** +- `openapi/v2.0/`: 8 JSON API specs +- `openapi/v3.0/`: **10** JSON API specs (Juno missed `ingram-micro.json`) +- `openapi/v3.1/`: 2 JSON specs +- `openapi/v3.2/`: 2 JSON specs +- `openapi/json-schema/`: 2 files — **skip** (JSON Schema meta-specs, not OpenAPI; detector returns `Unsupported`) +- Total testable: **22 API specs** + +**Unsupported inputs (confirmed in `generator.zig`):** +- `.yaml`/`.yml` files → `GeneratorErrors.UnsupportedExtension` (non-zero exit) +- Spec with no `openapi` or `swagger` root field → detector returns `Unsupported` → non-zero exit +- json-schema files fall in this category + +**Output directory recommendation:** +- Write to `generated/smoke-test/-.zig` pattern (e.g., `v3.0-petstore.zig`) +- Never use `/tmp/` (runtime-blocked) +- Clean the directory before each run, and after success + +**Existing CI coverage gap:** +- Current `smoke-tests` job only runs `zig build run-generate` (4 petstore specs) then `zig run generated/main.zig` +- 18 specs are completely untested during CI +- `compile_generated.zig` harness already demonstrates the `refAllDecls` pattern for the 4 petstore specs + ### PR #46 generated-code documentation brief (2026-04-28T23:01:58.137+02:00) - PR #46 (`Fix real-world OpenAPI code generation`, merge `4e8bc19`) substantially changed the generated output contract: generated files now start with `const std = @import("std");`, model declarations, then a reusable API runtime and endpoint/resource wrappers. @@ -66,3 +105,27 @@ - OpenAI-specific generation hooks exist for `extra_body` flattening on `CreateResponse`/`CreateChatCompletionRequest`, `reasoning_details` on assistant messages, JSON-value-backed schema unions, typed filters/tools/annotations/audio, and stream helpers `streamChatCompletion`/`streamResponse`. - README is stale around usage and examples: it still says CLI generation is under development, describes `-o` as output directory, and shows old no-Client API examples returning bare values/dangling parsed data. - Current local environment cannot run Zig because `C:\Users\chris\AppData\Local\Microsoft\WinGet\Links\zig.exe` fails to launch; source inspection and checked-in generated files were used instead of regenerating locally. + +### Smoke-test script implementation (2026-04-30) + +- est/smoke-tests.ps1 resolves repo root via $PSScriptRoot/.. and Set-Locations there, so it works from any CWD on PowerShell 7 (Windows + Linux + macOS GitHub runners). +- Uses Get-Command zig for tooling discovery, prints zig version, then builds once with zig build -Doptimize=ReleaseFast. Executable path uses \True to pick openapi2zig.exe vs openapi2zig. +- Spec discovery: enumerates openapi/v2.0, openapi/v3.0, openapi/v3.1, openapi/v3.2 for *.json. YAML files are counted as skipped (visibility) but never run. openapi/json-schema/ is excluded by not being in the include list. +- Output layout: est/output//__.zig. Double underscore prevents collision when basenames already contain a dash. Cleaned each run unless -KeepOutput. +- Compile check uses zig test (NOT zig run) because generated files are library modules. Confirmed locally: emits All 0 tests passed and exits 0. +- Denylist is a list of @{ Spec; Mode; Reason } hashtables; * wildcards both fields. Currently empty — keep it that way until CI signal proves a gap. +- Continues past failures, captures stdout+stderr per case via 2>&1, prints a final pass/fail/skip summary, and lists failing cases with phase (generate vs compile) before xit 1. +- Verified locally: focused run (-Filter v3.0/petstore.json) all 4 modes → 4/4 PASS in ~30s. Full 22 specs × 4 modes = 88 cases not run locally (too expensive); CI is expected to be the first full-matrix execution. +- Did NOT touch CI workflow or README — those are Juno's surfaces per the charter. + +### Smoke-test denylist — ingram-micro v3.0 (2026-04-30) + +- Starkiller's full sweep was 84/88; the only failures were `openapi/v3.0/ingram-micro.json` in the compile phase across all four wrapper modes (none, tags, paths, hybrid). Root cause: unified model generator emits duplicate `pub const X = struct` declarations for shared/nested schemas — same root cause for every mode, not a wrapper-specific bug. +- Denylist updated with a single wildcard-mode entry: `@{ Spec = "openapi/v3.0/ingram-micro.json"; Mode = "*"; Reason = "duplicate `pub const` emissions from unified model generator (all wrapper modes)" }`. Mode="*" keeps the list compact and reflects the shared root cause; if a fix lands per-mode, split entries then. +- Verified locally with `pwsh -NoProfile -File test/smoke-tests.ps1 -Filter "v3.0/ingram-micro.json"` → 0 pass / 0 fail / 4 skip, exit 0. Petstore filter still PASSes normally — denylist does not leak into other specs. +- Reminder for next denylist entry: keep `Reason` concise, mention the gap (not the symptom alone), and prefer Mode="*" when the failure is mode-independent. Remove entries the moment the underlying generator gap closes. + +## 2026-04-30 — Smoke-test harness shipped +- Designed/implemented/validated est/smoke-tests.ps1 (88 cases: 22 specs × 4 wrapper modes), CI job updated with failure-only artifact upload, README documented. +- Initial denylist: ingram-micro.json (duplicate pub const emissions in unified model generator — follow-up backend work). +- Decision recorded in decisions.md (2026-04-30 entry). Session-scoped directive: agents use Claude Opus 4.7 for this session only. diff --git a/.squad/agents/juno/history.md b/.squad/agents/juno/history.md index 85bef87..1150645 100644 --- a/.squad/agents/juno/history.md +++ b/.squad/agents/juno/history.md @@ -70,3 +70,61 @@ Users can now provide OpenAPI specs via remote URLs in addition to local file pa - Read `.squad/decisions/inbox/lando-pr46-doc-impact.md` and `.squad/decisions/inbox/fenster-pr46-codegen-docs.md` before finalizing docs. - Added remaining docs gaps from handoff: query percent-encoding, borrowed `default_headers`, OpenAI stream helper names, resource wrapper naming caveat, validation commands, and OpenAPI 3.1 composite-schema caveat. - Kept documentation scoped to README and `docs/`; did not stage other agents' `.squad` history changes. + +### Smoke Test Investigation (2026-04-30T16:31:22.685+02:00) + +**Scope:** User wants comprehensive smoke tests (`test/smoke-tests.ps1`) to generate + compile all 24 OpenAPI specs, integrated into PR verification. + +**Key findings:** +- **24 specs available** across v2.0 (8), v3.0 (9), v3.1 (2), v3.2 (2); exclude json-schema/ utilities +- **Current test coverage:** Only v2.0 + v3.0 petstore (2 specs); too narrow +- **Refitter reference:** 880-line PowerShell script with phase-based generation/build pattern; proves approach works at scale +- **CLI is mature:** Already supports `-i `, `-o `, `--base-url`, `--resource-wrappers`; **no changes needed** for smoke testing +- **No UX issues:** Script can use existing CLI as-is + +**Recommendation (MVP):** +1. Create `test/smoke-tests.ps1` (~250 lines): discover specs, generate sequentially, compile each with `zig run`, report per-spec pass/fail +2. Call from `.github/workflows/ci.yml` or new `smoke-tests-comprehensive.yml` +3. Output format: `[PASS]/[FAIL] ` per line, summary, exit code 0/1 +4. Update README with smoke test docs +5. **No CLI modifications required** + +**Clarifying questions for Christian:** +- Test `--resource-wrappers` variants (4× tests) or single mode? +- Skip json-schema/ utilities? +- Block PR on failures or warnings? +- Sequential or parallel generation? +- Temp file location: `generated/smoke-test/` or elsewhere? + +**Detailed plan:** `.squad/decisions/inbox/juno-smoke-test-plan.md` + +### Smoke Test CI + Docs Wiring (2026-04-30T16:31:22.685+02:00) + +**Scope:** Wired the approved broad smoke-test plan into CI and README without touching `test/smoke-tests.ps1` (Fenster owns that script). + +**CI changes (`.github/workflows/ci.yml` smoke-tests job):** +- Kept the existing petstore harness step (`zig build run-generate` + `zig run generated/main.zig`) untouched. +- Added a step `Run broad smoke-test script` mirroring Refitter's style: `run: ./smoke-tests.ps1`, `working-directory: test`, `shell: pwsh`. +- Added a final step `Upload smoke-test outputs on failure` gated on `if: failure()`, uploading `test/output/` as artifact `smoke-test-output` with 7-day retention (matches existing `test-binaries` retention). +- Job still gated on PR or `main`; no change to triggers, runner, caches, or `needs`. + +**README changes:** +- Added a new `### Smoke tests` subsection under `## Quick Start` → `### Development`, between formatting and cross-compilation. +- Documented: `pwsh test/smoke-tests.ps1`, scope (v2.0/v3.0/v3.1/v3.2 JSON), four resource-wrapper modes, YAML/json-schema exclusions, output to `test/output/`, continue-through-failure with summary, temporary denylist, and CI behavior + artifact upload. + +**Conventions reused from Refitter (`christianhelle/refitter/.github/workflows/smoke-tests.yml`):** +- `working-directory: test` + relative `./smoke-tests.ps1` invocation, `shell: pwsh`. + +**Validation:** +- YAML parsed cleanly via Python `yaml.safe_load`. +- Did not run the smoke script (Fenster's `test/smoke-tests.ps1` not yet on disk at task time). +- Did not touch `.gitignore` — task said the plan adds `test/output/` to gitignore, but coordinating that lives with Fenster's script PR. + +**Coordination notes:** +- Smoke job assumes script path `test/smoke-tests.ps1` and output dir `test/output/`; if Fenster picks different paths, both the CI step's `working-directory` and the artifact `path` need to match. +- Artifact upload `path: test/output/` is relative to repo root (correct for `actions/upload-artifact`), even though the run step uses `working-directory: test`. + +## 2026-04-30 — Smoke-test harness shipped +- Designed/implemented/validated est/smoke-tests.ps1 (88 cases: 22 specs × 4 wrapper modes), CI job updated with failure-only artifact upload, README documented. +- Initial denylist: ingram-micro.json (duplicate pub const emissions in unified model generator — follow-up backend work). +- Decision recorded in decisions.md (2026-04-30 entry). Session-scoped directive: agents use Claude Opus 4.7 for this session only. diff --git a/.squad/agents/lando/history.md b/.squad/agents/lando/history.md index 23d8dee..cd9661c 100644 --- a/.squad/agents/lando/history.md +++ b/.squad/agents/lando/history.md @@ -1,36 +1,107 @@ -## Project Context +## Project Context + +**Project:** openapi2zig — CLI tool for generating type-safe Zig API clients from OpenAPI/Swagger specs +**User:** Christian Helle +**Tech Stack:** Zig v0.15.2, OpenAPI v3.0, Swagger v2.0 +**Architecture:** Unified converter pattern (parse → normalize → generate) + +## Key Architecture + +- **Specs supported:** Swagger v2.0, OpenAPI v3.0 +- **Parsing:** Version-specific models in `src/models/v2.0/` and `src/models/v3.0/` +- **Converters:** `src/generators/converters/` transforms version-specific → unified document +- **Generation:** `src/generators/unified/` generates Zig code from unified representation +- **Memory model:** Every dynamic struct has `deinit(allocator)`. **Always use `defer parsed.deinit(allocator)`.** +- **Testing:** xUnit-style with `test_utils.createTestAllocator()`, test against both v2.0 and v3.0 + +## Known Patterns + +- Version detection happens in `src/detector.zig` by parsing JSON for `openapi` or `swagger` fields +- CLI routing in `src/cli.zig` and `src/generator.zig` +- Sample specs in `openapi/v2.0/petstore.json` and `openapi/v3.0/petstore.json` for testing +- Generated code output to `generated/main.zig` (test harness imports both versions) + +## Learnings + +*To be updated as the team works.* + +### 2026-04-28T23:01:58.137+02:00 — PR #46 generated-output docs impact + +- PR #46 (`Fix real-world OpenAPI code generation`) materially changed generated public API shape: generated clients now expose a `Client` object, `Owned(T)`, `RawResponse`, `ApiResult(T)`, parse-error raw preservation, raw/result endpoint helpers, default headers, query encoding, resource wrappers, and SSE helpers. +- README snippets around CLI output and generated API usage are architecturally stale. Docs must show `Client.init(...)`/`deinit`, passing `*Client` into operations, `Owned(T).deinit()`, and the `{operation}`, `{operation}Raw`, `{operation}Result` trio instead of old per-call `std.http.Client` examples. +- The unified converter pattern remains the documentation spine: version-specific parsers/converters normalize into `src/models/common/document.zig`, then `src/generators/unified/model_generator.zig` and `src/generators/unified/api_generator.zig` emit all current output. +- Docs must not overclaim YAML support. Current behavior is JSON file/URL support; YAML extension handling reports unsupported. +- OpenAPI 3.1 composite schema typing is currently richer than v3.0/v3.2: `allOf` merge plus preserved `oneOf`/`anyOf`/discriminator metadata. Docs should describe this as current behavior without implying full parity across all spec versions. +- Key files for review: `README.md`, `docs/index.html`, `docs/json-value-typing-policy.md`, `docs/openai-generation-issues.md`, `src/cli.zig`, `src/generator.zig`, `src/generators/unified/api_generator.zig`, `src/generators/unified/model_generator.zig`, `generated/generated_v3.zig`, `generated/compile_generated.zig`. +- User preference from this task: use GPT-5.5 for team agents during this session only; commit docs work in small logical groups, but Lando must not stage or commit for this review task. + +### 2026-04-30T16:31:22.685+02:00 — Smoke Test Architecture Investigation + +**Scope:** Architectural proposal for integrating smoke tests into openapi2zig (spec generation + compilation verification across all 41 JSON specs). + +**Findings:** + +1. **Current Test Coverage Gap** + - Only 4 specs are code-generation tested: petstore.json (v2.0, v3.0), webhook-example.json (v3.1), petstore.json (v3.2) + - 37 additional JSON specs in openapi/ folders are never touched during CI + - YAML files (8 files) are unsupported by the converter—should be skipped gracefully + +2. **Build System Architecture** + - `build.zig` uses hardcoded `run-generate-*` steps for stable reference specs + - Generated code is tested via `generated/compile_generated.zig` test harness (imports 4 generated files, uses `std.testing.refAllDecls()` pattern) + - This is correct for baseline reproducibility, but not scalable to 41 specs + +3. **Reference Pattern from refitter** + - Separate PowerShell script (`test/smoke-tests.ps1`) for spec enumeration, generation, and compilation + - Sequential generation (one spec at a time) for clarity and error isolation + - Continue-all policy: collect all failures and summarize at end (not fail-fast) + - Pattern is cross-platform (runs on Windows, Linux, macOS via PowerShell Core) + +4. **Proposed Architecture: PowerShell Script (NOT build.zig)** + - Keep hardcoded `run-generate-*` steps in build.zig for determinism and caching + - Add `test/smoke-tests.ps1` for integration testing (dynamically discovers .json specs, skips .yaml) + - Script responsibility: enumerate → generate → verify compilation → collect failures → report summary + - Each spec is compiled in isolation via `zig build-obj` to catch codegen errors + - Clean integration point: smoke test runs *after* `zig build test` succeeds + +5. **Architecture Rationale: Separation of Concerns** + - **Zig build system** (deterministic, caching-friendly) handles unit tests and reference generation + - **PowerShell** (workflow orchestration) handles dynamic spec enumeration and batch integration testing + - **Decoupling** keeps build.zig focused on compilation; PowerShell handles discovery + - Scalable: adding new specs requires only dropping files in openapi/; no build.zig changes + +6. **Key Risks & Mitigations** + - **Risk: Generated code may not compile in isolation** → Mitigate: verify compilation independently per spec, fail cleanly on errors + - **Risk: YAML unsupported** → Mitigate: skip .yaml files, log as "skipped" + - **Risk: Performance (41 specs × compile time)** → Sequential generation is intentional; can parallelize later if needed + - **Risk: json-schema/ contains non-API specs** → Mitigate: filter to v2.0/, v3.0/, v3.1/, v3.2/ folders only + +7. **CI Integration Recommendation** + - Add optional `smoke-tests` job to `.github/workflows/ci.yml` that runs after core tests pass + - Runs on PR and main branch (can be made conditional on full build if too slow) + - Captures failures in artifact for visibility + +8. **Clarifying Questions for Christian** (documented in proposal) + - PowerShell on Linux CI (pwsh availability)? + - Failure policy confirmed (continue-all, not fail-fast)? + - File cleanup strategy (temp directory or in-memory)? + - json-schema/ exclusion? + - CI timing (every PR or only on main)? + +**Architectural Decision:** Smoke test belongs in PowerShell (`test/smoke-tests.ps1`), not build.zig. This follows refitter's proven pattern and respects the unified converter → generation pipeline without coupling integration testing to the build system. + +**Decision recorded:** `.squad/decisions/inbox/lando-smoke-test-plan.md` -**Project:** openapi2zig — CLI tool for generating type-safe Zig API clients from OpenAPI/Swagger specs -**User:** Christian Helle -**Tech Stack:** Zig v0.15.2, OpenAPI v3.0, Swagger v2.0 -**Architecture:** Unified converter pattern (parse → normalize → generate) +### 2026-04-30T16:31:22.685+02:00 — Smoke Test Implementation Review (APPROVE) -## Key Architecture +- Reviewed Fenster's est/smoke-tests.ps1 + .gitignore and Juno's CI + README changes against the approved plan. +- Implementation matches all four confirmed decisions (temp denylist, all 4 wrapper modes, keep petstore harness, output under test/output/). +- Cross-platform handling is sound: repo-root via $PSCommandPath, $IsWindows for exe suffix, slash normalization after Resolve-Path -Relative, `shell: pwsh` in CI. +- CI scoping is correct: appended to existing smoke-tests job; artifact upload guarded by if: failure(); petstore harness preserved. +- Known limitation (acknowledged in plan): zig test relies on Zig's lazy analysis; deeply unused decls may escape compile-check. Acceptable for v1; can harden later with explicit efAllDeclsRecursive wrapper if generator gaps slip through. +- Reminder for staging: `test/smoke-tests.ps1` is currently untracked; it must be added in the same commit as the CI change. -- **Specs supported:** Swagger v2.0, OpenAPI v3.0 -- **Parsing:** Version-specific models in `src/models/v2.0/` and `src/models/v3.0/` -- **Converters:** `src/generators/converters/` transforms version-specific → unified document -- **Generation:** `src/generators/unified/` generates Zig code from unified representation -- **Memory model:** Every dynamic struct has `deinit(allocator)`. **Always use `defer parsed.deinit(allocator)`.** -- **Testing:** xUnit-style with `test_utils.createTestAllocator()`, test against both v2.0 and v3.0 - -## Known Patterns - -- Version detection happens in `src/detector.zig` by parsing JSON for `openapi` or `swagger` fields -- CLI routing in `src/cli.zig` and `src/generator.zig` -- Sample specs in `openapi/v2.0/petstore.json` and `openapi/v3.0/petstore.json` for testing -- Generated code output to `generated/main.zig` (test harness imports both versions) - -## Learnings - -*To be updated as the team works.* - -### 2026-04-28T23:01:58.137+02:00 — PR #46 generated-output docs impact - -- PR #46 (`Fix real-world OpenAPI code generation`) materially changed generated public API shape: generated clients now expose a `Client` object, `Owned(T)`, `RawResponse`, `ApiResult(T)`, parse-error raw preservation, raw/result endpoint helpers, default headers, query encoding, resource wrappers, and SSE helpers. -- README snippets around CLI output and generated API usage are architecturally stale. Docs must show `Client.init(...)`/`deinit`, passing `*Client` into operations, `Owned(T).deinit()`, and the `{operation}`, `{operation}Raw`, `{operation}Result` trio instead of old per-call `std.http.Client` examples. -- The unified converter pattern remains the documentation spine: version-specific parsers/converters normalize into `src/models/common/document.zig`, then `src/generators/unified/model_generator.zig` and `src/generators/unified/api_generator.zig` emit all current output. -- Docs must not overclaim YAML support. Current behavior is JSON file/URL support; YAML extension handling reports unsupported. -- OpenAPI 3.1 composite schema typing is currently richer than v3.0/v3.2: `allOf` merge plus preserved `oneOf`/`anyOf`/discriminator metadata. Docs should describe this as current behavior without implying full parity across all spec versions. -- Key files for review: `README.md`, `docs/index.html`, `docs/json-value-typing-policy.md`, `docs/openai-generation-issues.md`, `src/cli.zig`, `src/generator.zig`, `src/generators/unified/api_generator.zig`, `src/generators/unified/model_generator.zig`, `generated/generated_v3.zig`, `generated/compile_generated.zig`. -- User preference from this task: use GPT-5.5 for team agents during this session only; commit docs work in small logical groups, but Lando must not stage or commit for this review task. +## 2026-04-30 — Smoke-test harness shipped +- Designed/implemented/validated est/smoke-tests.ps1 (88 cases: 22 specs × 4 wrapper modes), CI job updated with failure-only artifact upload, README documented. +- Initial denylist: ingram-micro.json (duplicate pub const emissions in unified model generator — follow-up backend work). +- Decision recorded in decisions.md (2026-04-30 entry). Session-scoped directive: agents use Claude Opus 4.7 for this session only. diff --git a/.squad/agents/scribe/history.md b/.squad/agents/scribe/history.md index 3860141..5ba3474 100644 --- a/.squad/agents/scribe/history.md +++ b/.squad/agents/scribe/history.md @@ -22,3 +22,8 @@ Initial setup complete. - Merged and deduplicated remote URL support and PR #46 documentation decisions into decisions.md. - Deleted processed decision inbox files and wrote orchestration/session logs. - History summarization gate checked; no history file met the 15360-byte threshold. + +## 2026-04-30 — Smoke-test harness shipped +- Designed/implemented/validated est/smoke-tests.ps1 (88 cases: 22 specs × 4 wrapper modes), CI job updated with failure-only artifact upload, README documented. +- Initial denylist: ingram-micro.json (duplicate pub const emissions in unified model generator — follow-up backend work). +- Decision recorded in decisions.md (2026-04-30 entry). Session-scoped directive: agents use Claude Opus 4.7 for this session only. diff --git a/.squad/agents/starkiller/history.md b/.squad/agents/starkiller/history.md index c497967..fe2eee8 100644 --- a/.squad/agents/starkiller/history.md +++ b/.squad/agents/starkiller/history.md @@ -86,4 +86,96 @@ - Each test has descriptive name and focused scope - Tests validate both positive and negative cases +### Smoke Test Coverage Design (2026-04-30) + +**Task:** Design `test/smoke-tests.ps1` for PR verification — enumerate all specs under `openapi/`, generate code, compile-check each. + +**Key findings:** + +**Spec inventory:** 22 JSON specs are eligible: 8 in `v2.0/`, 10 in `v3.0/`, 2 in `v3.1/`, 2 in `v3.2/`. Total spec count across all YAML equivalents is ~40 files. + +**Skip rules (hard):** +- All `*.yaml` / `*.yml` — YAML returns `UnsupportedExtension` from `generator.zig`; would produce 100% false failures. +- `openapi/json-schema/**` — JSON Schema meta-documents, not OpenAPI/Swagger specs. + +**Compile check technique:** Generated files are library-style (no `main`). Use `zig build-lib -fno-emit-bin` — compiles as static lib, skips output emit, catches type/semantic errors. Does not require a `main` function. + +**CI integration note:** The existing `smoke-tests` job already exists in `.github/workflows/ci.yml` (lines 127–160) but only tests the petstore pair via `zig build run-generate`. The new script augments it — keep the petstore pair as a fast sanity check and add the PS script as a spec sweep step. `pwsh` is available on `ubuntu-latest` without extra setup. + +**Risk specs:** +- `v3.0/ingram-micro.json` (755 KB) — may expose large-spec codegen edge cases +- `v3.0/callback-example.json` — callbacks may not be fully supported +- `v3.0/link-example.json` — link objects may produce incomplete output + +**Open question for Christian:** start with a denylist (skip known-broken, keep CI green) or allowlist (only known-passing, add others as fixed)? This is the primary governance question. + +**Proposal written:** `.squad/decisions/inbox/starkiller-smoke-test-plan.md` + +### Smoke Test Validation — Fenster's `test/smoke-tests.ps1` (2026-04-30) + +**Outcome: PASS** (script behavior). Generator gap surfaced on `openapi/v3.0/ingram-micro.json` is a separate triage item, not a script defect. + +**Static review against the approved plan — every requirement met:** +- Repo root resolution: uses `$PSCommandPath` → `Split-Path -Parent` → `Resolve-Path "$ScriptDir/.."` and `Set-Location $RepoRoot`, so it runs correctly regardless of caller's CWD (CI uses `working-directory: test`). +- Zig version check: `Find-Zig` resolves `Get-Command zig`, prints `zig version` before build; missing zig → red error + `exit 1`. +- ReleaseFast build: default `-Optimize ReleaseFast`, invoked as `zig build -Doptimize=$Optimize`, build failure → `exit 1`. +- Spec discovery: enumerates only `openapi/v2.0`, `v3.0`, `v3.1`, `v3.2`. `json-schema/` correctly excluded by not being in `$IncludedDirs`. YAML is filtered via lowercased extension check and counted into `$SkippedYaml` for the summary. +- Sequential matrix: nested `foreach ($spec) { foreach ($mode) }`, indices `[idx/total]`, colored case labels. +- Output dir: `$OutputDir = test/output`, wiped unless `-KeepOutput`, parent directories auto-created per (version, base, mode). +- Compile check: `zig test ` — correct technique because generated files are library modules without `main()`. (Cleaner than `build-lib -fno-emit-bin`; `zig test` reports "All 0 tests passed" on success.) +- Continue-all summary: per-case results stored in `$results`, summary prints Pass/Fail/Skip/Total + a failing-cases list grouped by phase (`generate`/`compile`). +- Denylist: `Test-Denylisted` with `Spec` (path or `*`) × `Mode` (name or `*`) × `Reason`; intentionally empty with explanatory comment. +- Non-zero exit: `exit 1` iff any non-denylisted case fails. + +**PowerShell parse check:** `[System.Management.Automation.Language.Parser]::ParseFile()` → no errors. + +**Focused run** (`-Filter "petstore" -Modes paths`, 9 cases): 9 PASS / 0 FAIL, exit 0. + +**Full sweep** (`pwsh test/smoke-tests.ps1`, 88 cases = 22 specs × 4 modes, ~5m14s on Windows + Zig 0.16.0): +- 84 PASS / 4 FAIL / 0 SKIP, exit 1. +- All 4 failures are `openapi/v3.0/ingram-micro.json` × {none,tags,paths,hybrid}, all in the `compile` phase. +- Root cause is generator-side: emits multiple `pub const = struct { ... }` declarations in the same Zig file (duplicate struct member name errors). Names like `QuoteDetailsRequestQuoteProductsRequestRetrieveQuoteProductsRequest` collide because the inline-schema name composer produces the same identifier from different schema sites. This is the exact "huge spec" risk I called out in the pre-plan inventory. +- Same pattern across all four resource-wrapper modes → bug is in the unified model generator's name-uniquing, not the API/wrapper layer. + +**Governance call (open question for Lando + Christian):** the script is correct and the bug is real, so CI will go red on this PR. Three options: (a) Fenster fixes the duplicate-name path in `src/generators/unified/model_generator.zig` before merge; (b) populate `$Denylist` with the four ingram-micro entries, link a tracking issue, ship the script; (c) restrict full sweep to a curated allowlist for now. Recommendation: (b) — the smoke harness is valuable signal *now*, and the duplicate-name fix is its own focused PR. + +**CI integration (`.github/workflows/ci.yml`):** +- Workflow file is `./smoke-tests.ps1` with `working-directory: test` and `shell: pwsh` → resolves to `test/smoke-tests.ps1` ✓. +- `pwsh` is preinstalled on `ubuntu-latest` GitHub runners ✓. +- Linux binary path: script uses `$IsWindows` to pick `openapi2zig` vs `openapi2zig.exe` ✓. +- `actions/upload-artifact@v4` path `test/output/` matches `$OutputDir` ✓; `if: failure()` only — good, keeps green runs lean. +- Job depends on `lint-and-format` and is gated on PR or `main`, alongside the existing `zig build run-generate` + `zig run generated/main.zig` petstore harness — that's the right layering: fast petstore sanity first, broad sweep second. +- `.gitignore` correctly adds `test/output/`. +- README "Smoke tests" section accurately describes the pwsh invocation and CI behavior. + +**No script defects found.** Validation artifacts (`test/output/`, `test/smoke-full.log`) cleaned up. + +### Smoke Test Final Gate — Post-Denylist (2026-04-30T16:31:22+02:00) + +**Outcome: PASS.** Final validation gate after Fenster added the `ingram-micro` denylist entry. + +**Denylist scoping check (static, `test/smoke-tests.ps1` lines 67–84):** +- Single entry: `Spec="openapi/v3.0/ingram-micro.json"` (literal, slash-normalized) × `Mode="*"` × clear reason. +- `Test-Denylisted` requires both `specMatch` AND `modeMatch`; literal spec string only matches that exact relpath, so other v3.0 specs are unaffected. `Mode="*"` correctly fans out across `none|tags|paths|hybrid`. +- Reason string includes "(all wrapper modes)" — explicit and audit-friendly. +- Comment block above the entry explains the generator gap and merge gate intent. + +**Focused runs (Windows + Zig 0.16.0, ReleaseFast):** +- `pwsh -NoProfile -File test/smoke-tests.ps1 -Filter "v3.0/ingram-micro.json"` → 0 pass / 0 fail / 4 skip, **exit 0**. All four modes correctly skipped with denylist reason. +- `pwsh -NoProfile -File test/smoke-tests.ps1 -Filter "v3.0/petstore.json" -Modes paths` → 1 pass / 0 fail / 0 skip, **exit 0**. Denylist does not bleed into unrelated specs. + +**Docs/CI alignment:** +- `.github/workflows/ci.yml` smoke-tests job (lines 127–172) unchanged and still correct: invokes `./smoke-tests.ps1` from `test/`, uploads `test/output/` on failure only. +- `README.md` (lines 177–194) already documents the denylist mechanism: "Honors a temporary denylist for known-unsupported spec/mode combinations so the PR gate can stay green while generator gaps are tracked explicitly." No doc drift. +- `.gitignore` still covers `test/output/`. + +**No script defects, no doc drift, no CI drift.** Validation artifacts cleaned up. + +**Follow-up (separate work, not blocking this gate):** the duplicate `pub const` emission in `src/generators/unified/model_generator.zig` for `openapi/v3.0/ingram-micro.json` remains a real generator bug. Recommend a tracking issue so the denylist entry has a clear retire-when condition; until then the entry self-documents the gap. + *To be updated as the team works.* + +## 2026-04-30 — Smoke-test harness shipped +- Designed/implemented/validated est/smoke-tests.ps1 (88 cases: 22 specs × 4 wrapper modes), CI job updated with failure-only artifact upload, README documented. +- Initial denylist: ingram-micro.json (duplicate pub const emissions in unified model generator — follow-up backend work). +- Decision recorded in decisions.md (2026-04-30 entry). Session-scoped directive: agents use Claude Opus 4.7 for this session only. diff --git a/.squad/decisions.md b/.squad/decisions.md index fb73c38..dd8414d 100644 --- a/.squad/decisions.md +++ b/.squad/decisions.md @@ -32,3 +32,17 @@ - **Model/runtime policy:** Object schemas become structs, arrays become typed slices where known, required fields are non-nullable, nullable unions collapse to ?T, safe unions preserve raw fallbacks, and open or ambiguous schemas remain std.json.Value. - **Verification note:** Starkiller accepted the docs with snippet fixes; Zig validation was blocked by a broken local WinGet zig.exe, so verification was static against current source/generated fixtures. - **Sources merged:** lando-pr46-doc-impact.md, fenster-pr46-codegen-docs.md, juno-pr46-docs.md. + +### 2026-04-30: Smoke-test harness for code generation + +- **Decision:** Add a PowerShell-driven smoke-test harness that runs the full spec × resource-wrapper matrix and validates that generated Zig compiles, integrated into the existing `smoke-tests` CI job. +- **Script contract (`test/smoke-tests.ps1`, owned by Fenster):** Lives at `test/smoke-tests.ps1`; resolves repo root from `$PSCommandPath` (CWD-independent); builds once with `-Doptimize=ReleaseFast`; enumerates JSON specs under `openapi/v2.0`, `v3.0`, `v3.1`, `v3.2`; skips `openapi/json-schema/` and counts YAML files as a single skip total; iterates 22 specs × 4 wrapper modes (`none`, `tags`, `paths`, `hybrid`) = 88 cases sequentially; writes outputs to `test/output//__.zig` (cleared per run unless `-KeepOutput`); exposes `-Filter` glob and `-Modes` selector for focused runs; exits non-zero only when non-denylisted failures occur. +- **Compile-check method:** Use `zig test ` — generated files are library modules with no `main()`, so `zig run`/`build-exe` cannot work. `zig test` compiles + type-checks all decls and exits cleanly with "All 0 tests passed." +- **Denylist policy:** Inline `@{ Spec; Mode; Reason }` table in the script. `Mode = "*"` wildcards mode-independent failures. Denylisted cases report as `SKIP` with reason and do not influence exit code. Each entry must reference the underlying generator gap and be removed when fixed. Initial entry: `openapi/v3.0/ingram-micro.json` × `*` — duplicate `pub const X = struct` emissions from `src/generators/unified/model_generator.zig` (all wrapper modes fail compile identically). +- **CI wiring (`.github/workflows/ci.yml`, owned by Juno):** Existing `smoke-tests` job keeps its `zig build run-generate` + `zig run generated/main.zig` petstore step, then adds `Run broad smoke-test script` (`run: ./smoke-tests.ps1`, `working-directory: test`, `shell: pwsh`) and `Upload smoke-test outputs on failure` (`actions/upload-artifact@v4`, `path: test/output/`, `retention-days: 7`, `if: failure()`). No trigger/runner/cache changes. +- **README:** New Development → Smoke tests subsection documenting `pwsh test/smoke-tests.ps1`, scope (JSON specs across v2.0–v3.2), skips (YAML, `json-schema/`), wrapper-mode matrix, output layout, denylist behavior, and CI behavior. +- **`.gitignore`:** Excludes `test/output/`. +- **Validation results:** PowerShell parse clean. Focused run (petstore × paths, 9 cases) 9/9 green. Full sweep (88 cases, ~5m14s on Zig 0.16.0) was 84/88 PASS pre-denylist; all 4 failures were `ingram-micro.json` compile-phase across all wrapper modes. Post-denylist: ingram-micro reports SKIP, petstore-only run still PASS, exit 0 — denylist does not leak. +- **Follow-up (out of scope):** Track and fix the unified model generator's duplicate `pub const` emission for shared/nested schemas; remove the ingram-micro denylist entry in the same PR. +- **Session-scoped directive (2026-04-30T16:31:22.685+02:00, Christian Helle via Copilot):** All agents use Claude Opus 4.7 for the rest of this session only. +- **Sources merged:** lando-smoke-test-plan.md, starkiller-smoke-test-plan.md, juno-smoke-test-plan.md, juno-smoke-ci-docs.md, fenster-smoke-test-plan.md, fenster-smoke-implementation.md, fenster-smoke-denylist-ingram-micro.md, starkiller-smoke-validation.md, copilot-directive-2026-04-30T16-31-22-685+02-00.md. From fbf250c8165f49a3c8404f8ac776b4a2193f0d07 Mon Sep 17 00:00:00 2001 From: Christian Helle Date: Thu, 30 Apr 2026 17:52:03 +0200 Subject: [PATCH 2/5] smoke tests: add infrastructure and output directory exclusion - Add test/smoke-tests.ps1: PowerShell harness that enumerates all JSON specs under openapi/v2.0, v3.0, v3.1, v3.2 and runs each through all four resource-wrapper modes (none, tags, paths, hybrid) = 88 cases total - Resolves repo root from \ so it is CWD-independent - Builds binary once with -Doptimize=ReleaseFast before running cases - Writes outputs to test/output//__.zig - Continue-all failure policy: collects all failures, prints summary, exits non-zero only for non-denylisted failures - Initial denylist: openapi/v3.0/ingram-micro.json (all modes) due to duplicate pub const emissions in the unified model generator - Add test/output/ to .gitignore to exclude generated artifacts from VCS Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .gitignore | 1 + test/smoke-tests.ps1 | 265 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 266 insertions(+) create mode 100644 test/smoke-tests.ps1 diff --git a/.gitignore b/.gitignore index 20c7eb1..f68bdf9 100644 --- a/.gitignore +++ b/.gitignore @@ -21,6 +21,7 @@ desktop.ini .idea **/generated_models.zig test/generated.zig +test/output/ src/openapi2zig.rc generated/generated.zig generated/openapi-petstore.zig diff --git a/test/smoke-tests.ps1 b/test/smoke-tests.ps1 new file mode 100644 index 0000000..fd9b313 --- /dev/null +++ b/test/smoke-tests.ps1 @@ -0,0 +1,265 @@ +#!/usr/bin/env pwsh +<# +.SYNOPSIS + Smoke tests for openapi2zig. + +.DESCRIPTION + Discovers JSON OpenAPI/Swagger specs under openapi/v2.0, openapi/v3.0, + openapi/v3.1, and openapi/v3.2, then for each (spec, resource-wrapper mode) + pair runs `openapi2zig generate` and compile-checks the generated file with + `zig test`. Continues after individual failures and exits non-zero if any + non-denylisted case fails. + + Cross-platform: works on PowerShell 7+ (Windows, Linux, macOS). + +.PARAMETER Optimize + Optimize mode passed to `zig build`. Defaults to ReleaseFast. + +.PARAMETER Filter + Optional wildcard pattern matched against the spec relative path + (e.g. "v3.0/petstore*"). Useful for focused local runs. + +.PARAMETER Modes + Resource-wrapper modes to test. Defaults to all four: none, tags, paths, hybrid. + +.PARAMETER KeepOutput + If set, the test/output directory is not cleaned before the run. + +.EXAMPLE + pwsh test/smoke-tests.ps1 + +.EXAMPLE + pwsh test/smoke-tests.ps1 -Filter "v3.0/petstore*" -Modes paths +#> +[CmdletBinding()] +param( + [string]$Optimize = "ReleaseFast", + [string]$Filter = "*", + [string[]]$Modes = @("none", "tags", "paths", "hybrid"), + [switch]$KeepOutput +) + +$ErrorActionPreference = "Stop" +Set-StrictMode -Version Latest + +# --------------------------------------------------------------------------- +# Repo root resolution (independent of current working directory) +# --------------------------------------------------------------------------- +$ScriptDir = Split-Path -Parent $PSCommandPath +$RepoRoot = Resolve-Path (Join-Path $ScriptDir "..") | Select-Object -ExpandProperty Path +Set-Location $RepoRoot + +$OutputDir = Join-Path $RepoRoot "test/output" + +# --------------------------------------------------------------------------- +# Denylist: known unsupported (spec, mode) combinations. +# +# Format: each entry is a hashtable with: +# Spec - relative spec path using forward slashes (e.g. "openapi/v3.0/petstore.json") +# or "*" to match any spec +# Mode - one of none|tags|paths|hybrid, or "*" to match any mode +# Reason - short justification, included in skip output +# +# Keep this list small and explicit. Each entry is a generator gap to fix. +# This list intentionally starts empty; populate only when CI signal demands +# it and a tracking issue exists. +# --------------------------------------------------------------------------- +$Denylist = @( + # openapi/v3.0/ingram-micro.json fails the compile phase in every + # resource-wrapper mode (none, tags, paths, hybrid) because the unified + # model generator emits duplicate `pub const X = struct` declarations for + # several shared schemas. Denylisted across all modes via Mode="*" until + # the unified model generator dedupes shared/nested type emissions. + @{ Spec = "openapi/v3.0/ingram-micro.json"; Mode = "*"; Reason = "duplicate `pub const` emissions from unified model generator (all wrapper modes)" } +) + +function Test-Denylisted { + param([string]$RelSpec, [string]$Mode) + foreach ($entry in $Denylist) { + $specMatch = ($entry.Spec -eq "*") -or ($entry.Spec -eq $RelSpec) + $modeMatch = ($entry.Mode -eq "*") -or ($entry.Mode -eq $Mode) + if ($specMatch -and $modeMatch) { return $entry } + } + return $null +} + +# --------------------------------------------------------------------------- +# Tooling discovery +# --------------------------------------------------------------------------- +function Find-Zig { + $cmd = Get-Command zig -ErrorAction SilentlyContinue + if (-not $cmd) { + Write-Host "ERROR: zig was not found on PATH." -ForegroundColor Red + exit 1 + } + return $cmd.Source +} + +function Get-CliPath { + $exeName = if ($IsWindows) { "openapi2zig.exe" } else { "openapi2zig" } + return (Join-Path $RepoRoot (Join-Path "zig-out/bin" $exeName)) +} + +# --------------------------------------------------------------------------- +# Build CLI once +# --------------------------------------------------------------------------- +$ZigPath = Find-Zig +Write-Host "Using zig: $ZigPath" +& $ZigPath version +Write-Host "" +Write-Host "==> Building openapi2zig (-Doptimize=$Optimize)" -ForegroundColor Cyan +& $ZigPath build "-Doptimize=$Optimize" +if ($LASTEXITCODE -ne 0) { + Write-Host "ERROR: zig build failed" -ForegroundColor Red + exit 1 +} + +$Cli = Get-CliPath +if (-not (Test-Path $Cli)) { + Write-Host "ERROR: built CLI not found at $Cli" -ForegroundColor Red + exit 1 +} +Write-Host "CLI: $Cli" + +# --------------------------------------------------------------------------- +# Output dir prep +# --------------------------------------------------------------------------- +if (-not $KeepOutput -and (Test-Path $OutputDir)) { + Remove-Item $OutputDir -Recurse -Force +} +New-Item -ItemType Directory -Path $OutputDir -Force | Out-Null + +# --------------------------------------------------------------------------- +# Spec discovery +# --------------------------------------------------------------------------- +$IncludedDirs = @("openapi/v2.0", "openapi/v3.0", "openapi/v3.1", "openapi/v3.2") +$Specs = New-Object System.Collections.Generic.List[object] +$SkippedYaml = 0 + +foreach ($rel in $IncludedDirs) { + $abs = Join-Path $RepoRoot $rel + if (-not (Test-Path $abs)) { continue } + $files = Get-ChildItem -Path $abs -Recurse -File + foreach ($f in $files) { + $ext = $f.Extension.ToLowerInvariant() + if ($ext -eq ".yaml" -or $ext -eq ".yml") { $SkippedYaml++; continue } + if ($ext -ne ".json") { continue } + $relPath = (Resolve-Path -Relative $f.FullName) -replace '\\', '/' + $relPath = $relPath -replace '^\./', '' + # Pattern match against the slash-normalized relative path. + if ($relPath -notlike "*$Filter*" -and ($Filter -ne "*")) { continue } + $Specs.Add([pscustomobject]@{ + Abs = $f.FullName + Rel = $relPath + Version = (Split-Path $rel -Leaf) + Base = [System.IO.Path]::GetFileNameWithoutExtension($f.Name) + }) + } +} + +$Specs = $Specs | Sort-Object Rel +$totalCases = $Specs.Count * $Modes.Count + +Write-Host "" +Write-Host "Discovered $($Specs.Count) JSON specs across $($IncludedDirs.Count) directories" +Write-Host "Resource-wrapper modes: $($Modes -join ', ')" +Write-Host "Total cases to run: $totalCases (sequential)" +Write-Host "Skipped YAML files: $SkippedYaml (unsupported by generator)" +Write-Host "Output directory: $OutputDir" +Write-Host "" + +# --------------------------------------------------------------------------- +# Run loop +# --------------------------------------------------------------------------- +$results = New-Object System.Collections.Generic.List[object] +$idx = 0 + +foreach ($spec in $Specs) { + foreach ($mode in $Modes) { + $idx++ + $caseLabel = "[{0}/{1}] {2} :: {3}" -f $idx, $totalCases, $spec.Rel, $mode + + $deny = Test-Denylisted -RelSpec $spec.Rel -Mode $mode + if ($null -ne $deny) { + Write-Host "SKIP $caseLabel (denylist: $($deny.Reason))" -ForegroundColor Yellow + $results.Add([pscustomobject]@{ + Spec = $spec.Rel; Mode = $mode; Status = "skip"; Reason = $deny.Reason + }) + continue + } + + Write-Host "---- $caseLabel" -ForegroundColor Cyan + + $outFile = Join-Path $OutputDir (Join-Path $spec.Version ("{0}__{1}.zig" -f $spec.Base, $mode)) + $outParent = Split-Path -Parent $outFile + if (-not (Test-Path $outParent)) { New-Item -ItemType Directory -Path $outParent -Force | Out-Null } + + # ---- Generate ---- + $genOutput = & $Cli generate -i $spec.Abs -o $outFile --resource-wrappers $mode 2>&1 + $genExit = $LASTEXITCODE + if ($genExit -ne 0) { + Write-Host "FAIL (generate) $caseLabel" -ForegroundColor Red + Write-Host ($genOutput | Out-String) + $results.Add([pscustomobject]@{ + Spec = $spec.Rel; Mode = $mode; Status = "fail"; Phase = "generate" + ExitCode = $genExit; Output = ($genOutput | Out-String) + }) + continue + } + + if (-not (Test-Path $outFile)) { + Write-Host "FAIL (generate produced no file) $caseLabel" -ForegroundColor Red + $results.Add([pscustomobject]@{ + Spec = $spec.Rel; Mode = $mode; Status = "fail"; Phase = "generate" + ExitCode = 0; Output = "no output file" + }) + continue + } + + # ---- Compile-check ---- + # Generated files are library modules (no main()), so use `zig test`, + # which compiles them and reports "All 0 tests passed" on success. + $testOutput = & $ZigPath test $outFile 2>&1 + $testExit = $LASTEXITCODE + if ($testExit -ne 0) { + Write-Host "FAIL (compile) $caseLabel" -ForegroundColor Red + Write-Host ($testOutput | Out-String) + $results.Add([pscustomobject]@{ + Spec = $spec.Rel; Mode = $mode; Status = "fail"; Phase = "compile" + ExitCode = $testExit; Output = ($testOutput | Out-String) + }) + continue + } + + Write-Host "PASS $caseLabel" -ForegroundColor Green + $results.Add([pscustomobject]@{ + Spec = $spec.Rel; Mode = $mode; Status = "pass" + }) + } +} + +# --------------------------------------------------------------------------- +# Summary +# --------------------------------------------------------------------------- +$pass = @($results | Where-Object Status -eq "pass").Count +$fail = @($results | Where-Object Status -eq "fail").Count +$skip = @($results | Where-Object Status -eq "skip").Count + +Write-Host "" +Write-Host "================ Smoke-test summary ================" +Write-Host ("Pass : {0}" -f $pass) -ForegroundColor Green +Write-Host ("Fail : {0}" -f $fail) -ForegroundColor ($(if ($fail -gt 0) { "Red" } else { "Gray" })) +Write-Host ("Skip : {0}" -f $skip) -ForegroundColor Yellow +Write-Host ("Total: {0}" -f $results.Count) +Write-Host "====================================================" + +if ($fail -gt 0) { + Write-Host "" + Write-Host "Failing cases:" -ForegroundColor Red + foreach ($r in ($results | Where-Object Status -eq "fail")) { + Write-Host (" - [{0}] {1} :: {2} (exit {3})" -f $r.Phase, $r.Spec, $r.Mode, $r.ExitCode) -ForegroundColor Red + } + exit 1 +} + +exit 0 From acc0f93bc968dcb65f8bc143257a58d0b2f3aa15 Mon Sep 17 00:00:00 2001 From: Christian Helle Date: Thu, 30 Apr 2026 17:52:13 +0200 Subject: [PATCH 3/5] smoke tests: integrate into CI pipeline with artifact uploads - Append two steps to the existing smoke-tests job in ci.yml: 1. 'Run broad smoke-test script': runs ./smoke-tests.ps1 from the test/ working directory using shell: pwsh (PowerShell Core, cross-platform) 2. 'Upload smoke-test outputs on failure': uploads test/output/ as artifact 'smoke-test-output' with retention-days: 7, guarded by if: failure() - Petstore harness (zig build run-generate + zig run generated/main.zig) is preserved unchanged; smoke tests run after it in the same job - No trigger, runner, cache, or strategy changes Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/workflows/ci.yml | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1c02cbc..88787a1 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -157,3 +157,16 @@ jobs: run: | zig build run-generate zig run generated/main.zig + + - name: Run broad smoke-test script + run: ./smoke-tests.ps1 + working-directory: test + shell: pwsh + + - name: Upload smoke-test outputs on failure + if: failure() + uses: actions/upload-artifact@v4 + with: + name: smoke-test-output + path: test/output/ + retention-days: 7 From bad7cb5032dc4fe613aa6834451e1ae3e64d1eaf Mon Sep 17 00:00:00 2001 From: Christian Helle Date: Thu, 30 Apr 2026 17:52:21 +0200 Subject: [PATCH 4/5] smoke tests: document usage and behavior Add a 'Smoke tests' subsection under Development in README.md covering: - How to run: pwsh test/smoke-tests.ps1 - Scope: all JSON specs under openapi/v2.0, v3.0, v3.1, v3.2 - Resource-wrapper matrix (none, tags, paths, hybrid) - Skips: YAML files and openapi/json-schema/ meta-schemas - Output layout under test/output/ (gitignored) - Continue-all failure policy and summary output - Denylist behavior for known-unsupported spec/mode combinations - CI behavior: runs in smoke-tests job on PRs and main; uploads test/output/ as workflow artifact when the job fails Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- README.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/README.md b/README.md index 63c2db6..d99766f 100644 --- a/README.md +++ b/README.md @@ -174,6 +174,25 @@ zig fmt --check src/ zig fmt --check build.zig ``` +### Smoke tests + +Run the broad smoke-test script to validate code generation against every supported sample specification: + +```bash +pwsh test/smoke-tests.ps1 +``` + +What it does: + +- Validates all eligible JSON API specs under `openapi/v2.0`, `openapi/v3.0`, `openapi/v3.1`, and `openapi/v3.2`. +- Runs each spec through every resource-wrapper mode: `none`, `tags`, `paths`, and `hybrid`. +- Skips YAML files and the meta-schema documents under `openapi/json-schema/`. +- Writes generated outputs to `test/output/` (gitignored), with paths and filenames that include the resource-wrapper mode so variants do not collide. +- Continues through individual failures and prints a final summary listing every failing spec/mode combination, then exits non-zero if any case failed. +- Honors a temporary denylist for known-unsupported spec/mode combinations so the PR gate can stay green while generator gaps are tracked explicitly. + +In CI, the same script runs in the `smoke-tests` job on pull requests and `main`, alongside the existing `zig build run-generate` + `zig run generated/main.zig` petstore harness. When the smoke-tests job fails, `test/output/` is uploaded as a workflow artifact for triage. + ### Cross-compilation Build for different targets: From 791d9b572f903738c81f6e05fd141acb7b24fe96 Mon Sep 17 00:00:00 2001 From: Christian Helle Date: Thu, 30 Apr 2026 17:53:10 +0200 Subject: [PATCH 5/5] Update Lando's learnings from smoke tests verification Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .squad/agents/lando/history.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/.squad/agents/lando/history.md b/.squad/agents/lando/history.md index cd9661c..9eec3e3 100644 --- a/.squad/agents/lando/history.md +++ b/.squad/agents/lando/history.md @@ -101,6 +101,15 @@ - Known limitation (acknowledged in plan): zig test relies on Zig's lazy analysis; deeply unused decls may escape compile-check. Acceptable for v1; can harden later with explicit efAllDeclsRecursive wrapper if generator gaps slip through. - Reminder for staging: `test/smoke-tests.ps1` is currently untracked; it must be added in the same commit as the CI change. +## 2026-04-30 — Smoke-test commit strategy + +**Commit strategy learning:** +- Three logical groups is the right granularity for a feature like this: infrastructure (script + gitignore), CI wiring, documentation. Each group is independently reviewable and revertable. +- The `.squad/` state was already committed in a prior session (4812b92), so Group 4 was correctly skipped — always verify actual git status before assuming a group has changes. +- Untracked files (like `test/smoke-tests.ps1`) must be staged with `git add` explicitly; relying on `-a` would have mixed groups. Use per-file staging to keep groups clean. +- Commit messages benefit from the "what + why + scope" structure: first line is the imperative summary, body enumerates the specific behavioral changes, not just file names. +- The `Co-authored-by` trailer belongs on every commit regardless of group size; it is a project invariant, not optional. + ## 2026-04-30 — Smoke-test harness shipped - Designed/implemented/validated est/smoke-tests.ps1 (88 cases: 22 specs × 4 wrapper modes), CI job updated with failure-only artifact upload, README documented. - Initial denylist: ingram-micro.json (duplicate pub const emissions in unified model generator — follow-up backend work).