Skip to content

feat: pipeline config consumption via proteus#150

Open
mvillmow wants to merge 1 commit into
mainfrom
1-auto-impl
Open

feat: pipeline config consumption via proteus#150
mvillmow wants to merge 1 commit into
mainfrom
1-auto-impl

Conversation

@mvillmow

@mvillmow mvillmow commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #1. Implements config-driven pipeline execution via new proteus Python package.

  • Config files in configs/pipelines/*.yaml are now consumed by production code
  • Validation via JSON Schema (Draft-07) at parse time and commit time (pre-commit hook)
  • Stages are topologically sorted by depends_on with explicit edge-case guards
  • Each stage type (dagger, skopeo, dispatch) has a handler that emits fully-specified argv
  • just pipeline <CONFIG> delegates to python -m proteus.pipeline run
  • 25 unit/integration tests cover loader, topology, handlers, runner, and real achaean-fleet.yaml

Test plan

  • All 25 unit and integration tests pass
  • Dry-run produces expected output (4 stages in correct order)
  • Schema validation succeeds on achaean-fleet.yaml
  • Pre-commit hook validates configs
  • CI workflows updated to use pixi + pytest

Follow-up

#83 tracks fixing build/promote tag arithmetic in handlers once the foundation is in place.

Closes #1

@mvillmow mvillmow enabled auto-merge (squash) June 3, 2026 10:34
@mvillmow mvillmow force-pushed the 1-auto-impl branch 2 times, most recently from 261740a to 4a3e16a Compare June 28, 2026 17:08
@mvillmow mvillmow disabled auto-merge June 28, 2026 17:18
@mvillmow mvillmow enabled auto-merge (squash) June 28, 2026 18:45
Implement config-driven pipeline execution via new proteus package:
- Create schemas/pipeline.schema.json (Draft-07) with per-type and per-function arg validation
- Implement proteus/pipeline modules: loader, topology, handlers, runner, CLI
- Loader validates YAML against schema and builds Pipeline dataclass (no field injection)
- Topology sorts stages by depends_on with explicit edge-case guards (self-loops, unknown deps, cycles)
- Handlers emit config-driven argv via cmd_dagger/cmd_skopeo/cmd_dispatch with PipelineConfigError guards
- Runner executes stages via subprocess.run (list argv, no shell injection)
- CLI provides `validate` and `run` subcommands with --dry-run support
- Update justfile pipeline/validate recipes to delegate to proteus
- Add 25 unit/integration tests covering all modules and the real achaean-fleet.yaml config
- Add check-jsonschema pre-commit hook for schema validation at commit time
- Update CI workflows to use pixi + proteus for validation and execution
- Add schema header comment to achaean-fleet.yaml for editor integration
- Update CLAUDE.md to document pipeline config consumption (was known critical defect #1)

Fixes: #1, #82
Replaces hardcoded pipeline steps with config-driven dispatch.

Signed-off-by: mvillmow <4211002+mvillmow@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pipeline config YAML in configs/pipelines/ is not consumed by any code

1 participant