Skip to content

Refactor: remove legacy pipeline.Pipeline; have engine consume compose pipeline #449

Description

@sjawhar

Background

We currently have two pipeline representations:

  • pivot.compose.Pipeline (compositional API, produces ArtifactHandles)
  • legacy pivot.pipeline.pipeline.Pipeline (engine-facing; compose.Pipeline.build() bridges into it)

To fix cross-pipeline handles we’re moving toward a non-mutating compose.Pipeline.build() and building only the required closure of foreign stages. This raises the question of whether we can delete the legacy pipeline layer entirely and make the engine consume the compose representation (or a shared PipelineLike contract).

Goal

Eliminate pivot.pipeline.pipeline.Pipeline as a distinct concept and have the engine / DAG builder operate on a single canonical pipeline representation (ideally pivot.compose.Pipeline or a small shared interface).

Proposed direction (high level)

  • Define a minimal engine-consumable contract (stage inventory as RegistryStageInfo-equivalents, external dep resolution hook, DAG construction via pivot.engine.graph).
  • Implement that contract for compose.Pipeline (and optionally adapt legacy pipeline until removed).
  • Move/encapsulate resolve_external_dependencies() semantics behind the new contract (or re-home it).
  • Untangle any identity assumptions that currently depend on build-time prefix mutation.
  • Remove the legacy class + bridge code once engine is switched.

Acceptance criteria

  • Engine runs/explains using the new pipeline representation without going through compose.Pipeline.build() bridging.
  • Watch mode / caching / lockfile identity remain stable (or are deliberately migrated with tests).
  • External dependency resolution behavior remains equivalent (covered by tests).
  • pivot.pipeline.pipeline.Pipeline and related include/resolve code is deleted or clearly deprecated and unused.

Notes

  • Keep scope separate from the cross-pipeline handle fix; this is a follow-on refactor.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions