Support extensions that depend on unmerged Envoy dynamic module SDK/ABI branches

[codefromthecrypt]

TL;DR

Make BOE own the branch-only Envoy dynamic module workflow: resolve the Envoy branch, build the matching Envoy runtime, build composer/extensions against that same branch SDK, key the native cache by that tuple, and run normally.

Example:

boe run --envoy-branch codefromthecrypt:get_dynamic_metadata --local ./extensions/my-filter

Background

BOE can run extensions against released Envoy versions, but dynamic module development sometimes depends on an Envoy SDK or ABI change that is still only available on a branch. In that case, Envoy semver is not the dependency that matters. The Envoy runtime, dynamic modules SDK, composer build, local extension build, and native cache all need to resolve from the same Envoy branch commit.

Today this requires too much manual coordination: build Envoy separately, make composer use the matching SDK, avoid stale BOE cache entries, and manually handle Docker if the runtime needs to run in a container. BOE should own this path so downstream projects can use BOE directly in their own tests.

This is similar in shape to two previous BOE changes:

• #152 made missing platform artifacts seamless by fetching source artifacts and building them locally instead of requiring the user to work around missing binaries.
• #159 added boe run --docker so BOE owns the Docker setup when func-e cannot provide the needed Envoy binary for the user platform.

This is the same class of UX fix, but for an Envoy branch dependency that affects the dynamic modules SDK or ABI.

Dependency graph

flowchart TD
  A["Envoy branch source<br/>user:branch"] --> B["Resolved Envoy commit"]
  B --> C["Envoy runtime<br/>//source/exe:envoy-static"]
  B --> D["dynamic_modules SDK<br/>same checkout"]
  D --> E["composer build"]
  D --> F["local extension build"]
  E --> G["native cache<br/>commit + SDK + composer + toolchain"]
  F --> G
  C --> H["boe run"]
  G --> H
  H --> I["host mode"]
  H --> J["Docker mode"]

Loading

Desired behavior

BOE should accept an Envoy branch source using the same shape as gh pr create --head <user>:<branch>, for example codefromthecrypt:get_dynamic_metadata, and use it as the source of truth for the whole dynamic module dependency graph.

The run flow should resolve the branch to a concrete Envoy commit, build the needed Envoy runtime target from that commit, build composer and local extensions against the dynamic modules SDK from the same checkout, and cache native outputs by that resolved dependency tuple.

This should work through both boe run and boe run --docker. The Docker path should be BOE-owned, not a separate manual command copied from generated config.

Acceptance criteria

• A branch-only Envoy SDK/ABI change can be used by a BOE extension before it is merged or released.
• The Envoy runtime, dynamic modules SDK, composer build, extension build, and cache entry all resolve from the same Envoy commit.
• Stale native outputs from a different Envoy SDK/runtime are not reused silently.
• Host mode and Docker mode both work through BOE.
• BOE reports the resolved Envoy commit, SDK source, composer cache key, and runtime path.
• The new path has focused BOE validation and can be run through the project’s make-based workflow.

[codefromthecrypt]

ps probably we can decouple the actual envoy build to func-e somehow (like a magic fake version number), but for a while the ABI/SDK part will still be tough to test even if we decouple the envoy build. Meanwhile, I'm not blocked, just I have a fairly tricky Makefile target tree to test a PR, especially when testing inside docker.