Skip to content

deploy: build and verify the role images for real (#81) - #99

Merged
richardmhope merged 1 commit into
mainfrom
m0-verify-images
Jul 31, 2026
Merged

deploy: build and verify the role images for real (#81)#99
richardmhope merged 1 commit into
mainfrom
m0-verify-images

Conversation

@richardmhope

Copy link
Copy Markdown
Contributor

The images and compose stack merged in #80 had never been built or run — that session had no Docker
daemon, so static checks stood in for execution. This builds them, runs the stack end to end, fixes
what that contradicted, and adds a CI job so it stays proven.

What was actually broken

api image missing dramatiq iceberg_api.dispatch imports it, but only apps/engine declared it. The workspace install hid this locally; the container died on startup with ModuleNotFoundError.
engine image could import the ORM iceberg-core depended on sqlmodel unconditionally, so iceberg_core.db/models were importable in the engine — the ADR 0002 boundary held by convention, not by construction.
the obvious probe was wrong find_spec reported sqlalchemy absent but iceberg_core.db present: it locates a file without importing it. Only an import distinguishes "shipped" from "usable".
venvs are not relocatable Adopting --no-editable and copying only /src/.venv to /app left every console script with a shebang pointing nowhere — exec /app/.venv/bin/uvicorn: no such file or directory.

What changed

  • sqlmodel moves to an iceberg-core[db] extra that only apps/api requests. The engine's
    dependency closure now contains no ORM at all, so the isolation is a property of the build rather
    than of everyone remembering.
  • deploy/docker/verify-images.sh + make images-verify, run as a CI job. Nine checks against
    the built artefacts: both build, both run as uid 10001, both entrypoints serve, the engine has no
    importable database package and no DB/master-key variable, and the api resolves its packaged
    migrations. Adapted from Add api and engine role images (#20) #79 as the issue suggested, with the probe corrected.
  • alembic.ini moves inside the package so it ships in the wheel, and
    python -m iceberg_api migrate becomes the single entry point for compose, the tests, and the
    Helm pre-upgrade Job that Helm: secrets, values & migration Job #62 will need. The runtime stages now carry the venv alone — no source
    tree, no uv, no build dependencies.
  • tests/test_deploy_invariants.py strips comments before its substring checks (the engine
    Dockerfile now states "no Postgres driver, no alembic", which is the rule, not a breach of it),
    gains a check that core's base deps stay ORM-free, and asserts the CI job exists.

Verified on a Docker host

make images-verify   → all nine checks pass
make up              → api, engine, postgres, redis all healthy
make migrate         → five revisions applied
make seed            → demo source created
make scale N=3       → three engine replicas
make down            → clean teardown
curl /healthz        → 200 ;  curl /  → 303 to login

make lint, make type and the full suite pass. The README quickstart matches what happens.

One note for the maintainer

My local .env was stale (pre-dating REDIS_PASSWORD/ICEBERG_MASTER_KEY) and the dev Postgres
volume held data in postgres:18's pre-relocation layout, so neither could start. I backed the file
up to .env.backup-stale-20260731 and used make destroy on the dev volume. Nothing in the repo;
mentioned only so the local state isn't a surprise.

Closes #81

🤖 Generated with Claude Code

The images and compose stack merged in #80 had never been built or run — that
session had no Docker daemon, so static checks stood in for execution. Running
them found four things wrong, which is roughly what #81 predicted.

**What was broken**

* The api image was missing `dramatiq`. `iceberg_api.dispatch` imports it, but
  only `apps/engine` declared it; the workspace install hid that locally, and the
  api container died on startup with ModuleNotFoundError. Declared now.
* `iceberg_core.db`/`models` were importable in the engine image, because
  `iceberg-core` depended on sqlmodel unconditionally. sqlmodel moves to an
  `iceberg-core[db]` extra that only `apps/api` requests, so the engine's
  dependency closure now contains no ORM at all (ADR 0002) rather than merely
  not using one.
* The engine image reported `sqlalchemy` absent under a `find_spec` probe while
  `iceberg_core.db` looked present — find_spec locates a file without importing
  it. The probe imports instead, which is the property that matters.
* `--no-editable` + copying only the venv broke every console script: a venv is
  not relocatable, so building at /src and copying to /app left `uvicorn` with a
  shebang pointing nowhere. The builder now works at the runtime path.

**What is new**

`deploy/docker/verify-images.sh` (`make images-verify`), run as a CI job: builds
both images and asserts they build, run as uid 10001, serve their endpoints, keep
the engine free of any importable database package or DB/master-key variable, and
let the api resolve its packaged migrations. That is a stronger form of the ADR
0002 check than reading Dockerfile text, and it re-runs on every PR.

`alembic.ini` moves inside the package so it ships in the wheel, and
`python -m iceberg_api migrate` becomes the one entry point for compose, the
Helm pre-upgrade Job (#62) and the migration tests — the runtime image installs
the package and copies no source tree, so a repo-relative config would not be
there. The runtime stages now carry the venv alone: no source, no uv, no build
dependencies.

`tests/test_deploy_invariants.py` strips comments before its substring checks —
the engine Dockerfile now states "no Postgres driver, no alembic" in prose, which
is the rule, not a violation of it — and asserts the CI job exists.

**Verified on a Docker host**

Both images build; `make images-verify` passes all nine checks; `make up` →
healthy stack, `make migrate` applies all five revisions, `make seed` loads
fixtures, `make scale N=3` runs three engines, `make down` tears down. /healthz
answers 200 and the console redirects to login. README quickstart matches.

Closes #81

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@richardmhope
richardmhope merged commit 91bdc90 into main Jul 31, 2026
3 checks passed
@richardmhope
richardmhope deleted the m0-verify-images branch July 31, 2026 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Verify the role images and compose stack on a Docker host

1 participant