deploy: build and verify the role images for real (#81) - #99
Merged
Conversation
The images and compose stack merged in #80 had never been built or run — that session had no Docker daemon, so static checks stood in for execution. Running them found four things wrong, which is roughly what #81 predicted. **What was broken** * The api image was missing `dramatiq`. `iceberg_api.dispatch` imports it, but only `apps/engine` declared it; the workspace install hid that locally, and the api container died on startup with ModuleNotFoundError. Declared now. * `iceberg_core.db`/`models` were importable in the engine image, because `iceberg-core` depended on sqlmodel unconditionally. sqlmodel moves to an `iceberg-core[db]` extra that only `apps/api` requests, so the engine's dependency closure now contains no ORM at all (ADR 0002) rather than merely not using one. * The engine image reported `sqlalchemy` absent under a `find_spec` probe while `iceberg_core.db` looked present — find_spec locates a file without importing it. The probe imports instead, which is the property that matters. * `--no-editable` + copying only the venv broke every console script: a venv is not relocatable, so building at /src and copying to /app left `uvicorn` with a shebang pointing nowhere. The builder now works at the runtime path. **What is new** `deploy/docker/verify-images.sh` (`make images-verify`), run as a CI job: builds both images and asserts they build, run as uid 10001, serve their endpoints, keep the engine free of any importable database package or DB/master-key variable, and let the api resolve its packaged migrations. That is a stronger form of the ADR 0002 check than reading Dockerfile text, and it re-runs on every PR. `alembic.ini` moves inside the package so it ships in the wheel, and `python -m iceberg_api migrate` becomes the one entry point for compose, the Helm pre-upgrade Job (#62) and the migration tests — the runtime image installs the package and copies no source tree, so a repo-relative config would not be there. The runtime stages now carry the venv alone: no source, no uv, no build dependencies. `tests/test_deploy_invariants.py` strips comments before its substring checks — the engine Dockerfile now states "no Postgres driver, no alembic" in prose, which is the rule, not a violation of it — and asserts the CI job exists. **Verified on a Docker host** Both images build; `make images-verify` passes all nine checks; `make up` → healthy stack, `make migrate` applies all five revisions, `make seed` loads fixtures, `make scale N=3` runs three engines, `make down` tears down. /healthz answers 200 and the console redirects to login. README quickstart matches. Closes #81 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The images and compose stack merged in #80 had never been built or run — that session had no Docker
daemon, so static checks stood in for execution. This builds them, runs the stack end to end, fixes
what that contradicted, and adds a CI job so it stays proven.
What was actually broken
dramatiqiceberg_api.dispatchimports it, but onlyapps/enginedeclared it. The workspace install hid this locally; the container died on startup withModuleNotFoundError.iceberg-coredepended onsqlmodelunconditionally, soiceberg_core.db/modelswere importable in the engine — the ADR 0002 boundary held by convention, not by construction.find_specreportedsqlalchemyabsent buticeberg_core.dbpresent: it locates a file without importing it. Only an import distinguishes "shipped" from "usable".--no-editableand copying only/src/.venvto/appleft every console script with a shebang pointing nowhere —exec /app/.venv/bin/uvicorn: no such file or directory.What changed
sqlmodelmoves to aniceberg-core[db]extra that onlyapps/apirequests. The engine'sdependency closure now contains no ORM at all, so the isolation is a property of the build rather
than of everyone remembering.
deploy/docker/verify-images.sh+make images-verify, run as a CI job. Nine checks againstthe built artefacts: both build, both run as uid 10001, both entrypoints serve, the engine has no
importable database package and no DB/master-key variable, and the api resolves its packaged
migrations. Adapted from Add api and engine role images (#20) #79 as the issue suggested, with the probe corrected.
alembic.inimoves inside the package so it ships in the wheel, andpython -m iceberg_api migratebecomes the single entry point for compose, the tests, and theHelm pre-upgrade Job that Helm: secrets, values & migration Job #62 will need. The runtime stages now carry the venv alone — no source
tree, no uv, no build dependencies.
tests/test_deploy_invariants.pystrips comments before its substring checks (the engineDockerfile now states "no Postgres driver, no alembic", which is the rule, not a breach of it),
gains a check that core's base deps stay ORM-free, and asserts the CI job exists.
Verified on a Docker host
make lint,make typeand the full suite pass. The README quickstart matches what happens.One note for the maintainer
My local
.envwas stale (pre-datingREDIS_PASSWORD/ICEBERG_MASTER_KEY) and the dev Postgresvolume held data in postgres:18's pre-relocation layout, so neither could start. I backed the file
up to
.env.backup-stale-20260731and usedmake destroyon the dev volume. Nothing in the repo;mentioned only so the local state isn't a surprise.
Closes #81
🤖 Generated with Claude Code