[#749] fix: pin DuckLake extension via Docker bake + catalog preflight (incident 6)#750
Open
mlennie wants to merge 1 commit into
Open
[#749] fix: pin DuckLake extension via Docker bake + catalog preflight (incident 6)#750mlennie wants to merge 1 commit into
mlennie wants to merge 1 commit into
Conversation
Pins the DuckLake extension binary at Docker build time. The bake runs through the `duckdb@1.4.4` npm binding (not the CLI from base-deps, which is a different DuckDB engine version), so the extension lands at /root/.duckdb/extensions/v1.4.4/, the path the runtime server inspects when calling INSTALL ducklake; LOAD ducklake;. Verified offline LOAD in the built image with `docker run --network none`. Adds a postgres-side preflight in StorageManager.attachDuckLakeCatalog that reads ducklake_metadata.version directly (via plain postgres ATTACH, not DuckLake) and throws ConnectionAuthError -> HTTP 422 when the catalog format is newer than the baked extension supports. Reuses the #741 ConnectionAuthError -> 422 mapping. Failures of the preflight itself (table missing, query timeout, connect failure) return undefined and fall through to the existing ATTACH path so the preflight is not load-bearing for unrelated errors. Addresses incident item #6. The deliberate DuckLake 1.0 upgrade (duckdb -> 1.5+, @malloydata bump, AUTOMATIC_MIGRATION=TRUE) is a separate follow-up PR. README/public docs updates are intentionally deferred to a follow-up PR after #743 (the PG_CONNECT_TIMEOUT_SECONDS docs) merges, to avoid conflicts on the env-vars table. Signed-off-by: Monty Lennie <montylennie@gmail.com>
| # the extension at the path the runtime server uses, so its INSTALL | ||
| # ducklake; LOAD ducklake; at startup finds it and skips the network | ||
| # fetch (the actual pin). | ||
| RUN bun -e "const d=require('duckdb');const db=new d.Database(':memory:');await new Promise((r,j)=>db.exec('INSTALL ducklake; LOAD ducklake;',e=>e?j(e):r()));const rows=await new Promise((r,j)=>db.all(\"SELECT extension_version, install_path FROM duckdb_extensions() WHERE extension_name='ducklake'\",(e,x)=>e?j(e):r(x)));console.log('DuckLake baked:',JSON.stringify(rows));" || \ |
Collaborator
There was a problem hiding this comment.
Can we get rid of the install duckdb nodebindings then?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Issue: #749
Fixes incident item number 6 in slack https://credible-data.slack.com/archives/C085VT95H2T/p1778641006406909?thread_ts=1778630898.862899&cid=C085VT95H2T (the May 2026 DuckLake catalog version mismatch).
Pins the DuckLake extension at Docker build time via the
duckdb@1.4.4npm binding so all workers boot with the same extension binary. The bake lands at/root/.duckdb/extensions/v1.4.4/linux_arm64/, thepath the runtime server inspects (not the path the CLI installer in
base-depswould use, which is for a different DuckDB engine version).Also adds a postgres-side preflight in
StorageManager.attachDuckLakeCatalogthat readsducklake_metadata.versiondirectly via a plainTYPE postgresATTACH (not DuckLake) and throwsConnectionAuthError→ 422 when the catalog format is newer than the baked extension supports. Preflight failures (missing table, query timeout, connect failure) return
undefinedand fall through to the existing ATTACH path soit's not load-bearing.
Why
In May 2026, prod's DuckLake catalog had been written at format version 1.0.0 while Publisher workers ran an older extension that only supported up to 0.3. The workaround was a manual catalog downgrade. Root
cause: nothing pinned the DuckLake extension binary on the worker side, so workers and the catalog could drift independently.
Verified:
DuckLake baked: [{"extension_version":"3f1b372","install_path":"/root/.duckdb/extensions/v1.4.4/linux_arm64/..."}].docker run --network none ... bun -e "...LOAD ducklake..."succeeds, proving the runtime uses the baked binary (not a network re-fetch).Out of scope (separate PRs)
duckdbto 1.5+, bump@malloydata/db-duckdb, addAUTOMATIC_MIGRATION=TRUE). Irreversible on prod data, deserves its own deliberate rollout.Test plan
docker_smoke_testbuild log shows the newDuckLake baked: [...]line withv1.4.4in the install path.