chore(docker): multi-stage build with prebuilt IB Gateway + auto-update workflows#683
Closed
junyuanz1 wants to merge 3 commits into
Closed
chore(docker): multi-stage build with prebuilt IB Gateway + auto-update workflows#683junyuanz1 wants to merge 3 commits into
junyuanz1 wants to merge 3 commits into
Conversation
…te workflows Rewrite the Dockerfile to a two-stage build: a python:3.14 builder stage that produces the thetagang wheel via uv, and a debian:12 runtime stage that installs IB Gateway and IBC from the prebuilt extrange/ibkr-docker release artifacts. This removes the need to vendor the TWS installer and patch IBC's Java logging at build time. - Pin IBC 3.24.0 and IB Gateway 10.45.1g via ENV vars. - Add image-files/start.sh and image-files/replace.sh as the new entrypoint/setup scripts. - Drop the now-unused entrypoint.bash, extract-installer.sh, docker/patch-ibc-java-logging.sh and its test. - Add scheduled workflows (update-ibc.yml, update-ibkr-gateway.yml) that open PRs bumping the IBC / IB Gateway versions in the Dockerfile when upstream releases appear.
The new self-contained Dockerfile downloads IB Gateway/IBC and builds the wheel internally, so the publish workflow no longer needs to cache/extract the TWS installer or run uv build beforehand. Remove those steps and let docker/build-push-action build the image directly from source context.
There was a problem hiding this comment.
Pull request overview
This PR updates the project’s container build to a two-stage Dockerfile that builds a thetagang wheel in a Python builder stage, then assembles a Debian runtime image with IB Gateway + IBC sourced from extrange/ibkr-docker release artifacts. It also adds scheduled GitHub Actions workflows to automatically open PRs when upstream IBC / IB Gateway versions change.
Changes:
- Replaced the existing Docker build approach with a multi-stage build that installs IB Gateway + IBC from upstream release artifacts and then installs the built wheel.
- Added new container startup/config override scripts (
image-files/start.sh,image-files/replace.sh) and removed legacy installer/patch scripts + their tests. - Added two scheduled workflows to auto-bump pinned
IBC_VERSION/IB_GATEWAY_VERSIONin the Dockerfile.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
Dockerfile |
Converts to multi-stage build; installs IB Gateway/IBC from upstream artifacts and installs the built wheel. |
image-files/start.sh |
New image entrypoint script (Xvfb + config replacement + thetagang exec). |
image-files/replace.sh |
New script to apply IBC_ env var overrides into an ini file. |
.github/workflows/update-ibc.yml |
Scheduled workflow to open PRs bumping IBC_VERSION. |
.github/workflows/update-ibkr-gateway.yml |
Scheduled workflow to open PRs bumping IB_GATEWAY_VERSION. |
entrypoint.bash |
Removed legacy entrypoint. |
extract-installer.sh |
Removed legacy TWS extraction flow. |
docker/patch-ibc-java-logging.sh |
Removed legacy IBC Java logging patch script. |
tests/test_patch_ibc_java_logging.py |
Removed tests for the deleted patch script. |
Comment on lines
+8
to
+12
| export DISPLAY=:1 | ||
|
|
||
| ./replace.sh ~/ibc/config.ini | ||
|
|
||
| exec /usr/local/bin/thetagang "$@" |
Comment on lines
+17
to
+33
| # Only select environment variables with IBC_ prefix, | ||
| # then trim that prefix out | ||
| prefix="IBC_" | ||
| env_vars=$(printenv | grep -E "^${prefix}.*" | cut -c $((${#prefix} + 1))-) | ||
|
|
||
| printf "Set variables:\n%s\n" "${env_vars}" | ||
|
|
||
| # Generate sed script file from override | ||
| script=$(sed -r 's/^((\w+=).*$)/\/^\2.*$\/c\\\1/' <<<"$env_vars") | ||
|
|
||
| # Replace in-place, making a backup | ||
| sed --in-place=.bak -r "$script" "$target" | ||
|
|
||
| printf "Changes made to %s:\n" "$target" | ||
| printf "%s\n" "$(diff "$target.bak" "$target")" | ||
|
|
||
| exit 0 |
Comment on lines
+16
to
+17
| ENV IBC_VERSION=3.24.0 | ||
| ENV IB_GATEWAY_VERSION=10.45.1g |
Comment on lines
+38
to
+41
| # Download and setup IBC | ||
| RUN wget2 https://github.com/IbcAlpha/IBC/releases/download/${IBC_VERSION}/IBCLinux-${IBC_VERSION}.zip -O ibc.zip \ | ||
| && unzip ibc.zip -d /opt/ibc \ | ||
| && rm ibc.zip |
Comment on lines
+22
to
+42
| run: | | ||
| # Fetch latest release from GitHub API | ||
| LATEST_RELEASE=$(curl -s https://api.github.com/repos/IbcAlpha/IBC/releases/latest) | ||
| UPSTREAM_VERSION=$(echo "$LATEST_RELEASE" | jq -r '.tag_name') | ||
|
|
||
| # Extract current version from our Dockerfile | ||
| CURRENT_VERSION=$(grep -oP 'IBC_VERSION=\K[0-9]+\.[0-9]+\.[0-9]+' Dockerfile) | ||
|
|
||
| echo "upstream_version=$UPSTREAM_VERSION" >> $GITHUB_OUTPUT | ||
| echo "current_version=$CURRENT_VERSION" >> $GITHUB_OUTPUT | ||
|
|
||
| echo "Upstream version: $UPSTREAM_VERSION" | ||
| echo "Current version: $CURRENT_VERSION" | ||
|
|
||
| if [ "$UPSTREAM_VERSION" != "$CURRENT_VERSION" ]; then | ||
| echo "update_needed=true" >> $GITHUB_OUTPUT | ||
| echo "Update needed: $CURRENT_VERSION -> $UPSTREAM_VERSION" | ||
| else | ||
| echo "update_needed=false" >> $GITHUB_OUTPUT | ||
| echo "Already up to date" | ||
| fi |
Comment on lines
+47
to
+54
| UPSTREAM_VERSION="${{ steps.check-version.outputs.upstream_version }}" | ||
| CURRENT_VERSION="${{ steps.check-version.outputs.current_version }}" | ||
|
|
||
| # Update IBC_VERSION in Dockerfile | ||
| sed -i "s/IBC_VERSION=${CURRENT_VERSION}/IBC_VERSION=${UPSTREAM_VERSION}/" Dockerfile | ||
|
|
||
| echo "Updated Dockerfile:" | ||
| grep "IBC_VERSION" Dockerfile |
Comment on lines
+22
to
+45
| run: | | ||
| # Fetch upstream Dockerfile | ||
| UPSTREAM_DOCKERFILE=$(curl -s https://raw.githubusercontent.com/extrange/ibkr-docker/master/stable/Dockerfile) | ||
|
|
||
| # Extract version from upstream (format: ibgateway-X.XX.Xx-standalone) | ||
| UPSTREAM_VERSION=$(echo "$UPSTREAM_DOCKERFILE" | grep -oP 'ibgateway-\K[0-9]+\.[0-9]+\.[0-9]+[a-z]?' | head -1) | ||
|
|
||
| # Extract current version from our Dockerfile | ||
| CURRENT_VERSION=$(grep -oP 'IB_GATEWAY_VERSION=\K[0-9]+\.[0-9]+\.[0-9]+[a-z]?' Dockerfile) | ||
|
|
||
| echo "upstream_version=$UPSTREAM_VERSION" >> $GITHUB_OUTPUT | ||
| echo "current_version=$CURRENT_VERSION" >> $GITHUB_OUTPUT | ||
|
|
||
| echo "Upstream version: $UPSTREAM_VERSION" | ||
| echo "Current version: $CURRENT_VERSION" | ||
|
|
||
| if [ "$UPSTREAM_VERSION" != "$CURRENT_VERSION" ]; then | ||
| echo "update_needed=true" >> $GITHUB_OUTPUT | ||
| echo "Update needed: $CURRENT_VERSION -> $UPSTREAM_VERSION" | ||
| else | ||
| echo "update_needed=false" >> $GITHUB_OUTPUT | ||
| echo "Already up to date" | ||
| fi | ||
|
|
The IB Gateway installer ships only an x64 JRE, so the arm64 build leg failed with 'jre/bin/java: not found' when the bundled JRE ran under emulation. On aarch64, install Azul Zulu's JavaFX-enabled JRE and point the installer at it via app_java_home (approach borrowed from gnzsnz/ib-gateway-docker); amd64 keeps using the bundled JRE. Also add dist/ to .dockerignore so a stale local wheel isn't copied into the build context alongside the freshly built one (pip would otherwise try to install two thetagang versions and fail to resolve). Verified with a native linux/arm64 build: gateway installs and the thetagang CLI runs on the Zulu 17 arm64 JRE.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rewrites the Dockerfile to a two-stage build, keeps it multi-arch (amd64 + arm64), and adds automation to keep the IB Gateway / IBC versions current.
python:3.14-bookworm): builds thethetagangwheel withuv build.debian:12): installs IB Gateway (10.45.1g) and IBC (3.24.0) from a prebuilt installer, then installs the wheel.This removes the need to vendor the TWS installer, extract it at build time, and patch IBC's Java logging — the prebuilt installer handles that.
Credit / prior art
This is not original work — the Docker approach is adapted from the
extrange/ibkr-docker/gnzsnz/ib-gateway-dockerlineage. To be precise about what came from where:extrange/ibkr-docker: the multi-stage layout, downloading a prebuilt IB Gateway installer from anibkr-dockerGitHub release (the runtime stage pulls the installer fromextrange's release artifacts), the IBC fetch fromIbcAlpha/IBC, theyes '' | ./installerinvocation, and theimage-files/start.sh+image-files/replace.shscripts are adapted essentially verbatim fromextrange's Dockerfile.gnzsnz/ib-gateway-docker:gnzsnzis the actively-maintained fork ofextrange. IB Gateway only ships an x64 JRE, so onaarch64we install Azul Zulu's JavaFX-enabled arm64 JRE and point the installer at it viaapp_java_home— exactly the techniquegnzsnzuses.pythonbuilder stage that builds thethetagangwheel, and wiringstart.shto launchthetagang.Maintainability note: since
extrangeis largely inactive andgnzsnzis the maintained successor, it would likely be cleaner and more durable to base the image more directly ongnzsnz/ib-gateway-docker(or buildFROMits prebuilt multi-arch image and just layer thethetagangwheel + entrypoint on top), so the IB Gateway/IBC/JRE plumbing is maintained upstream instead of re-implemented here. This PR takes the self-contained route for now; happy to switch if preferred.Changes
Dockerfileto the multi-stage build; pinIBC_VERSION/IB_GATEWAY_VERSIONviaENV.linux/amd64,linux/arm64/v8) builds succeed.dist/to.dockerignoreso a stale local wheel can't be copied into the build context alongside the freshly built one.image-files/start.shandimage-files/replace.shas the new entrypoint/setup scripts (fromextrange/ibkr-docker).entrypoint.bash,extract-installer.sh,docker/patch-ibc-java-logging.sh, and its test..github/workflows/docker-publish.yml: the self-contained Dockerfile builds everything internally, so the workflow no longer caches/extracts the TWS installer or pre-builds the wheel.update-ibc.yml— checks the latest IbcAlpha/IBC release.update-ibkr-gateway.yml— tracks theextrange/ibkr-dockerstable Dockerfile.Verification
Built natively for
linux/arm64on Apple Silicon: IB Gateway installs cleanly and thethetagangCLI runs on the Zulu 17 arm64 JRE.