Skip to content

ci: build base/gromacs on hosted runners -> private Docker Hub #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bielj merged 1 commit into from
Jun 30, 2026
Merged

ci: build base/gromacs on hosted runners -> private Docker Hub #9

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 59 additions & 39 deletions .github/workflows/build-images.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
name: Build & push md-workflows images

# Builds the three-stage image chain on the self-hosted diffuse builder and pushes to Harbor.
# Builds the three-stage image chain and publishes it. The ChimeraX-dependent base/gromacs
# builds run on GitHub-hosted runners (open egress reaches cgl.ucsf.edu + Docker Hub) and push
# the intermediates to a PRIVATE Docker Hub repo. Only the final actl assembly runs on the
# self-hosted diffuse builder, because only it pushes to Harbor (reachable over Tailscale).
#
# Security: this workflow has NO pull_request trigger, so fork PRs can never reach the
# self-hosted runner or the Harbor credentials. Heavy builds run only on push to `astera`
# (path-filtered) and manual dispatch. Credentials live in the `harbor` GitHub Environment,
# which must be restricted to the `astera` deployment branch; the `diffuse-sh-builder` runner
# should sit in a runner group scoped to this repo. See the runbook in the PR description.
# Security: this workflow has NO pull_request trigger, so fork PRs can never reach the runners
# or the publish credentials. Heavy builds run only on push to `astera` (path-filtered) and
# manual dispatch. Credentials live in the `harbor` GitHub Environment (branch-restricted to
# `astera`, with required reviewers if configured).
#
# NOTE: runs-on matches only the unique custom label `diffuse-sh-builder` (not `self-hosted`),
# because that runner was registered with --no-default-labels and so lacks the implicit
# `self-hosted` label. The custom label alone unambiguously targets it.
# NOTE: the actl job's runs-on matches only the unique custom label `diffuse-sh-builder` (not
# `self-hosted`), because that runner was registered with --no-default-labels and so lacks the
# implicit `self-hosted` label. The custom label alone unambiguously targets it.
on:
push:
branches: [astera]
Expand All @@ -31,10 +33,14 @@ concurrency:
cancel-in-progress: true

env:
REGISTRY: harbor.astera.sh
# base + gromacs (bake the non-redistributable ChimeraX) live in the proprietary project,
# distinguished by a stage tag prefix. The final consumable actl image lands in library.
PROPRIETARY_IMAGE: harbor.astera.sh/diffuse-proprietary/md-workflows
HARBOR_REGISTRY: harbor.astera.sh
# Docker Hub registry host for login + image refs. NOTE: it is `docker.io`, NOT the website
# `hub.docker.com` (that's the UI, not a registry endpoint).
DOCKERHUB_REGISTRY: docker.io
# base + gromacs bake the non-redistributable ChimeraX, so they publish to the PRIVATE Docker
# Hub repo (not public), distinguished by a stage tag prefix. The final consumable actl image
# lands in the Harbor library project.
DOCKERHUB_IMAGE: docker.io/diffuseproject/md
LIBRARY_IMAGE: harbor.astera.sh/library/md-workflows

jobs:
Expand Down Expand Up @@ -62,19 +68,23 @@ jobs:
base:
name: Build base
needs: version
runs-on: [diffuse-sh-builder]
runs-on: ubuntu-latest
environment: harbor
outputs:
digest: ${{ steps.build.outputs.digest }}
steps:
- uses: actions/checkout@v4
- name: Free disk space
run: |
sudo rm -rf /usr/share/dotnet /opt/ghc /usr/local/lib/android /opt/hostedtoolcache/CodeQL || true
docker system prune -af || true
- uses: docker/setup-buildx-action@v3
- name: Login to Harbor
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ secrets.HARBOR_USERNAME }}
password: ${{ secrets.HARBOR_PASSWORD }}
registry: ${{ env.DOCKERHUB_REGISTRY }}
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- id: build
uses: docker/build-push-action@v6
with:
Expand All @@ -84,29 +94,33 @@ jobs:
push: true
provenance: false
tags: |
${{ env.PROPRIETARY_IMAGE }}:base-${{ needs.version.outputs.semver }}
${{ env.PROPRIETARY_IMAGE }}:base-${{ needs.version.outputs.build }}
${{ env.PROPRIETARY_IMAGE }}:base-${{ needs.version.outputs.sha }}
${{ env.PROPRIETARY_IMAGE }}:base
cache-from: type=registry,ref=${{ env.PROPRIETARY_IMAGE }}:base-buildcache
cache-to: type=registry,ref=${{ env.PROPRIETARY_IMAGE }}:base-buildcache,mode=max
${{ env.DOCKERHUB_IMAGE }}:base-${{ needs.version.outputs.semver }}
${{ env.DOCKERHUB_IMAGE }}:base-${{ needs.version.outputs.build }}
${{ env.DOCKERHUB_IMAGE }}:base-${{ needs.version.outputs.sha }}
${{ env.DOCKERHUB_IMAGE }}:base
cache-from: type=registry,ref=${{ env.DOCKERHUB_IMAGE }}:base-buildcache
cache-to: type=registry,ref=${{ env.DOCKERHUB_IMAGE }}:base-buildcache,mode=max

gromacs:
name: Build gromacs
needs: [version, base]
runs-on: [diffuse-sh-builder]
runs-on: ubuntu-latest
environment: harbor
outputs:
digest: ${{ steps.build.outputs.digest }}
steps:
- uses: actions/checkout@v4
- name: Free disk space
run: |
sudo rm -rf /usr/share/dotnet /opt/ghc /usr/local/lib/android /opt/hostedtoolcache/CodeQL || true
docker system prune -af || true
- uses: docker/setup-buildx-action@v3
- name: Login to Harbor
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ secrets.HARBOR_USERNAME }}
password: ${{ secrets.HARBOR_PASSWORD }}
registry: ${{ env.DOCKERHUB_REGISTRY }}
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- id: build
uses: docker/build-push-action@v6
with:
Expand All @@ -116,14 +130,14 @@ jobs:
push: true
provenance: false
build-args: |
BASE_IMAGE=${{ env.PROPRIETARY_IMAGE }}@${{ needs.base.outputs.digest }}
BASE_IMAGE=${{ env.DOCKERHUB_IMAGE }}@${{ needs.base.outputs.digest }}
tags: |
${{ env.PROPRIETARY_IMAGE }}:gromacs-${{ needs.version.outputs.semver }}
${{ env.PROPRIETARY_IMAGE }}:gromacs-${{ needs.version.outputs.build }}
${{ env.PROPRIETARY_IMAGE }}:gromacs-${{ needs.version.outputs.sha }}
${{ env.PROPRIETARY_IMAGE }}:gromacs
cache-from: type=registry,ref=${{ env.PROPRIETARY_IMAGE }}:gromacs-buildcache
cache-to: type=registry,ref=${{ env.PROPRIETARY_IMAGE }}:gromacs-buildcache,mode=max
${{ env.DOCKERHUB_IMAGE }}:gromacs-${{ needs.version.outputs.semver }}
${{ env.DOCKERHUB_IMAGE }}:gromacs-${{ needs.version.outputs.build }}
${{ env.DOCKERHUB_IMAGE }}:gromacs-${{ needs.version.outputs.sha }}
${{ env.DOCKERHUB_IMAGE }}:gromacs
cache-from: type=registry,ref=${{ env.DOCKERHUB_IMAGE }}:gromacs-buildcache
cache-to: type=registry,ref=${{ env.DOCKERHUB_IMAGE }}:gromacs-buildcache,mode=max

actl:
name: Build actl (consumable)
Expand All @@ -135,10 +149,16 @@ jobs:
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- name: Login to Harbor
- name: Login to Docker Hub (pull private gromacs intermediate)
uses: docker/login-action@v3
with:
registry: ${{ env.DOCKERHUB_REGISTRY }}
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to Harbor (push final image)
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
registry: ${{ env.HARBOR_REGISTRY }}
username: ${{ secrets.HARBOR_USERNAME }}
password: ${{ secrets.HARBOR_PASSWORD }}
- id: build
Expand All @@ -150,7 +170,7 @@ jobs:
push: true
provenance: false
build-args: |
GROMACS_IMAGE=${{ env.PROPRIETARY_IMAGE }}@${{ needs.gromacs.outputs.digest }}
GROMACS_IMAGE=${{ env.DOCKERHUB_IMAGE }}@${{ needs.gromacs.outputs.digest }}
tags: |
${{ env.LIBRARY_IMAGE }}:${{ needs.version.outputs.semver }}
${{ env.LIBRARY_IMAGE }}:${{ needs.version.outputs.build }}
Expand Down
Loading