Docker Image Layer Caching Optimization for CI Pipeline

[JamesEjembi]

Problem Statement

CI pipeline builds Docker images from scratch each time, taking 12+ minutes for the full service stack. Implement multi-stage Dockerfiles with pinned base image digests, layer caching via GitHub Actions cache backend, and scheduled base image refresh for security patches.

Technical Bounds

• Base image layers cached in registry: max 5GB per repository.
• Cache hit restore: full layer restore < 30s from cache.
• Build time target: < 3 minutes on cache hit, < 8 minutes on cache miss.
• Weekly base image refresh: rebuild from scratch to pick up security patches.
• Cache invalidation: on Cargo.lock change, invalidate dependency layer.

Steps

1. Restructure Dockerfiles with multi-stage builds: builder stage (Rust compilation), runtime stage (distroless base).
2. Pin base image digests: FROM rust:1.75-slim-bookworm@sha256:abc123.
3. Configure GitHub Actions cache backend: docker/build-push-action@v5 with cache-from: type=gha and cache-to: type=gha,mode=max.
4. Separate dependency compilation layer: copy Cargo.toml and Cargo.lock first, run cargo fetch, then copy source.
5. Add weekly scheduled workflow: cron: 0 2 * * 0 rebuilds all images with latest base images.
6. Benchmark and document build time improvements in README.