Switch to CompilerCaching.jl#794
Merged
Merged
Conversation
This was referenced May 12, 2026
Member
That's unfortunate and will likely mean that we will have to maintain the prior version of GPUCompiler until there is a new LTS. |
Member
Author
I'm not convinced that's needed. As long as there's no breaking releases in the back-ends, users can simply use different versions of say CUDA.jl 6.x depending on which Julia version they're using. And for critical features I'd rather they request backports over there rather than maintaining multiple compiler stacks. |
Member
|
The issue is that for Enzyme we won't be able to drop 1.10 support (due to DiffEq and so forth) and there the situation is less stable than for the GPU backends. |
Member
Author
|
Grmbl. I'll try to come up with something that keeps 1.10 working then. |
maleadt
force-pushed
the
tb/compilercaching
branch
2 times, most recently
from
806ec79 to
a1e6c06
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #794 +/- ##
==========================================
+ Coverage 80.41% 81.40% +0.99%
==========================================
Files 25 25
Lines 4866 4755 -111
==========================================
- Hits 3913 3871 -42
+ Misses 953 884 -69 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
maleadt
force-pushed
the
tb/compilercaching
branch
3 times, most recently
from
734f95a to
fe40456
Compare
This was referenced Jul 10, 2026
maleadt
force-pushed
the
tb/compilercaching
branch
from
5b728f4 to
014ed68
Compare
Drop the hand-rolled CodeCache, on-disk kernel cache, and various pre-1.11 compatibility shims; route inference and CI lookup through CompilerCaching.CacheView with consumer-defined results structs attached to each CodeInstance. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously partitioned only by `typeof(target)`, which collided across target instances that produce different IR (e.g. different macOS, SM arch, or CPU features). Folding the full target and params into the token matches the spirit of `runtime_slug`, while staying within the inference-determinant scope of the docstring. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds optional `bitcode`/`bitcode!` hooks on the consumer's `results_type`. When opted in, `emit_function!` reads renamed per-function bitcode from the cached CI on a hit, and writes it on a miss. Cross-session persistence rides on package precompilation; a small session-local assembled-module cache (keyed by `(cache_owner, opaque_pointers)`) keeps the within-session fast path. Drops the `runtime_slug` interface — `cache_owner` now subsumes its role of identifying compatible IR. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`emit_function!` now memoizes each `gpu_*` runtime function's renamed, post-irgen LLVM bitcode on its own `CodeInstance`'s `analysis_results` when the back-end opts in via the new `bitcode`/`bitcode!` trait pair. Cross-session persistence rides on package precompilation; the session-local `_runtime_libs` assembled-module cache keeps repeated within-session linking cheap. Gated on `HAS_INTEGRATED_CACHE` so 1.10 falls through to plain compile + link (still serviced by `_runtime_libs`). With Metal opted in, a second kernel compile in the same session — even after `reset_runtime()` invalidates `_runtime_libs` — completes ~25× faster than a cold rebuild on 1.12, because each runtime function is now a parse-and-link instead of a full Julia → LLVM run. Restores the optimization originally landed in aa4e64d (reverted in 566811d for 1.10 compat); the new version sits behind the same infrastructure as `cached_compilation`, so the 1.10 path is no longer load-bearing on the trait being callable. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
On Julia 1.13+, `jl_emit_native_impl` itself sets every `jl_sysimg_gvar` to a null initializer before returning (aotcompile.cpp:865), leaving relocation to the caller. On 1.12, `jl_emit_native_impl` instead bakes session-local pointer values into the initializer via `literal_static_pointer_val` — so without intervention, the bitcode we hand to `bitcode!` for caching carries live pointers from the current session and isn't safe to reload in a future one. After collecting `gv_to_value` from `jl_get_llvm_gvs` / `jl_get_llvm_gvs_globals`, immediately reset each tracked GV's initializer to null. `relocate_gvs!` at the toplevel link step then re-applies the session-current values regardless of which Julia we're on, so optimization still sees the resolved constants. On 1.13+ the null-out is a no-op (Julia already nulled them); on 1.12 this is what makes per-CI runtime bitcode caching genuinely cross-session-safe for back-ends that pull in `julia.constgv`-touching runtime functions (CUDA's `gc_pool_alloc`, `box_*`/`unbox_*`, …). Metal's stubs don't trip this either way. Verified: 27 `julia.constgv` GVs in Metal's cached runtime-fn bitcode on 1.12, all with null initializers post-change (was 27/27 non-null). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CompilerCaching becomes a strong dep (it loads as an empty shell on 1.10,
so there's no overhead). The `GPUCompilerCompilerCachingExt` extension —
which existed just to wire the parametric `CC.finish!` override that
attaches a `CachedResult{V}` to every inferred CodeInstance — moves into
`jlgen.jl` directly, gated on `HAS_INTEGRATED_CACHE`. Same code, one less
file.
With CompilerCaching available unconditionally we can also drop the inline
copies of its inference machinery from `jlgen.jl`:
- `drive_inference!` on 1.11+ is now a two-line delegation to
`CompilerCaching.typeinf!` + `get(cache, mi, nothing)`. The 1.10
implementation (which talks to the per-interpreter `CodeCache`) moves to
`deprecated.jl`.
- `collect_codeinfos` / `_ci_codeinfo` go away; the single call site in
`compile_method_instance` calls `CompilerCaching.get_codeinfos` directly.
- `StackedMethodTable` is re-exported from CompilerCaching on 1.11+; the
1.10 variant (with the older `MethodMatchResult`-shape `findall`) moves
to `deprecated.jl`.
Net result: ~200 lines deleted from `jlgen.jl`, no behavior change. All
1.10-only code is now in `deprecated.jl`, ready to disappear in one diff
when 1.10 support drops.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the hand-rolled `CC.finish!` override with `@setup_results GPUInterpreter` plus a one-line `CompilerCaching.results_type` trait that reads V from the interpreter's type parameter. Same generated code, less boilerplate. `drive_inference!` collapses to a one-liner calling `CompilerCaching.typeinf!(interp, mi)` — which now constructs the CacheView internally and returns the root CI directly, saving the lookup the old cache-taking form required. The `cache_view(interp)` helper inside `jlgen.jl` goes away with it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…rotocol.
The old `bitcode(results)` / `bitcode!(results, bytes)` pair was a single-purpose
hook bolted onto the consumer's results struct. Extending it to memoize more
phases (LLVM IR, intermediate AIR, whatever) meant adding a parallel pair per
phase. Worse, it required the consumer's results struct to be in the loop on
every cache touch — which forced `rtlib.jl` to know about `CompilerCaching` to
do the per-CI lookup that fetched the right results instance.
Replace with a back-end-managed key→bytes protocol on `CompilerJob`:
cache_get(job::CompilerJob, key::Symbol) -> Union{Nothing, Vector{UInt8}}
cache_put!(job::CompilerJob, key::Symbol, ::Vector{UInt8})
GPUCompiler hands the back-end a job + a key (`:llvm_ir` currently — the
post-irgen LLVM bitcode for runtime library functions). The back-end stores it
wherever it likes — typically on a CI's `analysis_results` via CompilerCaching,
but it could equally be an in-memory `Dict`, on-disk storage, or nothing. The
default no-op pair means no caching.
`rtlib.jl` no longer imports `CompilerCaching` — it just calls the hooks. New
phase keys can be added without growing the API surface; back-ends opt in
selectively by matching on keys they care about.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the previous V-threaded design (GPUInterpreter{V}, results_type(job),
@setup_results, cache_get/cache_put!, cache_view) with a single back-end-facing
entry point:
cached_results(::Type{V}, job::CompilerJob)::V
which returns the (lazily created) results struct for a job. Back-ends define
one mutable struct holding their per-stage artifacts, check completeness, and
compile into it — a single code path on all supported Julia versions:
- On 1.11+, the struct lives on the CodeInstance in Julia's integrated cache
(running inference to create one when needed), wrapped in a config-keyed
JobResults container. CompilerCaching attaches results lazily, so the
GPUInterpreter no longer carries a results type, and independent consumers
(e.g. our own runtime-library cache) can attach to the same CI.
- On 1.10, the struct lives in a session-local Dict keyed by the same job
identity, kept alongside the other legacy code in deprecated.jl.
Keying results by the full CompilerConfig (not just cache_owner) fixes a
latent bug in the previous design where two jobs differing only in codegen
settings — e.g. the kernel name — would share artifacts. The owner token
still covers only what affects inference, so inference results remain shared
across such jobs.
The runtime library now uses the same mechanism: emit_function! memoizes each
runtime function's renamed bitcode in a RuntimeFunctionResults attached to its
CI, replacing the cache_get/cache_put! protocol. Back-ends no longer opt in:
runtime bitcode persists through precompilation automatically on 1.11+.
Also moves the 1.10 drive_inference! definition from deprecated.jl to
jlgen.jl: its signature references GPUInterpreter, which isn't defined yet
when deprecated.jl is included (1.10 loading was broken on the previous
branch), and fixes the ptx precompile test to construct its cache token from
the standalone package's helper module (the sandbox copy defines a distinct
CompilerParams type, so its token can never match the precompiled CIs).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dynamic construction of the cache token made a cached lookup ~3x slower than the legacy cached_compilation path (684 vs 234 ns); specialized, it is now faster (166 vs 182 ns). Instantiations are bounded: one per back-end (and results type). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A Vector field made the target mutable under jl_egal, so owner tokens and configs deserialized from package images never matched and cached kernels were silently recompiled. Use a single --spirv-ext specifier string instead, mirroring LLVM feature strings (and GCN's features). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Artifacts derived from IR with relocated GVs embed absolute pointers from the precompilation process. Mark such jobs during output generation and drop their JobResults entries from an atexit hook, which runs before jl_write_compiler_output: within-session lookups still hit, but later sessions recompile instead of loading dangling pointers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CIs deposited by our own precompile workload carry world ages from the precompilation process and are dead weight in later sessions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The derived runtime config inherited cosmetic fields like name=, so runtime function artifacts were cached (and persisted) once per kernel config variation instead of once per cache owner. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The debug level (among other codegen-relevant settings) is baked into the runtime IR as a constant, so keying the session memo on cache_owner alone shared one assembled library across debug levels. Back-ends no longer need to widen their owner token to compensate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A vestige of the on-disk runtime cache: per-function bitcode now lives on CodeInstances that Julia invalidates automatically, and the session memo cannot contain stale entries at back-end load time. Also fixes the PTX runtime-rebuild test, which still poked the removed disk-cache global. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Kernels that make no calls produce CodeInstances without inference edges; on Julia 1.12 those are serialized with the revalidation sentinel but excluded from the edge-verification list, so they never revalidate after loading. Upstream bug, fixed by the 1.13 serialization rework. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Runtime functions like box_bool reference Julia singletons through julia.constgv globals. Kernel modules get those relocated when the fully-linked toplevel module is finalized, but runtime bitcode is cached before that step ever sees it: bake the session-absolute addresses into the per-function bitcode instead, and mark such jobs session-dependent so the baked pointers stay out of package images. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fold cached_results_if_present into cached_results, which is now lookup-only: it returns nothing when no code exists for the job, and back-ends compile and re-fetch (codegen drives inference itself, so a standalone inference walk is never needed). Consumers that want to attach results without generating code call precompile(job) first. Drop the method table from GPUCompilerCacheToken: everything inference derives from a job (method_table, method_table_view, inference_params, optimization_params) must be a pure function of the config's target and params values plus always_inline, so the default token already partitions correctly and back-ends no longer override cache_owner. Also chase the compilable MethodInstance in cache lookups on 1.14+, where inference caches vararg kernels under the vararg-widened signature rather than the fully-specialized job.source. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The on-disk runtime library cache it exercises does not exist anymore; runtime bitcode is cached per-CodeInstance and assembled libraries are memoized in-memory. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Julia codegen never emits an ijl_box_bool call when boxing a Bool: it inlines a select over the jl_true/jl_false singletons instead, so the runtime's box_bool is never linked into this kernel and the module contains no constgv slots to inspect. The test only appeared to pass locally because the NVPTX back-end is unavailable there, skipping the entire assembly section; CI was the first to actually run it. Relocation of runtime-library constgv globals remains covered by the native-target "runtime constgv relocation" testset, which runs (and passes) on all CI platforms. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
maleadt
force-pushed
the
tb/compilercaching
branch
from
7e23b95 to
02ebc13
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Will be a breaking release. Sadly also drops 1.10.