Skip to content

Serialize build_callable under a global lock to prevent segfaults in threaded usage#227

Merged
sunxd3 merged 6 commits into
mainfrom
copilot/fix-julia-1-10-segfault
Jun 15, 2026
Merged

Serialize build_callable under a global lock to prevent segfaults in threaded usage#227
sunxd3 merged 6 commits into
mainfrom
copilot/fix-julia-1-10-segfault

Conversation

Copilot AI commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

On Julia 1.10, concurrent threads can race through build_callable during first-time callable IR generation, causing segfaults in threaded Particle Gibbs sampling. The existing GlobalMCCache lock only protects setindex!, not the full IR derivation + compilation pipeline.

  • Wrap build_callable body in a ReentrantLock so only one thread performs callable IR generation at a time
  • Cache lookup is rechecked after acquiring the lock to avoid redundant compilation
const build_callable_lock = ReentrantLock()

function build_callable(sig::Type{<:Tuple})
    # argument validation (no lock needed)
    lock(build_callable_lock)
    try
        # existing body unchanged
    finally
        unlock(build_callable_lock)
    end
end

Fix #226

Serialise build_callable under a ReentrantLock so that concurrent
threads cannot race during first-time callable IR generation and
cache population, which caused segfaults in threaded Particle Gibbs
sampling on Julia 1.10.
Copilot AI changed the title [WIP] Fix Julia 1.10 segfault in threaded PG due to concurrent Libtask callable generation Serialize build_callable under a global lock to prevent segfaults in threaded usage Jun 5, 2026
Copilot finished work on behalf of yebai June 5, 2026 07:14
Copilot AI requested a review from yebai June 5, 2026 07:14
@yebai yebai marked this pull request as ready for review June 5, 2026 07:14
@sunxd3 sunxd3 requested review from sunxd3 and removed request for yebai June 5, 2026 08:29
- Move `build_callable_lock` above the docstring so the docstring attaches
  to `build_callable` again (fixes Documenter `docs_block`/`missing_docs`)
- Wrap the `misty_closure` call to stay within blue style's margin at its
  new indentation (fixes JuliaFormatter check)
- Update the SMC test to FlexiChains' `(niters, nchains)` layout: the chain
  count is `size(chain, 2)`, not `size(chain, 3)`

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

Libtask.jl documentation for PR #227 is available at:
https://TuringLang.github.io/Libtask.jl/previews/PR227/

Comment thread src/copyable_task.jl
Comment thread test/integration/turing/main.jl
Comment thread src/copyable_task.jl Outdated
sunxd3 and others added 2 commits June 14, 2026 15:26
seed_id! mutates the shared global _id_count, the same structure the
ID() constructor mutates during IR derivation. Calling it from TapedTask
ran it outside build_callable_lock, so it could race with ID generation
on another thread and crash. Move it into the cache-miss branch of
build_callable, under the lock, alongside the ID() calls it pairs with.

Addresses review feedback from @sunxd3.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@sunxd3 sunxd3 merged commit d24443b into main Jun 15, 2026
22 checks passed
@sunxd3 sunxd3 deleted the copilot/fix-julia-1-10-segfault branch June 15, 2026 10:46
@yebai yebai mentioned this pull request Jun 26, 2026
20 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Julia 1.10 segfault in threaded PG due to concurrent Libtask callable generation

3 participants