Summary
When a swiftly-managed toolchain is used for a highly parallel build — many concurrent invocations of clang/clang++/swiftc through the shims in ~/.local/share/swiftly/bin/ — the build fails with a thread-creation / "too many threads" error.
Each invocation of a swiftly shim starts the swiftly dispatcher which launches at least 17 threads before it execs the real tool. With dozens of compiler processes launching at once, the combined live thread count exceeds the process/thread limit and the build dies.
Background
stuartcarnie/godot-build, a set of containers that cross-compile Godot for Apple platforms from Linux using a swiftly-installed Swift/clang toolchain.
Godot builds with scons, which invokes the compiler once per translation unit and, under -j<N>, runs up to N of those compiler processes concurrently. So a normal build is dozens to hundreds of short-lived clang++/swiftc invocations, N of them alive at any instant.
The shims that swiftly installs under ~/.local/share/swiftly/bin/ are symlinks to the swiftly dispatcher. Every time scons runs clang++, the dispatcher launches, creating a large number of threads (at least 17), and only then execs the actual compiler. The live thread count is therefore roughly ~17 × (concurrent compiler invocations).
The scons build runs inside a podman container, whose default pids_limit is 2048. That cap is enforced by the pids cgroup controller, which counts tasks (threads), not just processes — so every dispatcher thread counts against it.
We were able to work around the issue by updating PATH at the real toolchain binaries (~/.local/share/swiftly/toolchains/<version>/usr/bin) ahead of the dispatcher directory makes the tools exec directly, bypassing the dispatcher entirely, and the parallel build runs cleanly.
Summary
When a swiftly-managed toolchain is used for a highly parallel build — many concurrent invocations of
clang/clang++/swiftcthrough the shims in~/.local/share/swiftly/bin/— the build fails with a thread-creation / "too many threads" error.Each invocation of a swiftly shim starts the
swiftlydispatcher which launches at least 17 threads before itexecs the real tool. With dozens of compiler processes launching at once, the combined live thread count exceeds the process/thread limit and the build dies.Background
stuartcarnie/godot-build, a set of containers that cross-compile Godot for Apple platforms from Linux using a swiftly-installed Swift/clang toolchain.
Godot builds with scons, which invokes the compiler once per translation unit and, under
-j<N>, runs up toNof those compiler processes concurrently. So a normal build is dozens to hundreds of short-livedclang++/swiftcinvocations,Nof them alive at any instant.The shims that swiftly installs under
~/.local/share/swiftly/bin/are symlinks to theswiftlydispatcher. Every time scons runsclang++, the dispatcher launches, creating a large number of threads (at least 17), and only thenexecs the actual compiler. The live thread count is therefore roughly~17 × (concurrent compiler invocations).The scons build runs inside a podman container, whose default
pids_limitis 2048. That cap is enforced by thepidscgroup controller, which counts tasks (threads), not just processes — so every dispatcher thread counts against it.We were able to work around the issue by updating
PATHat the real toolchain binaries (~/.local/share/swiftly/toolchains/<version>/usr/bin) ahead of the dispatcher directory makes the toolsexecdirectly, bypassing the dispatcher entirely, and the parallel build runs cleanly.