zsh: sigsuspend autoconf probe fails in new darwin stdenv sandbox

[jwiegley]

Nixpkgs version

• Unstable (26.05)

Describe the bug

zsh hangs in __sigsuspend on darwin after PR #508474

Both of my macOS machines started hanging in interactive zsh sometime in the last few days. Fresh Terminal, type pwd, get the output, and the next prompt never draws. The shell still accepts keystrokes, but Ctrl-C does nothing and Enter does nothing. New ssh sessions hang the same way. Bash is fine. zsh -c 'echo OK' (no .zshrc) is fine. zsh -ic 'echo OK' reliably hangs.

After digging through it I'm fairly sure this is a regression from #508474 (darwin: migrate source releases from apple-sdk to darwin). The build still succeeds, but the resulting zsh binary embeds a race-prone signal-handling path that loses SIGCHLD wakeups.

Steps to reproduce

On a darwin host rebuilt against post-#508474 nixpkgs, with any non-empty interactive .zshrc:

zsh -c 'echo OK'     # works (skips .zshrc)
zsh -ic 'echo OK'    # hangs
bash -ic 'echo OK'   # works (control)

The hang triggers on the first $(...) reaping race during init or precmd. Common triggering call sites I've observed:

• eval "$(starship init zsh)" during init
• eval "$(direnv hook zsh)" in precmd
• $(hostname -f 2>/dev/null) inside iTerm2 shell-integration's per-prompt hook

Expected behaviour

I expected the shell to operate as normal, so that after executing pwd I receive a following shell prompt. Instead, the next shell prompt never comes and I cannot interact with my machine any further using zsh.

Screenshots

No response

Relevant log output

Additional context

What's actually wrong

zsh's configure.ac (around line 2334) does an AC_RUN_IFELSE test: fork, have the child exit, then call sigsuspend() with an empty mask and check whether the SIGCHLD handler observed the child. On the new stdenv that test program returns nonzero inside the build sandbox, so autoconf records zsh_cv_sys_sigsuspend=no and defines BROKEN_POSIX_SIGSUSPEND. Src/signals.c:signal_suspend() then compiles a sigprocmask(SIG_UNBLOCK,…) + pause() fallback instead of the atomic sigsuspend(&saved_mask). That fallback has a race window between unmask and pause; when a $(...) command-substitution child exits, the SIGCHLD fires its handler in that window and updates zsh's job-table bookkeeping, then pause() waits forever for another signal that never comes.

The nix log output for both builds is the cleanest evidence:

pre-#508474 build:  checking if POSIX sigsuspend() works... yes
post-#508474 build: checking if POSIX sigsuspend() works... no

And the symbol-import diff between the resulting binaries:

pre:  nm -u zsh | grep -E '_(sigsuspend|pause)$'  ->  _sigsuspend
post: nm -u zsh | grep -E '_(sigsuspend|pause)$'  ->  _pause

Same source, same upstream version (5.9), different code path compiled in.

Diagnostic checks

If you're staring at a wedged zsh and want to confirm:

1. sample <pid> shows signal_suspend → pause → __sigsuspend. The pause frame above __sigsuspend is the broken code path. (macOS implements pause(3) internally as sigsuspend(0), so the kernel-side symbol shows up either way; the userspace frames are what distinguish the two paths.)
2. The stuck zsh has no child in ps (running or zombie); the $(...) subprocess already exited.
3. lsof -p <pid> shows the command-sub pipe FD with no peer.
4. nm -u $(command -v zsh) | grep -E '_(sigsuspend|pause)$' prints _pause. A working zsh prints _sigsuspend. This is the most direct check and works without attaching to a running process.

Workaround

I'm running this overlay on both machines and it produces a working zsh:

zsh = prev.zsh.overrideAttrs (old: prev.lib.optionalAttrs prev.stdenv.isDarwin {
  preConfigure = (old.preConfigure or "") + ''
    export zsh_cv_sys_sigsuspend=yes
  '';
});

After rebuild, nix log says ... (cached) yes, the binary imports _sigsuspend instead of _pause, and zsh -ic 'echo OK' returns in under a second.

What I don't know

I haven't bisected why the autoconf test fails inside the new sandbox. The same runtime check works correctly on the user-side machine running the resulting binary -- _sigsuspend is in libSystem.B.dylib and behaves as POSIX requires. So the bug isn't in macOS or in zsh; it's a sandbox-vs-real-kernel discrepancy that causes autoconf to bake the wrong code path into the binary. Plausible candidates: sandbox profile rewriting SIGCHLD delivery between sandbox-internal processes, feature-test macros from the new SDK headers altering sigaction flag defaults, or fork-accounting differences in the new builder bash. I haven't proved any of these.

It's worth thinking about whether other AC_RUN_IFELSE probes in other darwin packages are quietly failing the same way and downgrading to inferior fallbacks. The zsh case is loud because the result is a hang.

Suggested fix

The cheapest unblock is to add the zsh_cv_sys_sigsuspend=yes override to nixpkgs' zsh package as a darwin-conditional preConfigure, same shape as the workaround above. That fixes every affected user immediately. The cost is skipping a runtime probe whose answer we already know on darwin.

The more durable fix is figuring out what changed about the sandbox-side signal-delivery in #508474 and making the runtime test return the right answer. That's preferable long-term, but the cache override is straightforward and I'm happy to send a PR if maintainers want it.

Environment

• Two aarch64-darwin hosts, macOS 26.4.1 (build 25E253), kernel Darwin 25.4.0 / xnu-12377.101.15~1
• nix-darwin + home-manager + flakes
• Boundary on each host: rebuild that swapped zsh from 2sbihdb…-zsh-5.9 to 80qadsn…-zsh-5.9 (same upstream version, different hash).
• Closure diff at the boundary: apple-sdk +354 MiB, plus zsh + pcre2 + ncurses + libiconv all rebuilt with same versions and new hashes -- the canonical fingerprint of an stdenv-level rebuild.

System metadata

• system: "aarch64-darwin"
• host os: Darwin 25.4.0, macOS 26.4.1
• multi-user?: yes
• sandbox: no
• version: nix-env (Determinate Nix 3.11.2) 2.31.1
• nixpkgs: /nix/store/i2gsp87gqp16whm9mw0ybk9n84zir01x-source

Are you using nix-darwin?

Yes, I am using nix-darwin.

Notify maintainers

cc @reckenrode (who landed #508474), @NixOS/darwin-maintainers

---

Note for maintainers: Please tag this issue in your pull request description. (i.e. Resolves #ISSUE.)

I assert that this issue is relevant for Nixpkgs

• I assert that this is a bug and not a support request.
• I assert that this is not a duplicate of an existing issue.
• I assert that I have read the NixOS Code of Conduct and agree to abide by it.

Is this issue important to you?

Add a 👍 reaction to issues you find important.

[emilazy]

After digging through it I'm fairly sure this is a regression from #508474 (darwin: migrate source releases from apple-sdk to darwin). The build still succeeds, but the resulting zsh binary embeds a race-prone signal-handling path that loses SIGCHLD wakeups.

Per the labels, that PR caused zero rebuilds. So it is essentially not possible that it caused any regression at all.

(I say “essentially” because it did actually manage to turn up a cursed Nix–Lix evaluation strictness discrepancy that broke the release checks due to unexpected evaluation failures, but that has nothing to do with actual builds using the resulting packages, which are identical before and after.)

[jwiegley]

I did notice that a HUGE number of packages rebuilt on my machine yesterday, but I don't know for certain (yet) that it was caused by that merge.

[jwiegley]

I can confirm on my laptop and desktop that with the workaround proposed, zsh is working for me again. Since I layer many other things on top (starship, iTerm2 support), I haven't ruled out yet some evil interaction between these bits. Bash was unaffected.

[emilazy]

I did notice that a HUGE number of packages rebuilt on my machine yesterday, but I don't know for certain (yet) that it was caused by that merge.

It’s zero rebuilds, so there’s effectively no chance of this. You would have to have some kind of very arcane custom overlay setup designed to be sensitive to arbitrary changes to Nixpkgs internals that don’t otherwise change a single package hash.

Any change you’re seeing is far more likely to be caused by something that landed as part of #507470. Probably #504506. (Sorry, I only skimmed after the first obvious error since the rest looked like LLM output predicated on a false assumption.)

[jwiegley]

Yes, it was indeed AI output predicated on a false assumption, I shared it because the fix actually did work after a few hours of tackling the problems from various angles. I'll head back to the drawing board to see if I can't narrow this down further.

[jwiegley]

@emilazy was right and I was wrong about the cause. Sorry for the noise. I should have checked the rebuild label on #508474 before publishing, and I should have done the controlled A/B before theorizing about sandboxes.

I went back and ran the experiment I should have run the first time. The actual culprit is #504506 (autoconf: 2.72 -> 2.73), which landed in unstable via the staging-next merge #507470 on April 22, right when the breakage showed up. The failure is entirely compile-time, not runtime. The AC_RUN_IFELSE test program never actually runs; my "sandbox signal delivery" theory was nonsense.

What's actually happening

Autoconf 2.73 added _AC_PROG_CC_STDC_EDITION (commit 653956f4, "AC_PROG_CC now tries C23 too"). It iterates [23, 11, 99, 89] and picks -std=gnu23 whenever the compiler accepts it. The NEWS entry for that release calls out exactly the relevant breakage:

*** AC_PROG_CC now prefers C23 if available.
C23 has removed old-style (K&R) function definitions/declarations.
Old-style functions should be updated to use prototypes.

zsh's nixpkgs derivation puts autoreconfHook in nativeBuildInputs, so configure is regenerated on every build with whatever autoconf is currently in the tree. Bumping autoconf rebuilds zsh even though autoconf isn't a runtime dep -- which is also the answer to "why did so much rebuild on my machines?" After the bump, zsh's regenerated configure compiles the zsh_cv_sys_sigsuspend test with -std=gnu23.

The test program (zsh-5.9 configure.ac:2336-2362) contains a K&R-style parameter declaration:

void handler(sig)
    int sig;
{if(sig==SIGCHLD) child=1;}

Apple Clang 21 hard-errors on that under -std=gnu23. Straight from config.log of a fresh autoreconf'd build:

configure:14178: gcc -std=gnu23 -o conftest [...]
conftest.c:267:14: error: unknown type name 'sig'
  267 | void handler(sig)
1 warning and 3 errors generated.
configure:14181: $? = 1
configure: program exited with status 1

AC_RUN_IFELSE treats compile failure as the action-if-false branch (lib/autoconf/general.m4:2995-3028), so zsh_cv_sys_sigsuspend=no gets cached and BROKEN_POSIX_SIGSUSPEND ends up defined. Src/signals.c:387-394 then compiles the racy sigprocmask + pause() fallback, and you get the hang.

The decisive A/B

Same source (zsh-5.9), same compiler, same nixpkgs patches (including ab4d62e), same environment. Only autoconf changes:

autoconf	sigsuspend test	BROKEN_POSIX_SIGSUSPEND
2.72 (pre-#504506)	yes	not defined
2.73 (current unstable)	no	defined

Single variable. Reproduction:

nix-prefetch-url --print-path mirror://sourceforge/zsh/zsh-5.9.tar.xz
mkdir -p /tmp/zsh && tar -C /tmp/zsh -xf <prefetched-path>
curl -sSL https://github.com/zsh-users/zsh/commit/ab4d62eb975a4c4c51dd35822665050e2ddc6918.patch \
  | (cd /tmp/zsh/zsh-5.9 && patch -p1)

# 2.72: sigsuspend test passes
nix shell "github:NixOS/nixpkgs/5b2c2d84341b2afb5647081c1386a80d7a8d8605#autoconf" \
          "github:NixOS/nixpkgs/5b2c2d84341b2afb5647081c1386a80d7a8d8605#m4" \
  -c bash -c 'cd /tmp/zsh/zsh-5.9 && autoreconf -fiv >/dev/null && \
              ./configure --disable-dynamic --without-tcsetpgrp 2>&1 | grep sigsuspend'

# 2.73: sigsuspend test fails to compile, cached as "no"
nix shell "github:NixOS/nixpkgs/nixpkgs-unstable#autoconf" \
          "github:NixOS/nixpkgs/nixpkgs-unstable#m4" \
  -c bash -c 'cd /tmp/zsh/zsh-5.9 && autoreconf -fiv >/dev/null && \
              ./configure --disable-dynamic --without-tcsetpgrp 2>&1 | grep sigsuspend'

Worth noting: nixpkgs already applies the upstream patch ab4d62e ("use 'int main()' in test C-codes in configure"), which fixed main() to int main() back in the clang-16 era. That patch left void handler(sig) int sig; untouched, presumably because clang 16's -Wimplicit-int only fired on main, not on K&R parameter declarations. C23 catches the second one too.

What to do about it

The workaround I posted earlier (preset zsh_cv_sys_sigsuspend=yes so the cache check short-circuits before the test ever compiles) still works -- it doesn't matter what -std= autoconf picks if the cache is already populated. After rebuild nm -u zsh shows _sigsuspend and BROKEN_POSIX_SIGSUSPEND is #undef'd in config.h. So the immediate unblock holds; only the explanation was wrong.

For a proper nixpkgs fix, three options in increasing order of niceness:

1. Add "zsh_cv_sys_sigsuspend=yes" to the unconditional configureFlags in pkgs/by-name/zs/zsh/package.nix. The file already does this for several zsh_cv_* vars in the cross-compile branch (lines 105-114). This just hoists the right one out.
2. postPatch to substitute void handler(sig)\n int sig; for void handler(int sig) in configure.ac, then let autoreconf regenerate.
3. Backport upstream commit 8dd271fd (2024-09-14, "53081: remove old BeOS support code"), which removed the entire BROKEN_POSIX_SIGSUSPEND test in zsh master. Cleanest answer but won't ship until 5.10.

The same C23 question presumably applies to other AC_RUN_IFELSE tests in zsh and elsewhere that still use K&R declarations. A wider survey is probably worth doing.

Happy to send a PR for option 1 or 2 if maintainers prefer one over the other.

[4evy]

I've been having same issue where zsh just gets stuck forever but haven't looked into it yet, glad to see I'm not the only one

[winterqt]

@jwiegley Please don’t post AI output without even as much as a disclaimer… it’s a waste of time for everyone involved.

If you’d like to use AI to help debug, that’s totally fine! At the very least, though, write up things in your own words, or make it very obvious that a comment is AI output.

[emilazy]

Backporting zsh-users/zsh@8dd271f seems fine. Can includes = [ "configure.ac" ]; if the rest doesn’t apply cleanly.

[jwiegley]

@jwiegley Please don’t post AI output without even as much as a disclaimer… it’s a waste of time for everyone involved.

If you’d like to use AI to help debug, that’s totally fine! At the very least, though, write up things in your own words, or make it very obvious that a comment is AI output.

I will do so, @winterqt, thank you for your directness.

John

[Prince213]

Backporting zsh-users/zsh@8dd271f seems fine. Can includes = [ "configure.ac" ]; if the rest doesn’t apply cleanly.

includes = [ "configure.ac" ]; still won't work and I think we can just set zsh_cv_sys_sigsuspend=true (if that works) instead of vendoring a patch.