Skip to content

fix(macos): leave host-owned NSApplication alone (#5)#6

Merged
Kitty-Hivens merged 4 commits into
mainfrom
fix/macos-host-nsapp-guard
May 29, 2026
Merged

fix(macos): leave host-owned NSApplication alone (#5)#6
Kitty-Hivens merged 4 commits into
mainfrom
fix/macos-host-nsapp-guard

Conversation

@Kitty-Hivens

Copy link
Copy Markdown
Owner

Summary

When a host UI toolkit (JavaFX/Glass, Compose/Skiko, AWT) already owns
NSApplication and is running its event loop, the macOS backend was
re-running [NSApp finishLaunching] and flipping the activation policy to
Accessory on tray creation. On JavaFX/Intel this destabilised Glass's
CVDisplayLink pulse timer and crashed the JVM with a SIGSEGV in
objc_msgSend right after startup.

  • Gate the NSApp bootstrap on [NSApp isRunning]: a host-owned app gets
    just the status item; libtray still bootstraps NSApp when it owns it
    (headless / smoke).
  • Add a JavaFX repro harness (runJavaFxSmoke, isolated source set) that
    recreates the crash from the FX Application Thread; pins
    prism.order=sw so JavaFX starts on headless / VM Macs.

Reproduced and verified on an Intel macOS VM: unpatched build SIGSEGVs,
patched build brings the tray up with no crash.

Closes #5

Test plan

  • ./gradlew test green
  • runJavaFxSmoke: unpatched crashes in objc_msgSend, patched does not (Intel macOS VM A/B)
  • runSmoke still works (libtray owns NSApp, isRunning false at create)

When a host UI toolkit (JavaFX/Glass, Compose/Skiko, AWT) already owns
NSApplication and is running its event loop, libtray was re-running
[NSApp finishLaunching] and flipping the activation policy to Accessory.
On JavaFX/Intel that destabilised Glass's CVDisplayLink pulse timer and
crashed the JVM with a SIGSEGV in objc_msgSend right after startup.

Gate the bootstrap on [NSApp isRunning]: only set the activation policy
and finishLaunching when libtray itself owns NSApp (headless/smoke,
where isRunning is false at create time). Host-owned apps get just the
status item, since the host already did the launch bootstrap.
Adds an isolated javafxSmoke source set + runJavaFxSmoke task that brings
up a real JavaFX toolkit and then creates a libtray tray from the FX
Application Thread -- the exact state ([NSApp isRunning] == true) where
the upstream consumer crashed. runSmoke could not reproduce it because it
owns the NSApp bootstrap itself.

JavaFX is verification-scoped only: separate source set, never in the
published artifact or the main/test classpath. The task does not force
-XstartOnFirstThread (JavaFX manages its own main thread); pass
-PfxFirstThread to override if a host needs it.
…Macs

JavaFX 25 defaults to Metal on macOS; a QEMU macOS VM has no Metal GPU
and dies in MTLContext_nInitialize before reaching the tray. Pin
prism.order=sw for runJavaFxSmoke. The #5 crash is in Glass's
CVDisplayLink pulse timer, independent of the render backend, so the
repro stays valid under software rendering.
@Kitty-Hivens Kitty-Hivens merged commit f8f2543 into main May 29, 2026
1 check passed
@Kitty-Hivens Kitty-Hivens deleted the fix/macos-host-nsapp-guard branch May 29, 2026 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error reported when running on MacOS 11

1 participant