Skip to content

feat: Preserve hover tooltips in live capture via event-tap passthrough#416

Open
duongductrong wants to merge 4 commits into
masterfrom
feature/live-capture-passthrough
Open

feat: Preserve hover tooltips in live capture via event-tap passthrough#416
duongductrong wants to merge 4 commits into
masterfrom
feature/live-capture-passthrough

Conversation

@duongductrong

Copy link
Copy Markdown
Owner

Problem

In live (non-frozen) capture mode, the full-screen AreaSelectionWindow panel intercepted every mouse event. Because the overlay was hit-testable and consumed all mouseMoved events, the app being captured never received hover tracking events, so tooltips, hover cards, and other hover-driven UI disappeared as soon as capture started.

Solution

Adopt a CleanShot-style passthrough capture architecture: consume mouse events at the WindowServer level with an active CGEventTap, keep the overlay click-transparent so apps beneath retain their hover state, and draw the crosshair ourselves instead of relying on NSCursor/cursor rects (which don't work for a background LSUIElement app).

Key changes

  • New CaptureEventTapController (Snapzy/Services/Capture/CaptureEventTapController.swift)

    • Active .cgSessionEventTap, head-insert, consumes buttons/drags/keys/mouseMoved.
    • Passes observed pointer positions to the selection engine for the crosshair/selection rect/window highlight.
    • Re-enables itself on timeout/user-input tap suspension.
    • Falls back to legacy panel-event input when Accessibility permission is missing.
  • LivePassthroughInputLogic helpers (Snapzy/Services/Capture/LivePassthroughInputLogic.swift)

    • Quartz→AppKit coordinate flip, hover coalescing (≤1 UI update per run-loop pass, <1px skip), dim visibility driver, and a per-display cursor-hide balance wrapper.
  • AreaSelectionWindow passthrough mode

    • Overlay panels are click-transparent (ignoresMouseEvents = true) during live capture so WindowServer never synthesizes tracking-exit events to apps beneath.
    • Dim layer is hidden until the first drag; drawn crosshair/camera proxy follows the pointer.
    • 60 Hz NSEvent.mouseLocation timer is disabled in passthrough mode.
    • Single teardown funnel guarantees tap stop + cursor state restoration on commit, cancel, Esc, right-click, session replacement, and dealloc.
  • Preferences

    • New screenshotLivePassthrough toggle (default on) in Screenshot settings.
    • One-time Accessibility prompt when the feature is enabled but not yet trusted; silent fallback to legacy behavior otherwise.
  • Instrumentation & docs

    • OSSignpost category CapturePassthrough for profiling.
    • Updated docs/CAPTURE.md to describe the new input architecture.
    • Localization strings added for all shipped locales.
  • Tests

    • CaptureEventTapControllerTests (16 tests): lifecycle, permission-denied path, consume/pass routing, tap-disabled re-enable.
    • LivePassthroughInputLogicTests (14 tests): coordinate flip, hover coalescing, dim driver, cursor-hide balance.

Known limitation

CGDisplayHideCursor requires the calling application to be in the foreground. Snapzy deliberately stays a background LSUIElement agent during capture (activating would itself dismiss the target app's hover UI), so the system cursor may remain visible alongside the drawn proxy crosshair. This matches CleanShot X's actual behavior — its binary imports no cursor-hiding API and lets the arrow show next to its drawn crosshair.

Two alternative public-API approaches were prototyped and rejected because they broke other requirements:

  • Transparent cursor rect via hittable overlay — correctly hides the arrow but makes WindowServer synthesize tracking-exit events, killing hover persistence.
  • Disassociate mouse+cursor and park the real cursor off-screen — hides the arrow but warps the cursor and adds state machine complexity.

Testing

  • xcodebuild build Debug → BUILD SUCCEEDED.
  • Capture test subset (CaptureEventTapControllerTests, LivePassthroughInputLogicTests, AreaSelectionOverlayTests, AreaSelectionOverlayDimmingTests, AreaSelectionSessionLifecycleTests, AreaSelectionOverlayCursorDragTests, LiveAreaMouseUpSnapshotTests, FrozenAreaCaptureSessionTests) → 0 failures.

Manual QA checklist

  • Hover a toolbar item in Safari/Finder so its tooltip is visible, then start live capture → tooltip stays visible while moving the mouse.
  • Clicking/hovering content beneath the overlay does nothing (interaction freeze).
  • Crosshair proxy tracks the pointer; window mode shows the camera proxy.
  • After commit, Esc, right-click, and session replacement, the cursor is restored.
  • Flag off or Accessibility denied → legacy behavior unchanged.
  • Frozen capture mode unchanged.

Snapzy's live capture overlay previously intercepted all mouse events,
causing hover UI (tooltips, hover cards) in the captured app to disappear.
Adopt CleanShot-style passthrough capture:

- Drive the capture UI from an active CGEventTap (.cgSessionEventTap),
  consuming every mouse/button/key event while observing pointer motion.
- Render the overlay click-transparent so apps beneath retain hover state.
- Draw the crosshair cursor via a proxy CALayer instead of relying on
  NSCursor/cursor rects, which don't work for a background LSUIElement.
- Add screenshot.livePassthrough preference (default on) with Accessibility
  gating and silent fallback to legacy behavior.
- Add hover coalescing and OSSignpost instrumentation for performance.
- Update docs/CAPTURE.md and localization strings.

Known limitation: CGDisplayHideCursor requires a foreground application,
so the system cursor may remain visible alongside the proxy crosshair during
passthrough capture. This matches CleanShot X's actual behavior.

Tests: CaptureEventTapControllerTests, LivePassthroughInputLogicTests, plus
existing AreaSelection* suites (146 passes).
@duongductrong duongductrong added the enhancement New feature or request label Jul 23, 2026
@duongductrong duongductrong changed the title feat: preserve hover tooltips in live capture via event-tap passthrough feat: Preserve hover tooltips in live capture via event-tap passthrough Jul 23, 2026
…sion

- Introduce BackgroundCursorControl to grant background agent cursor control via CGS connection property
- Improve system cursor hiding and restoration during live passthrough capture
- Add unit tests for BackgroundCursorControl and AreaSelectionOverlay cursor handling
- Add performance profiling scripts and update CAPTURE.md documentation
…e-passthrough

Resolve conflict in AreaSelectionWindow.swift cacheBackdropPixels: keep
master's freeze-screen backdrop-cache downscale optimization (#411); the
passthrough branch only had cosmetic self.-prefix changes to that function.

All live-capture passthrough code preserved (verified: diff vs branch tip =
exactly master's 3 hunks). Master's window-ordering and magnifier signature
changes integrate cleanly. Merged tree: TEST BUILD SUCCEEDED.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant