Skip to content

Add follow-the-cursor multi-monitor cursor support#40

Open
n8bar wants to merge 1 commit into
AlynxZhou:masterfrom
n8bar:multi-monitor-cursor
Open

Add follow-the-cursor multi-monitor cursor support#40
n8bar wants to merge 1 commit into
AlynxZhou:masterfrom
n8bar:multi-monitor-cursor

Conversation

@n8bar

@n8bar n8bar commented Jun 24, 2026

Copy link
Copy Markdown

What this does

ReFrame runs one server per monitor, but on Wayland compositors (e.g. COSMIC/cosmic-comp) an absolute pointer device is confined to a single output, so the system cursor can't move between monitors. This adds "follow the cursor":

  • each streamer publishes which monitor currently holds the cursor (detected from the DRM cursor plane each frame) to a flock-guarded file under /run/reframe;
  • on a pointer event for a monitor that doesn't currently hold the cursor, the streamer relocates it with one short burst of relative motion on a dedicated relative-only uinput device (a single large REL_X is ignored/capped by libinput; many small steps a few ms apart are honored; related to Can't control mouse #36), then forwards the absolute event for precise placement;
  • within a monitor it's pure absolute input, so normal movement stays smooth and pixel-accurate.

New per-monitor config key position-x (the monitor's real desktop X position), kept separate from monitor-x so absolute scaling stays full-range per monitor. Documented in dists/example.conf. The existing device stays absolute-only; the relative-only device is added specifically to avoid the absolute+relative same-device problem from #36.

Tested

Pop!_OS 24.04, COSMIC (cosmic-comp), Wayland, NVIDIA open kernel module 580.159.03, two monitors (1920×1080 + 1680×1050), VNC via libvncserver, client = Apache Guacamole. Verified crossing and precise in-monitor positioning in both directions and from either monitor's stream.

Not tested / out of scope

  • Touchscreen devices on either the local or remote side; we weren't equipped to test touch input, and since this area touches input-device classification (cf. Can't control mouse #36), touchscreen behavior is unverified.
  • Cross-monitor drag-and-drop is not handled (relocation teleports the cursor; a drag won't carry across).
  • Only exercised on cosmic-comp; other Wayland compositors and X11 untested.

Happy to adjust the approach or naming to fit the project's conventions.

On Wayland compositors (e.g. COSMIC/cosmic-comp) an absolute pointer device
is confined to a single output, so reframe's per-monitor servers could not
move the system cursor between monitors. This adds a "follow the stream"
behaviour:

- The streamer publishes which monitor currently shows the cursor (detected
  from the DRM cursor plane each frame) to a shared, flock-guarded file
  (/run/reframe/cursor-owner).
- On a pointer event for a monitor that does NOT currently hold the cursor,
  the streamer emits one short relative burst on a dedicated relative-only
  uinput device ("reframe-rel") to relocate the cursor onto this monitor
  (a single large REL_X is ignored/capped by libinput; many small steps a
  few ms apart are honoured — see AlynxZhou#36), then forwards the absolute event
  unchanged for precise in-monitor positioning.

Within a monitor there is no relative motion at all (pure absolute), so
normal movement stays smooth and pixel-accurate.

New config key: position-x (the monitor's real desktop X position), kept
separate from monitor-x so absolute scaling stays full-range per monitor.

Known limitation: cross-monitor drag-and-drop is not handled.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@n8bar

n8bar commented Jun 24, 2026

Copy link
Copy Markdown
Author

Hi @AlynxZhou, a transparency note up front.

This PR was implemented with AI assistance, and I want to be straight about that given your HACKING.md policy: "Do NOT submit AI/LLM generated code, unless you really understand what they are doing!" and your preference for PRs from a responsible human.

I believe I land on the "unless" side of that line, and I'll own it:

  • The approach is mine. Rather than fight the compositor confining an absolute pointer to a single output, read which monitor actually holds the cursor (from the DRM cursor plane) and relocate it only when the monitor being driven isn't the one already holding it.
  • I reviewed every line of this diff and understand it; happy to talk through any part.
  • I tested it on my own hardware (config is in the PR description) and worked through the failure modes myself.

I'm a responsible human, and I'll engage with your review personally and in my own words. If you'd do any of it differently, say so and I'll revise. Thanks for reframe; it's the foundation all of this builds on.

@AlynxZhou

Copy link
Copy Markdown
Owner

I don't think you really understand why we have to mapping pointer position.

I don't use COSMIC, and AFAIK a mouse with more than 1 monitors for most desktops is:

  1. Mouse declare a X,Y range for your max values.
  2. Desktops maps this range to its virtual desktop which should contain all monitors.
  3. Because we can only get a X,Y position relative to a monitor, we need to know the monitor's position, and virtual desktop size, so we can convert this relative position back into the mouse X,Y range, this is what we should report to uinput. And then desktop could map it in reverse.

So before reviewing the code, you need to tell me what you are going to solve:

Do you mean COSMIC behave differently from other desktops? If they are the only one doing that, you should ask them first for following other desktop's behavior.

I checked your code, while I cannot say the logic is good or bad, I think you should not change the meaning of current config keys, desktop-width is virtual desktop width, no matter how you are going to map the pointer position, you should not use it for "THIS monitor's width", you already call it "monitor's width", why you don't add a monitor-width key? And I don't understand what's the difference between position-x and monitor-x, I read "this monitor's real desktop X-position (top-left corner) in desktop pixels" three times and I still think it is monitor-x. If you don't want to confuse user, you should not confuse me first.

@AlynxZhou

Copy link
Copy Markdown
Owner

I've tested KDE/GNOME. They follow the behavior I said, don't know how COSMIC behaves.

Comment thread reframe-streamer/main.c
// Publish ourselves as the owner. When the cursor is on
// the other monitor, our CRTC has no cursor plane, so we
// don't publish and the other streamer claims it.
cursor_owner_publish(this);

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obviously this is not enough, not all implementations have/use cursor plane.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants