Force xcb platform on Wayland to fix tab drag indicators

[aniro]

Disclosure: This pull request was authored by an LLM (Claude Opus 4.7) under my direction. I have built it locally, reproduced the bug before the change, verified the fix on my Wayland session, and confirmed both escape hatches behave as documented. Maintainer review is welcome and expected; please push back on anything that doesn't match the project's conventions.

Summary

Qt Advanced Docking System positions its drop-indicator overlays (the cross of split-direction buttons and the highlighted preview rectangles) as top-level Qt::Tool windows using mapToGlobal() + move() to absolute global coordinates. Wayland does not let clients position top-level windows, so during a tab drag the indicators appear at wrong locations across the screen, making the dock UX effectively unusable on Wayland.

ADS upstream has documented this limitation and officially recommends running through XWayland. The newer ADS releases (4.5.0) still don't fix it.

This PR sets QT_QPA_PLATFORM=xcb;wayland on Linux Wayland sessions in main.cpp before constructing QApplication, so the app runs through XWayland where the positioning works correctly. The semicolon-separated form is Qt 6 syntax for a fallback list: Qt tries xcb first and falls back to wayland on systems where the xcb plugin is unavailable, so it stays safe on minimal containers.

The override only kicks in when:

• QT_QPA_PLATFORM is not already set by the user (their explicit choice always wins), and
• The opt-out NOTEPADNEXT_FORCE_NATIVE_WAYLAND is not set.

The Wayland platform plugin is still bundled with the AppImage (added in #1022), so users who set NOTEPADNEXT_FORCE_NATIVE_WAYLAND=1 still get a working app, just with the original misaligned indicators during tab drags.

Test plan

• Reproduced the misalignment on Wayland (GNOME) before the change: drop indicators rendered far outside the dock area during tab drag.
• Built and ran with the change on the same Wayland session: drop indicators now align correctly with the dock area, and split-on-drop works as on X11.
• Verified NOTEPADNEXT_FORCE_NATIVE_WAYLAND=1 ./NotepadNext reproduces the original misalignment, confirming the escape hatch works.
• Verified QT_QPA_PLATFORM=wayland ./NotepadNext honors the explicit user setting.
• X11 sessions: no behavioral change expected (the block is gated on XDG_SESSION_TYPE=wayland / WAYLAND_DISPLAY).
• Non-Linux platforms: no behavioral change (entire block is #ifdef Q_OS_LINUX).

[aniro]

before:

after:

[dail8859]

So if I understand this correctly:

• Wayland support was previously added in [feat] [Feature] Fix wayland not being there for the appimage + Add linux build doc #1022
• Wayland doesn't work for ADS
• This PR defaults to not using Wayland
• And allows the user to opt into Wayland if they really want it

CC: @Twig6943

[Twig6943]

@aniro howd u get dark theme?.

[aniro]

@Twig6943 there is another PR #1040

[dail8859]

@aniro @Twig6943 My root question is...so what is the point of adding wayland support in #1022 if it is intentionally disabled due to these issues?

Would it just be best to just revert #1022?

[Twig6943]

I didnt know of such issues it seemed to work good enough on my sysyem

[aniro]

It is just something I stumbled upon and got an LLM to fix. Please treat it as a bug report with some details on the particular code that breaks.
Wayland support is probably nice going forward, but I lack context to suggest the best approach here.

[dail8859]

Is the Wayland issue purely visual? Or does it affect the user's ability to drag and drop?

[Twig6943]

Is the Wayland issue purely visual? Or does it affect the user's ability to drag and drop?

Will let you know when I get home

[aniro]

It is not only visual. Snapping to the edges works (if you know where to point). Placing an editor tab inside another tab by spitting it - does not (using "crosshair" elements in the middle).
I can make a screencast if that helps

[aniro]

Also, if you resize the window, things get completely broken. Here is the screen recording:
https://github.com/user-attachments/assets/b595bd53-82af-4f60-90a9-6644bad67c74

[Twig6943]

Video_2026-05-09_02-44-33.mp4

[Twig6943]

Video_2026-05-09_02-48-01.mp4

[dail8859]

Supporting Linux is becoming quite tedious. Multiple distribution mechanisms, windows managers, OS versions and any combination of these. Even when trying to isolate it to a common windows manager, things are not trivial.

Ultimately as suggested in the ADS repository this is a Qt issue. For now, if someone uses Wayland, let them deal with Qt's support. If they do not like how it behaves, they can manually set QT_QPA_PLATFORM to fall back to something that works better for them.

I do appreciate the input and discussion.