Skip to content

[2/5] Complete the native v2 action and protocol surface - #2

Draft
cm2435-hcomp wants to merge 1 commit into
agent/background-driver-v2-foundationfrom
agent/background-driver-v2-actions
Draft

[2/5] Complete the native v2 action and protocol surface#2
cm2435-hcomp wants to merge 1 commit into
agent/background-driver-v2-foundationfrom
agent/background-driver-v2-actions

Conversation

@cm2435-hcomp

Copy link
Copy Markdown
Owner

Why

This is the second PR in the background-driver v2 native stack. It completes the protocol and action surface on top of the typed controller foundation so the real-host workflows can exercise one coherent Rust implementation.

What changed

  • Wires the v2 JSON protocol through the server and CLI.
  • Adds the complete requested action vocabulary: semantic click, targeted point click, scroll, key/chord, text insertion, set value, text selection, drag, and secondary actions.
  • Adds typed route decisions, focused-element validation, modifier cleanup, post-action settlement, and partial-delivery evidence.
  • Adds the protocol fixture used by the Python client boundary tests.
  • Refuses background cells that have an implementation but no published real-host recipe.

Stack

Validation

Validated at the final stack head:

  • cargo test -p cua-driver-core: 148 tests passed, 0 failed.
  • cargo test -p platform-macos: 197 tests passed, 0 failed, 1 doc test ignored.
  • git diff --check: passed.

Repository-wide formatting remains red on unrelated 0.7.1 files and is not rewritten here.

Known limits

  • press_key is implemented but deliberately unpublished in every framework cell pending an exact keyboard recipe.
  • Drag, delta scroll, menu dismissal, Catalyst, and several framework cells remain unproved on the real host.
  • The macOS action implementation in this commit still reflects the earlier Chromium-first experiment. The next PR contains the Swift-reference correction and accepted Calculator evidence.

@cm2435-hcomp cm2435-hcomp changed the title [2/3] Complete the native v2 action and protocol surface [2/5] Complete the native v2 action and protocol surface Jul 23, 2026
@github-actions

Copy link
Copy Markdown

Linux visual regression artifacts

Matrix jobs now run independently. Download visual artifacts from this workflow run.
Each background-GUI job uploads a .gif of the interaction plus two annotated PNGs (<app>.png raw, <app>-atspi.png with AT-SPI element boxes); the cua-driver-linux-som-overlays artifact adds <app>-som.png cua Set-of-Marks overlays:

  • cua-driver-linux-cursor-click-gif
  • cua-driver-linux-background-terminal-gif
  • cua-driver-linux-parallel-drag-xserver
  • cua-driver-linux-background-gui-chromium
  • cua-driver-linux-background-gui-tk
  • cua-driver-linux-background-gui-gtk3-gedit
  • cua-driver-linux-background-gui-gtk3-mousepad
  • cua-driver-linux-background-gui-gtk3-scite
  • cua-driver-linux-background-gui-gtk4-characters
  • cua-driver-linux-background-gui-qt5-manuskript
  • cua-driver-linux-background-gui-qt5-klog
  • cua-driver-linux-background-gui-qt5-openambit
  • cua-driver-linux-background-gui-qt6-kate
  • cua-driver-linux-background-gui-qt6-kcalc
  • cua-driver-linux-background-gui-qt6-okular
  • cua-driver-linux-background-gui-qt6-qownnotes
  • cua-driver-linux-background-gui-electron-zettlr
  • cua-driver-linux-background-gui-electron-joplin
  • cua-driver-linux-background-gui-electron-logseq
  • cua-driver-linux-som-overlays

Open workflow run and download artifacts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant