cua-driver test suite — how the tests look

The integration tests in libs/cua-driver/rust/crates/cua-driver/tests/ all sit on one shared foundation — the cua-driver-testkit crate — and are named by a four-family taxonomy so ls tests/ reads as a map. Companion: TEST_HARNESS_STRUCTURE.md (the harness apps the harness_* tests drive).

The shared foundation: cua-driver-testkit

A dev-only crate (crates/cua-driver-testkit) that every test builds on, so no test re-implements process plumbing:

• Two transports, one shape. McpDriver (one long-lived cua-driver server over stdio JSON-RPC, session-scoped state) and CliDriver (stateless cua-driver call per action) both implement the Driver trait and normalize their differing payloads — the CLI prints structuredContent directly, MCP returns the JSON-RPC envelope — into one ToolResponse (text() / structured() / is_error()). RawDriver adds raw send/recv with no auto-initialize for the protocol tests that drive the handshake themselves.
• ChildReaper kills every spawned child on drop; on Windows it assigns them to a kill-on-close Job Object (no orphaned windows / held ports).
• paths (driver_binary() resolves release-or-debug + .exe), ax (snapshot-text parsers), McpDriver::find_window.

The loop every behavior test runs

flowchart LR
    T["test (tests/*.rs)"]
    TK(["cua-driver-testkit
McpDriver / CliDriver / RawDriver"])
    D(["cua-driver"])
    A["target app / harness app"]
    T -->|"Driver::call(tool, args)"| TK
    TK -->|"spawn + drive"| D
    D -->|"acts on"| A
    T -->|"re-snapshot & assert (ToolResponse)"| T

Inventory by family (18 files)

The protocol_* family is the renamed + split successor of the old mcp_protocol_test (3,412 lines / 65 tests). The split deduped 24 macOS↔Windows mirror pairs into single cfg!-branching tests, sharing one RawDriver.

Coverage on three axes: transport × modality × platform

                    ┌────────────── MODALITY ──────────────┐
                    │ background    foreground/desktop-scope│
   ─────────────────┼───────────────────────────────────────
   MCP transport    │ ✅ thorough    ✅ desktop_scope (win)   │
                    │ (background,   focus (mac)             │
                    │  input_e2e,                            │
                    │  focus, guard)                         │
   CLI transport    │ ⬚              ✅ config-persistence    │
                    │                (transport_ test)       │
                    └───────────────────────────────────────┘
   Per-app AX (MCP): harness_{wpf,winui3,web,libreoffice}=win · {appkit,swiftui}=mac
   Protocol (RawDriver): mac+win, deduped; element_token = mac/linux

• Background modality is the most-covered: the sentinel oracle (modality_background), unified input (modality_input_e2e), the macOS no-focus-steal check (modality_focus), and the UX guards.
• Foreground / desktop-scope is covered by modality_desktop_scope.
• Transport is now a first-class axis: transport_config_persistence exercises CLI (disk) vs MCP (session) directly; most other tests run over MCP.

The modality matrix: action target × dispatch × capture_scope

The user-facing matrix is documented in docs/content/docs/explanation/capture-and-dispatch-modalities.mdx. get_window_state always returns both the accessibility tree and a screenshot; AX vs pixel modality is chosen at action time by using element_index vs x,y. Coverage of its valid cells, per platform:

capture_mode is deprecated and ignored: get_window_state returns both the accessibility tree and a screenshot regardless of the parameter. The modality_capture_mode_test now exercises compatibility only: deprecated ax/vision/som values are accepted but do not change the returned shape.

• Remaining gaps (honest): window/x,y/foreground has no explicit test on any platform; the desktop-scope window-less actuator is Windows-only (macOS click still requires pid), so the desktop cell and its negative gate are Windows-only until that loop lands on macOS/Linux; CLI transport is otherwise lightly exercised.

Where they run

Run an #[ignore] suite explicitly: cargo test -p cua-driver --test <name> -- --ignored --nocapture --test-threads=1