Back to Qwen Code

Cua Driver action and icon catalog

packages/cua-driver/docs/action-icon-catalog.md

0.21.624.1 KB
Original Source

Cua Driver action and icon catalog

Status: Semantic source catalog for cursor theme profile v2

Source: Cua Driver public contract and platform tool registries

Scope: Public MCP/CLI-callable tools and their meaningful runtime action variants. Internal implementation helpers are excluded.

What is being cataloged

A public tool name is not always a complete action identity.

For example, click may resolve to:

  • an accessibility action or a pixel action;
  • background or foreground delivery;
  • a window target or the full desktop;
  • left, right, or middle button;
  • one, two, or three clicks.

The icon system should therefore resolve a call into:

text
surface.intent.target.delivery.qualifiers

Examples:

text
native.click.px.background.left.single
native.click.px.foreground.left.single
native.click.ax.background.press
native.scroll.desktop.down.page
browser.click.trusted.ref
browser.pointer.drag.dom_event

Foreground and background are separate resolved actions, but they should not need unrelated illustrations. A maintainable asset system uses:

  1. a base intent glyph or animation (click, type, scroll, drag);
  2. a targeting treatment (ax, px, desktop, page, browser);
  3. a host-rendered delivery chip (background, foreground);
  4. a small qualifier where it changes the visible gesture (right, double, down, page, and so on).

Canonical variant vocabulary

Targeting

TokenMeaningHow it is selected
axAccessibility-addressed element actionelement_token or element_index
pxWindow-local pixel actionWindow target plus x/y coordinates
focusedActs on the target's currently focused controlNo element or point supplied
desktopScreen-absolute actionscope:"desktop" without a window target
pageLegacy page/DOM actionA page[action=…] operation
browser_trustedTyped browser action through trusted CDP Input eventsinput_route:"trusted" or omitted
browser_domExplicit synthetic DOM eventinput_route:"dom_event"

Delivery

TokenMeaning
backgroundTargeted delivery without raising or fronting the target
foregroundBriefly front target, act, then restore the previous foreground app
persistent_foregroundbring_to_front; target stays foreground
fixed_backgroundBrowser or Linux low-level operation whose contract never fronts
noneRead-only or lifecycle operation with no input-delivery posture

The desktop input family exposing delivery_mode is:

text
click
double_click
right_click
drag
type_text
press_key
hotkey
scroll

browser_dialog also exposes background/foreground delivery for accept and dismiss. Typed browser pointer/click operations never foreground the browser.

Gesture qualifiers

DimensionValues
Buttonleft, right, middle
Click countsingle, double, triple
AX click actionpress, show_menu, pick, confirm, cancel, open
Scroll directionup, down, left, right
Scroll unitline, page
Browser input routetrusted, dom_event
Browser typinginsert_text, keystrokes

Modifiers such as Command, Control, Alt/Option, and Shift belong in the animation payload or label. They do not require separate base icons.

Resolved native input actions

These are the action identities that should drive cursor animations. Brace-delimited values are finite variants, not literal asset names.

Public toolResolved action keys
clicknative.click.ax.{background|foreground}.{press|show_menu|pick|confirm|cancel|open}
clicknative.click.px.{background|foreground}.{left|right|middle}.{single|double|triple}
clicknative.click.desktop.{left|right|middle}.{single|double|triple}
double_clicknative.double_click.{ax|px}.{background|foreground}
right_clicknative.right_click.{ax|px}.{background|foreground}
dragnative.drag.px.{background|foreground}.{left|right|middle}
dragnative.drag.desktop.{left|right|middle}
type_textnative.type_text.{ax|px|focused}.{background|foreground}
type_textnative.type_text.desktop
press_keynative.press_key.{ax|px|focused}.{background|foreground}
press_keynative.press_key.desktop
hotkeynative.hotkey.{px|focused}.{background|foreground}
hotkeynative.hotkey.desktop
set_valuenative.set_value.ax.background
scrollnative.scroll.{ax|px}.{background|foreground}.{up|down|left|right}.{line|page}
scrollnative.scroll.desktop.{up|down|left|right}.{line|page}
move_cursoragent_cursor.move.window, native.pointer.move.desktop
zoomnative.zoom.window
bring_to_frontnative.window.persistent_foreground

Do not generate every Cartesian product as a fully independent illustration. For example, all native.scroll.* actions share one scroll animation. Delivery and targeting are composable host-rendered badge chips rather than theme animations.

Page compound actions

The public page tool contains seven operations:

Resolved action keyEffectPlatform note
page.get_textRead visible page textCross-platform
page.query_domQuery elements with a CSS selectorCross-platform
page.execute_javascriptExecute JavaScriptCross-platform where a supported page backend is available
page.click_elementClick a CSS-selected elementCross-platform
page.insert_textInsert text at current DOM focusmacOS implementation currently documented
page.type_keystrokesDispatch durable per-character key eventsmacOS implementation currently documented
page.enable_javascript_apple_eventsEnable Safari JavaScript automationmacOS only

Typed browser actions

Public toolResolved action keys
get_browser_statebrowser.bind, browser.snapshot.dom_refs_v1, browser.snapshot.semantic_v2
browser_preparebrowser.prepare.detect, browser.prepare.isolated_new, browser.prepare.isolated_named, browser.prepare.existing_profile
browser_navigatebrowser.navigate
browser_clickbrowser.click.trusted.{ref|viewport}, browser.click.dom_event.ref
browser_typebrowser.type.insert_text, browser.type.keystrokes
browser_dialogbrowser.dialog.inspect
browser_dialogbrowser.dialog.accept.{background|foreground}
browser_dialogbrowser.dialog.dismiss.{background|foreground}
browser_set_input_filesbrowser.files.set_input
browser_downloadbrowser.download
browser_pointerbrowser.pointer.{hover|right_click|double_click|scroll|drag}.{trusted|dom_event}

Typed browser clicks and pointer operations have fixed background posture. The trusted and dom_event routes are materially different and should remain distinct resolved action IDs.

Session, recording, cursor, and maintenance variants

Public toolResolved action keys
start_sessionsession.start.{auto|window|desktop}
escalate_sessionsession.escalate.desktop
get_session_statesession.inspect
end_sessionsession.end
start_recordingrecording.start.trajectory, recording.start.video
stop_recordingrecording.stop
get_recording_staterecording.inspect
replay_trajectoryrecording.replay
set_agent_cursor_enabledagent_cursor.show, agent_cursor.hide
set_agent_cursor_motionagent_cursor.configure_motion
set_agent_cursor_themeagent_cursor.set_theme
get_agent_cursor_stateagent_cursor.inspect
check_permissionspermissions.inspect, permissions.request
health_reportsystem.health.inspect
get_configconfig.inspect
set_configconfig.update
check_for_updateupdate.inspect
install_ffmpegdependency.ffmpeg.plan, dependency.ffmpeg.install

Complete public tool-name union

The source-built macOS registry contains 49 tools. Windows adds debug_window_info. Linux adds four low-level pointer tools. The union is 54 public tool names.

#ToolCategoryPlatformPrimary icon intent
1list_appsInspectionAllApps/list
2list_windowsInspectionAllWindows/list
3get_window_stateInspectionAllWindow inspect; optional capture treatment
4launch_appApp lifecycleAllLaunch
5kill_appApp lifecycleAllForce stop
6bring_to_frontWindow lifecycleAllPersistent foreground
7clickPointer inputAllClick plus target/delivery/button/count
8double_clickPointer inputAllDouble click plus target/delivery
9right_clickPointer inputAllContext click plus target/delivery
10dragPointer inputAllDrag plus delivery
11type_textKeyboard inputAllType plus target/delivery
12press_keyKeyboard inputAllSingle key plus target/delivery
13hotkeyKeyboard inputAllKey chord plus delivery
14set_valueAccessibility inputAllSet semantic value
15scrollPointer/accessibility inputAllScroll plus target/delivery/direction
16get_screen_sizeInspectionAllScreen dimensions
17get_desktop_stateInspectionAllDesktop capture
18get_cursor_positionInspectionAllCursor location
19move_cursorPointer/overlay inputAllAgent-cursor move in window scope; real-pointer move in desktop scope
20set_agent_cursor_enabledAgent cursorAllShow/hide cursor
21set_agent_cursor_motionAgent cursorAllMotion configuration
22set_agent_cursor_themeAgent cursorAllSelect an installed visual theme
23get_agent_cursor_stateAgent cursorAllCursor inspect
24check_permissionsPermissionsAll; prompting is macOS-specificPermission inspect/request
25health_reportMaintenanceAllHealth check
26get_configConfigurationAllConfig inspect
27set_configConfigurationAllConfig update
28get_accessibility_treeInspectionAllDesktop accessibility inspect
29zoomInspectionAllZoom/crop
30pagePage/DOMAll, with operation-specific limitsUse the seven page operation icons
31get_browser_stateTyped browserAllBind or snapshot
32browser_prepareTyped browserAllBrowser preparation/profile
33browser_navigateTyped browserAllNavigate
34browser_clickTyped browserAllBrowser click plus route
35browser_typeTyped browserAllBrowser type plus mode
36browser_dialogTyped browserAllInspect/accept/dismiss dialog
37browser_set_input_filesTyped browserAllUpload/attach files
38browser_downloadTyped browserAllDownload
39browser_pointerTyped browserAllUse the ten browser-pointer operation icons
40start_recordingRecordingAllStart trajectory/video recording
41stop_recordingRecordingAllStop recording
42get_recording_stateRecordingAllRecording inspect
43replay_trajectoryRecordingAllReplay
44install_ffmpegMaintenanceWindows/Linux; no-op when unnecessaryPlan/install dependency
45start_sessionSession lifecycleAllStart session plus capture scope
46escalate_sessionSession lifecycleAllEscalate to desktop
47get_session_stateSession lifecycleAllSession inspect
48end_sessionSession lifecycleAllEnd session
49check_for_updateMaintenanceAllUpdate check
50debug_window_infoDiagnosticWindows onlyWindow diagnostic
51mouse_button_downLow-level pointerLinux onlyHold button
52mouse_dragLow-level pointerLinux onlyMove held pointer
53mouse_button_upLow-level pointerLinux onlyRelease button
54parallel_mouse_dragMulti-pointerLinux onlyConcurrent drags

Linux low-level pointer action keys

These tools are intentionally separate because a held button survives across calls and multiple session cursors can drag concurrently:

text
linux.mouse_button_down.{left|right|middle}.fixed_background
linux.mouse_drag.fixed_background
linux.mouse_button_up.fixed_background
linux.parallel_mouse_drag.fixed_background

Aliases and removed surfaces

  • type_text_chars is a deprecated invoke-time alias for type_text. It is deliberately hidden from tools/list and should reuse the type_text icon.
  • The old screenshot tool is removed. Window capture is represented by get_window_state; full-display capture is get_desktop_state.
  • Deprecated get_window_state.capture_mode values are accepted but ignored. They do not represent distinct actions and should not receive icons.
  • Internal platform delivery paths such as cgevent, x11_atspi, msaa, and key_events_fg are result metadata, not public requested actions. They may be shown in diagnostics, but should not expand the primary icon catalog.

Generate these as reusable base glyphs/animations first:

text
inspect
capture
launch
kill
foreground
click
double_click
right_click
drag
type_text
press_key
hotkey
set_value
scroll
move_cursor
zoom
navigate
dialog
upload
download
record
replay
session
permissions
settings
health
update

Then generate small composable treatments for:

text
ax
px
desktop
page
browser_trusted
browser_dom
background
foreground
left
right
middle
double
triple
up
down
horizontal
vertical

This produces distinct foreground/background click states while keeping the system small enough to remain coherent as new tools are added.