packages/cua-driver/docs/browser-existing-profile-attachment-plan.md
7fa6c80f across macOS, Windows, Linux X11, and Linux Wayland on Swaymain at 0835daa6 (feat(cua-driver): add capability-aware browser tools)codex/browser-existing-profile-reconnect-plan; canonical replay is complete and the PR remains draft for reviewThis plan is based on a read-only audit of the merged browser tools, platform adapters, approval path, CDP transport, legacy macOS page implementation, recording system, and Rust E2E harnesses. A separate Fable review challenged the public contract, consent boundary, reconnect ordering, concurrency model, issue grouping, and pull request sequence.
The project is done when an agent can attach to an existing authenticated Chromium profile after one genuine host/user approval, continue through bounded socket reconnects without asking the model to approve security UI, and either act on the exact intended tab or return an exact structured refusal. Completion requires:
The merged browser tools support exact Chromium binding and driver-owned
isolated_new or isolated_named profiles. They deliberately do not prepare a
person's already-running authenticated profile.
The legacy macOS page route proves that an existing Chrome profile can expose a
browser-level CDP endpoint through chrome://inspect/#remote-debugging. That
route has three limitations:
Allow remote debugging? prompt for every genuinely new
WebSocket connection;The current core CdpPool replaces a closed socket and retries once, but it
does not have a consent state machine, connection generations, cancellation,
or a mutation coordinator. The current MCP server also injects an internal
approval marker for every browser_prepare call. That marker is protected from
client forgery, but it proves transport provenance, not an interactive user
decision. It is insufficient for attaching to a personal browser profile.
browser_prepare without changing
the meaning of driver-owned isolated profiles.get_browser_state strictly read-only.page surface in the same pull request stack.PrepareProfileMode continues to mean driver-owned profile lifecycle and keeps
only isolated_new and isolated_named. Existing profiles are not owned by the
driver and must not inherit isolated-profile creation or cleanup behavior.
Add an optional tagged strategy object. Calls that omit it retain today's
behavior. Current allow_launch and profile fields remain accepted as a
compatibility form and are normalized internally.
{
"pid": 4242,
"window_id": 991,
"session": "agent-session",
"strategy": {
"kind": "existing_profile"
}
}
window_id is required for existing_profile. The endpoint grant is
process/profile-wide, but the selected native window provides the user-visible
approval anchor and an exact initial ownership proof. Browser product is
derived from the process and is never trusted from a caller claim.
Supplying strategy together with legacy allow_launch or profile fields is
an invalid request. A later cleanup may expose tagged isolated strategies, but
that is not required for this feature.
The public request never contains approved: true or a reusable grant ID.
When no valid grant exists, browser_prepare returns the existing structured
browser_consent_required refusal with a short-lived approval request ID in
detail. The request ID correlates a host UI action; possession does not grant
permission.
Approval providers are:
browser-approve CLI artifact extended to bind the exact
existing-profile request digest; andThe current automatically injected MCP host marker remains sufficient only for
the existing isolated-launch contract. It must never authorize
existing_profile.
An approved request mints an in-memory ExistingProfileGrant scoped to:
The internal profile identity may be a daemon-salted hash of a canonical path, but the path or hash is never returned, logged, or recorded. If a platform cannot prove profile identity independently, the initial release binds the grant to the current process fingerprint and requires approval after a browser restart.
Daemon restart always discards the grant. Durable grants backed by Keychain, DPAPI, or Secret Service require a separate security design and are deferred.
Chrome may prompt once per genuinely new WebSocket, not once per operation.
The coordinator therefore owns exactly one browser-level
/devtools/browser socket for an attachment generation and multiplexes tabs
through flattened Target.attachToTarget sessions.
Core must not open independent per-target sockets for an approved existing profile. If a browser product cannot support the browser-level route, that product is unsupported until its prompt count and connection semantics are modeled explicitly.
The new route becomes the sole socket owner for a grant-owned endpoint. The legacy page cache must refuse access to such an endpoint and direct callers to the first-class browser tools. This prevents duplicate prompts and split-brain connection ownership.
The browser prompt is caused by opening the WebSocket, so the driver cannot click it before starting the connection. The state machine must initiate a bounded connect, observe the exact resulting prompt, press its exact semantic action, and then confirm that the same pending connection completed.
approval_required
-> grant_active
-> endpoint_preflight
-> connecting_pending_consent
-> socket_ready
-> consent_prompt_observed -> consent_pressed -> socket_ready
-> consent_dismissed -> grant_revoked
-> timeout_or_ambiguity -> refused
-> endpoint_reproved
-> capabilities_invalidated
-> ready_for_rebind
On socket loss:
ready
-> reconnect_singleflight
-> mutation_gates_closed
-> process_profile_endpoint_revalidated
-> connecting_pending_consent
-> endpoint_generation_incremented
-> old_capabilities_invalidated
-> mutation_gates_reopened
-> ready_for_rebind
The first implementation uses a fixed policy rather than caller-tunable security limits:
Prompt handling reuses the same native perception and semantic action
primitives that support get_window_state and AX/UIA/AT-SPI clicks, but it does
not invoke public MCP tools from inside browser_prepare or CDP reconnect.
Recursive public calls would create policy, recording, and lifecycle deadlocks.
Each platform provides a narrow internal BrowserConsentUi adapter that can:
A candidate is actionable only when all available independent evidence agrees:
Localized text may corroborate identity but cannot be the sole proof. If a browser/version does not expose stable semantics, the adapter refuses it.
Every connection has an internal monotonically increasing endpoint generation.
TargetRecord, TabRecord, and page refs record the generation that minted
them. A successful reconnect increments the generation and invalidates every
older target, tab, snapshot, frame, and ref capability.
The caller receives capabilities_invalidated: true and must call
get_browser_state again with the new native (pid, window_id) before any
mutation. The driver never silently remaps an old capability to a new CDP
target.
Add a mutation gate keyed by canonical browser identity plus CDP target ID, not by public session ID. This serializes mutations from multiple sessions that resolve to the same real tab while allowing proven independent tabs to proceed in parallel.
browser_navigate, browser_click, browser_type, and future file upload
hold the gate across revalidation, dispatch, and outcome verification. A
reconnect closes all gates for that browser identity. In-flight actions either
complete on the old generation or return a stale-generation refusal; they
never continue on the new socket.
browser_type additionally reports requested and delivered character counts.
Keystroke mode updates the delivered count only after the full key-down/key-up
pair succeeds. A partial write returns a structured browser_input_incomplete
refusal with retryability metadata instead of status: ok.
This prevents the first-class browser surface from repeating the partial and
interleaved delivery classes tracked in
#2255 and
#2256. Those issues' native
type_text paths remain separate fixes.
Ordinary before/after screenshots of a live authenticated browser can expose URLs, page text, or authentication state. Consent turns therefore record:
Recording and telemetry must never contain profile paths, profile hashes, ports, WebSocket URLs or tokens, cookies, storage, page content, tab URLs, JavaScript, or approval artifacts.
Existing successful isolated results remain unchanged. Existing-profile success is additive:
{
"status": "ok",
"prepared": true,
"action": "attached_existing_profile",
"prepared_pid": 4242,
"attachment": {
"kind": "existing_profile",
"browser": "chromium",
"capabilities_invalidated": true,
"next_action": "get_browser_state"
},
"endpoint_ownership": {
"method": "listening_socket_pid",
"owner_pid": 4242
},
"side_effects": {
"launched_browser": false,
"restarted_browser": false,
"created_profile": false,
"reused_driver_profile": false,
"copied_profile_data": false,
"changed_preferences": false,
"displayed_consent_prompt": true
}
}
Do not return a grant ID, profile identity, endpoint generation, port, or WebSocket URL. Those are internal security and lifecycle facts, not caller capabilities.
Use existing refusals where their meaning is exact:
browser_consent_required for no valid user grant;browser_endpoint_owner_mismatch for endpoint identity failure;browser_binding_stale after generation change;browser_wrong_target_refused for prompt/window/target mismatch;browser_route_unavailable for unsupported browser/platform setup.Add only the codes callers need to branch on distinctly:
browser_consent_revoked when the person dismisses or denies the prompt;browser_reconnect_exhausted after the bounded reconnect policy fails;browser_input_incomplete with requested and delivered counts.All refusal additions must also update the testkit's closed RefusalCode
mirror and serialization tests.
Three events have different authorization semantics:
| Event | Default behavior |
|---|---|
| Socket drop, same process/profile | Reconnect under the live grant after full revalidation |
| Browser restart | Reuse the grant only if exact browser and opaque profile identity can be re-proven; otherwise require approval |
| Driver daemon restart | Always require approval in v1 |
If no remote-debugging endpoint exists, v1 must not restart the browser with
flags. The first macOS implementation may automate Chrome's own internal
remote-debugging setup UI only after the same user grant and only when every
navigation, toggle, prompt, and cleanup step is exact and recorded as a
declared side effect. Until that path has real evidence, return
browser_requires_setup with a human action and leave the profile untouched.
| Platform | Initial scope | Required proof before support | Unsupported/refusal behavior |
|---|---|---|---|
| macOS | Chrome first; Edge only after separate evidence | Bundle/team identity, process fingerprint, CGWindow ownership, exact AX prompt semantics, PID-owned loopback endpoint, exact native/CDP rebind | Refuse unrecognized prompt or browser build |
| Windows | Chrome and Edge after a UIA feasibility spike | Authenticode/process identity, HWND ownership, UIA modal/action semantics, exact PID listener, interactive desktop E2E | Refuse when prompt behavior or interactive session is unproven |
| Linux X11 | Chromium/Chrome after AT-SPI spike | Executable/process identity, _NET_WM_PID, exact geometry, AT-SPI modal/action semantics, /proc socket inode owner | Refuse unrecognized desktop/browser combinations |
| Linux Wayland | Exact compositors only, beginning with validated Sway/GNOME paths | Exact compositor pid/window identity plus AT-SPI prompt proof | Generic Wayland remains read-only or returns browser_route_unavailable |
Support is per browser version range and desktop environment, not inferred from the existence of AX/UIA/AT-SPI alone.
| Path | Planned responsibility |
|---|---|
crates/cua-driver-core/src/browser/platform.rs | Tagged prepare strategy, approval scope, platform consent/identity contracts |
crates/cua-driver-core/src/browser/approval.rs | Existing-profile request digest and single-use approval artifact validation |
crates/cua-driver-core/src/browser/grant.rs (new) | In-memory grant store, expiry, scope, revocation, session cleanup |
crates/cua-driver-core/src/browser/reconnect.rs (new) | Bounded state machine, singleflight, cancellation, endpoint generations |
crates/cua-driver-core/src/browser/mutation.rs (new) | Canonical-target mutation gates and reconnect exclusion |
crates/cua-driver-core/src/browser/cdp_ws.rs | One browser-level socket per generation, flattened target sessions, generation-aware pool eviction |
crates/cua-driver-core/src/browser/store.rs | Generation on target/tab/ref records and browser-wide invalidation |
crates/cua-driver-core/src/browser/engine.rs | Coordinator integration, rebind and mutation gate lifecycle |
crates/cua-driver-core/src/browser/tools.rs | Additive schema/result fields and honest type completion metadata |
crates/cua-driver-core/src/browser/refusal.rs | Additive reconnect, revocation, and incomplete-input refusals |
crates/cua-driver-core/src/server.rs and crates/cua-driver/src/serve.rs | Operation-bound host approval broker; no automatic existing-profile approval |
crates/cua-driver/src/cli.rs | Interactive existing-profile approval request and exact scope display |
Each *BrowserPlatform becomes stateful enough to own a narrow consent adapter
constructed from the platform's existing internal desktop primitives. Do not
give core access to the full platform ToolState.
| Platform | Planned paths |
|---|---|
| macOS | platform-macos/src/browser/platform.rs, new browser/consent_ui.rs, registration in tools/mod.rs, reuse ax and input/ax_actions.rs |
| Windows | platform-windows/src/browser_platform.rs, new browser_consent_ui.rs, registration in tools/impl_.rs, reuse UIA semantic actions |
| Linux | platform-linux/src/browser_platform.rs, new browser_consent_ui.rs, registration in tools/impl_.rs, reuse AT-SPI and compositor identity paths |
| Path | Planned responsibility |
|---|---|
cua-driver-core/src/browser/v2_tests.rs | Fake grant, prompt, endpoint, generation, and concurrent mutation tests |
cua-driver-testkit/src/e2e.rs | New refusal codes, browser attachment case declarations, evidence contract |
cua-driver/tests/standalone_browser_behavior_test.rs | Disposable existing-like Chromium profile E2E and adversarial endpoint rows |
.github/workflows/e2e-rust-standalone-browsers.yml | Maintainer-dispatched Windows/Linux compatibility evidence |
tests/runners/macos-lume/ | TCC-stable macOS real-browser evidence |
docs/content/docs/ | Diataxis tutorial, how-to, reference, and explanation updates |
rust/Skills/cua-driver/BROWSER.md and platform skill docs | Agent flow, refusal handling, and platform limits |
Tests use a disposable repository-created profile that behaves like an existing profile but contains no personal data. The driver does not own or delete it during the test. Independent fixture journals verify page effects; native sentinels verify focus, z-order, cursor, and leaked input where the operation claims background behavior.
Required rows per supported platform/browser:
browser_type completion guarantees;| Issue | Relationship | Landing decision |
|---|---|---|
| #2192 | Direct existing authenticated-profile tracker | Closed by this implementation stack |
| #2255, #2256 | Partial and concurrent native text delivery; same bug class applies to browser mutations | Browser regression guarantees in this stack; native fixes remain separate |
| #2176 | Web fallback should prefer a background browser route over foreground native input | Update recommendation/escalation to typed browser tools in this stack; close only when live behavior matches |
| #2240 | CDP file upload action | Separate stacked PR after mutation gates land |
| #2084 | Legacy page typing parity | Mark superseded by shared browser tools where applicable; legacy delegation remains separate |
| #2245 | Legacy macOS page fallback provenance | Separate legacy migration/cleanup PR |
| #2201, #2202 | Native Chrome click/drag behavior | Linked compatibility issues, not closed by browser attachment |
| #1616 | Electron/Chromium native accessibility exposure | Separate native/embedded surface issue |
These issues should be addressed in the same initiative where they share core primitives, but not placed in one code review. A single large pull request would mix approval security, transport lifecycle, mutation correctness, three native accessibility stacks, and legacy migration.
| PR | Scope | User-visible change | Exit gate |
|---|---|---|---|
| 1 | Approval and schema contract | existing_profile request returns a real approval requirement; no attachment yet | Security review; forged/replayed/cross-session approval tests pass; isolated behavior unchanged |
| 2 | Reconnect coordinator and connection generations | No advertised platform support yet | Fake-CDP state machine, singleflight, cancellation, prompt-dismissal, generation invalidation, and pool ownership tests pass |
| 3 | Mutation gates and honest typing | Browser mutations serialize by canonical target; partial type is explicit | Concurrent same-tab/cross-session tests, cross-tab parallel test, mid-write failure counts, reconnect race tests pass |
| 4 | macOS Chrome existing-profile adapter | Experimental user-approved attach and reconnect on macOS | Lume real-Chrome first attach, repeat, socket loss, restart/refusal, decoy, dismissal, privacy, and regression rows pass |
| 5 | Windows Chrome/Edge adapter | Experimental support for products proven by the UIA spike | Interactive Windows E2E matches PR 4 evidence; unsupported product builds refuse |
| 6 | Linux X11 and exact Wayland adapters | Experimental support only for validated browser/desktop combinations | X11 and each advertised compositor pass real-browser rows; generic Wayland refusal is tested |
| 7 | Docs, skills, diagnostics, and issue reconciliation | Public setup/reconnect guidance and accurate support matrix | Diataxis docs, bundled skills, schemas, diagnostics, and issue states agree with evidence |
| 8 | Stacked file upload from #2240 | Typed browser file-input action through the mutation gate | CDP fixture verifies files, count, input/change events, path validation, and reconnect race behavior |
No incomplete platform is advertised between PRs. Experimental registration or capability reporting remains disabled until that platform's E2E exit gate is accepted.
Use the Diataxis structure already established for CUA Driver:
browser_consent_required, revocation, and reconnect exhaustion.browser_prepare strategy schema and result fields.Reviewers should explicitly decide these points before PR 1 implementation:
browser_input_incomplete include a caller-retry hint when repeating
the remaining suffix may be unsafe for stateful editors?The canonical standalone-browser matrix was replayed from exact source SHA
7fa6c80f5c433a7d4dadef75e4873af89de0cd3f after the final testkit posture
correction. That correction waits for the foreground sentinel to become ready,
then explicitly activates and proves the exact sentinel instead of assuming an
application will take focus during launch. It changes setup reliability only;
no action oracle or expected product outcome was relaxed.
| Platform | Representative products | Rows | Delivered | Expected refusals | Failed | Skipped | Videos |
|---|---|---|---|---|---|---|---|
| Windows/Win32 | Chrome, Edge | 20 | 18 | 2 | 0 | 0 | 20 |
| macOS/Quartz | Chrome, Edge | 20 | 16 | 4 | 0 | 0 | 20 |
| Linux/X11 | Chrome | 10 | 8 | 2 | 0 | 0 | 10 |
| Linux/Wayland | Chromium on Sway | 10 | 8 | 2 | 0 | 0 | 10 |
| Total | 60 | 50 | 10 | 0 | 0 | 60 |
Every expected refusal was exact and fail-closed. All platforms refused a
native-window/CDP bind when two browser windows had indistinguishable bounds
and titles. macOS and Linux also refused Chromium's trusted CDP Input click
because that route activated the standalone browser; callers can use the
ref-targeted dom_event route when a synthetic full-background click is
acceptable. Windows delivered the trusted route without changing the sentinel
posture.
The final reports prove existing-profile setup, attach, exact multi-tab and frame binding, isolated launch, background type and mutation, stale-ref invalidation, trusted-input policy, ambiguity refusal, focus and z-order preservation, no leaked input, cursor preservation where available, and video capture. Local testkit validation also passed all 46 library tests. CI at the same branch head passed Windows and Linux unit/compile, Nix package and policy, documentation, link, distribution-compatibility, and publication checks.
No approval token, profile path, endpoint token, host address, machine name, or credential is recorded in committed documentation or the pull request.
The implementation is ready for security and maintainer review in draft PR #2261. Keep the advertised support matrix limited to the representative products and desktops proven above: Chrome and Edge on macOS and Windows, Chrome on X11, and Chromium on Sway. Linux product descriptors for Chrome, Chromium, and Edge remain available, but an untested product/desktop pair must not be described as release-accepted.
Keep Safari, Firefox, unknown browser builds or locales, and generic Wayland compositors as structured refusals until they have an exact native setup adapter and representative real-browser evidence. Treat #2240 as the first consumer of the mutation gate in a separate follow-up; legacy page migration remains separate.