Skip to content

fix(computer): fail closed on malformed macOS AXSubrole - #1385

Merged
swjnxyf merged 1 commit into
yyjeqhc:mainfrom
krsnak:fix/macos-ax-subrole-fail-closed-20261107
Nov 7, 2026
Merged

swjnxyf merged 1 commit into
yyjeqhc:mainfrom
krsnak:fix/macos-ax-subrole-fail-closed-20261107

Conversation

@krsnak

@krsnak krsnak commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Recover from macOS AXUIElementCopyAttributeValue(AXSubrole) returning AXError(-25200) without discarding the entire accessibility tree.
  • Treat an unreadable or unexpectedly typed AX subrole as unverified and protected, suppress identifying and text metadata for that element, and propagate protection through its descendants.
  • Preserve fail-closed behavior for other AX failures and existing permission-denied semantics. Include the fixed AX attribute name in diagnostic errors without exposing its value.
  • Add focused redaction/identity unit tests and an opt-in live macOS Calculator regression.

Observed reproduction

Directly reproduced with WebCodex 0.5.0 on macOS (Apple M1). observe_computer(action=accessibility_tree) on a Calculator window failed with:

accessibility_failed: AXUIElementCopyAttributeValue failed with AXError(-25200)

On isolated current main (d8e437b4), diagnostic instrumentation identified attribute=AXSubrole. An opt-in Calculator test passed at tree depths 0 and 1, but failed at depth 2 before this fix. After this change it passes at depths 0–3.

The root cause inside Apple's AX provider is not established; the fix conservatively handles a bounded failure case rather than assuming the subrole is absent.

Safety

  • Unverified-subrole elements are marked protected; their text/values are not read, they lack positive identifying metadata, and they cannot be used for native control or text entry. Descendant protection is inherited; verified siblings remain unaffected.
  • A successful AX response carrying a non-CFString (tested using CFBoolean) is also protected, not treated as a benign missing subrole.
  • AXError::APIDisabled remains permission_denied; unrelated failures remain failures. No changes to authorization, installation, Server, or Runner process lifecycle.

Validation

  • cargo fmt --all -- --check — passed
  • git diff --check — passed
  • cargo test --locked --offline -p webcodex-computer --lib — 90 passed, 0 failed, 6 ignored
  • cargo test --locked --offline -p webcodex-computer computer_macos_accessibility_calculator_live_regression -- --ignored --nocapture — 1 passed
  • Read-only cross-application check on the same Mac: Chrome, Terminal and System Settings still pass; Calculator now passes. TextEdit exposes a separate AXDescription AXError(-25200), deliberately left outside this PR.
  • Production WebCodex 0.5.0 was not replaced or restarted and its /healthz returned HTTP 200.

Related to #1161 (Computer Use ergonomics/reliability); this patch does not close the broader issue.

@swjnxyf swjnxyf left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all three changed files and the macOS AX read / fingerprint / tree / scoped-correlation / native control paths at 87b8d18 on mini. AXSubrole provider Failure and unexpected CF value types conservatively mark the element protected; inherited protection suppresses title, description, placeholder, AXValue, native press/focus/scroll and text input. Unverified nodes lack positive identity evidence; ordinary NoValue/unsupported, API-disabled, and other errors retain their respective original fail-closed handling. Exact AX attribute names improve diagnostics without exposing values. On mini (macOS arm64), cargo test --locked --offline -p webcodex-computer --lib passed: 90 passed, 0 failed, 6 intentionally ignored; cargo fmt --all -- --check and git diff --check passed. The optional live Calculator test is intentionally not run here because no Calculator window is open; the PR documents successful author's opt-in live test. Hosted CI was still running at review time, with no failures observed. No blocking findings or follow-up commits required. Approve; do not merge.

@swjnxyf
swjnxyf enabled auto-merge (squash) November 7, 2026 09:02
@swjnxyf
swjnxyf merged commit 09f1322 into yyjeqhc:main Nov 7, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants