fix: rewrite getByRole to use CDP accessibility tree with ref-based element resolution (#1145)

* fix: rewrite getByRole to use CDP accessibility tree instead of CSS selectors

The old `handle_getbyrole` generated `querySelectorAll('[role="link"], link')`
which matched `<link>` stylesheet elements instead of `<a>` anchor tags.
This happened because ARIA role names were used directly as CSS tag selectors,
and several roles differ from their HTML element names (e.g. link → a,
heading → h1-h6, textbox → input/textarea).

The fix replaces the JS-based DOM query with the CDP `Accessibility.getFullAXTree`
API, where the browser engine correctly computes implicit ARIA roles per the
WAI-ARIA / HTML-AAM spec. This is the same approach already used by `snapshot.rs`
and `element.rs` in this codebase.

Changes:
- Rewrite `handle_getbyrole` to query the browser's accessibility tree via CDP
- Add `find_ax_node_by_role` helper for AX tree traversal with role/name/exact matching
- Use `DOM.resolveNode` + `Runtime.callFunctionOn` to bridge AX node → DOM marker
- Add iframe support via `resolve_ax_session` (missing in old implementation)
- Fix cleanup to use correct CDP session (old code used default session, breaking iframe cleanup)
- Export `extract_ax_string` as `pub(super)` for reuse
- Add 4 regression tests for `find_ax_node_by_role`

Fixes #1123

* style: apply cargo fmt

* chore: remove redundant comments

* refactor: replace marker attribute with temporary ref for element resolution

Eliminates 3 CDP round-trips (DOM.resolveNode, Runtime.callFunctionOn,
Runtime.evaluate cleanup) by registering a temporary ref in the ref_map.
execute_subaction resolves the element via backendNodeId directly.
No more DOM pollution with marker attributes.

* fix: ref counter collision, ref_map leak, and stale fallback name

- Increment next_ref_num after inserting temp ref to prevent id collision
- Remove temp ref after execute_subaction to prevent unbounded ref_map growth
- Return actual AX name from find_ax_node_by_role for accurate fallback resolution
- Add RefMap::remove method

---------

Co-authored-by: hyunjinee <leehj0110@kakao.com>
This commit is contained in:
jin.2
2026-04-05 09:10:24 -05:00
committed by GitHub
co-authored by hyunjinee
parent 1205e2ca9c
commit 9f51879012
2 changed files with 158 additions and 60 deletions
+5 -1
View File
@@ -103,6 +103,10 @@ impl RefMap {
entries
}
pub fn remove(&mut self, ref_id: &str) {
self.map.remove(ref_id);
}
pub fn clear(&mut self) {
self.map.clear();
self.next_ref = 1;
@@ -380,7 +384,7 @@ async fn find_node_id_by_role_name(
))
}
fn extract_ax_string(value: &Option<AXValue>) -> String {
pub(super) fn extract_ax_string(value: &Option<AXValue>) -> String {
match value {
Some(v) => match &v.value {
Some(Value::String(s)) => s.clone(),