feat: enhance snapshot usability by reducing AI cognitive load of semantic noise and -C flag (#968)

* fix: add ref for cursor-interactive content roles

* fix: format

* feat: always include cursor-interactive elements in snapshot, -C is deprecated

* feat: process StaticText aggregation and deduplication

* update test

* clean up

* fix: escape text of elements in snapshot

* fix: redundant slicing

* fix: cargo fmt

* feat: deduplicate redundant StaticText

---------

Co-authored-by: 羲洋 <lipengyang.lpy@alibaba-inc.com>
This commit is contained in:
Lppy
2026-03-23 09:35:43 -05:00
committed by GitHub
co-authored by 羲洋
parent 9c0955ca99
commit b48c3e9dd2
11 changed files with 160 additions and 116 deletions
-1
View File
@@ -113,7 +113,6 @@ agent-browser close # Close browser
# Snapshot
agent-browser snapshot -i # Interactive elements with refs (recommended)
agent-browser snapshot -i -C # Include cursor-interactive elements (divs with onclick, cursor:pointer)
agent-browser snapshot -s "#selector" # Scope to CSS selector
# Interaction (use @refs from snapshot)
-1
View File
@@ -217,7 +217,6 @@ AGENT_BROWSER_COLOR_SCHEME=dark agent-browser connect 9222
### Elements not appearing in snapshot
- The app may use multiple webviews. Use `agent-browser tab` to list targets and switch to the right one
- Use `agent-browser snapshot -i -C` to include cursor-interactive elements (divs with onclick handlers)
### Cannot type in input fields
-9
View File
@@ -235,15 +235,6 @@ agent-browser console
agent-browser errors
```
### View raw HTML of an element
```bash
# Snapshot shows the accessibility tree. If an element isn't there,
# it may not be interactive (e.g., div instead of button)
# Use snapshot -i -C to include cursor-interactive divs
agent-browser snapshot -i -C
```
### Get current page state
```bash
+2 -8
View File
@@ -334,19 +334,13 @@ If you can't find an element:
agent-browser snapshot -i
```
3. **Try snapshot with extended range**
```bash
# Include cursor-interactive elements (divs with onclick handlers)
agent-browser snapshot -i -C
```
4. **Check current URL**
3. **Check current URL**
```bash
agent-browser get url
# Verify you're in the right section
```
5. **Wait for page to load**
4. **Wait for page to load**
```bash
agent-browser wait --load networkidle
agent-browser wait 1000