annotated screenshots (#503)
* screenshot annotation * fixes * fix CI checks * fixes * fixes * fixes * fixes * fixes
This commit is contained in:
@@ -23,6 +23,7 @@ agent-browser scrollintoview <sel> # Scroll element into view
|
||||
agent-browser drag <src> <dst> # Drag and drop
|
||||
agent-browser upload <sel> <files> # Upload files
|
||||
agent-browser screenshot [path] # Screenshot (--full for full page)
|
||||
agent-browser screenshot --annotate # Annotated screenshot with numbered element labels
|
||||
agent-browser pdf <path> # Save page as PDF
|
||||
agent-browser snapshot # Accessibility tree with refs
|
||||
agent-browser eval <js> # Run JavaScript
|
||||
@@ -236,6 +237,7 @@ agent-browser reload # Reload page
|
||||
--device <name> # iOS device name (e.g., "iPhone 15 Pro")
|
||||
--json # JSON output (for scripts)
|
||||
--full, -f # Full page screenshot
|
||||
--annotate # Annotated screenshot with numbered element labels
|
||||
--headed # Show browser window (not headless)
|
||||
--cdp <port|url> # Connect via Chrome DevTools Protocol (port or WebSocket URL)
|
||||
--auto-connect # Auto-discover and connect to running Chrome
|
||||
|
||||
@@ -78,12 +78,28 @@ agent-browser snapshot -i # Get fresh refs
|
||||
agent-browser click @e1 # Use new refs
|
||||
```
|
||||
|
||||
## Annotated screenshots
|
||||
|
||||
For visual context alongside text snapshots, use `screenshot --annotate` to overlay numbered labels on interactive elements. Each label `[N]` maps to ref `@eN`:
|
||||
|
||||
```bash
|
||||
agent-browser screenshot --annotate ./page.png
|
||||
# -> Screenshot saved to ./page.png
|
||||
# [1] @e1 button "Submit"
|
||||
# [2] @e2 link "Home"
|
||||
# [3] @e3 textbox "Email"
|
||||
agent-browser click @e2
|
||||
```
|
||||
|
||||
Annotated screenshots also cache refs, so you can interact with elements immediately. This is useful when the text snapshot is insufficient -- unlabeled icons, canvas content, or visual layout verification.
|
||||
|
||||
## Best practices
|
||||
|
||||
1. Use `-i` to reduce output to actionable elements
|
||||
2. Re-snapshot after page changes to get updated refs
|
||||
3. Scope with `-s` for specific page sections
|
||||
4. Use `-d` to limit depth on complex pages
|
||||
5. Use `screenshot --annotate` when visual context is needed alongside refs
|
||||
|
||||
## JSON output
|
||||
|
||||
|
||||
Reference in New Issue
Block a user