docs(stream): document the bidirectional WS as the real-time driving path
Dogfooding (driving a canvas game) showed the slow, low-fidelity way — one screenshot + one CLI call per action — when chrome-use already ships the right tool: the session WebSocket is BIDIRECTIONAL. It streams ~60fps screencast frames AND accepts input_keyboard/input_mouse/input_touch on the same socket, straight to CDP Input.dispatch* — verified live over the extension relay (217 frames in 3.4s, ~64fps, and the input drove the game). But the inbound input protocol was undocumented, so agents default to the CLI-per-action grind. Document it in --help (stream) and the core skill: the frame + input message schemas and the 'connect once, read frames, send timed input' loop, with a node snippet. Reserve screenshots for one-off checks; use the WS for sustained real-time control.
This commit is contained in:
@@ -2778,6 +2778,20 @@ Notes:
|
||||
- Streaming is always enabled. Set AGENT_BROWSER_STREAM_PORT to bind to a
|
||||
specific port instead of the default OS-assigned port.
|
||||
|
||||
The WS is BIDIRECTIONAL — the high-throughput way to drive a live/real-time page
|
||||
(games, canvas apps) instead of one screenshot + one CLI call per action:
|
||||
- Server -> client (JSON text frames):
|
||||
{"type":"frame","data":"<base64 jpeg>"} live screencast (~60fps)
|
||||
plus status / tabs messages.
|
||||
- Client -> server (send JSON text):
|
||||
{"type":"input_keyboard","eventType":"keyDown|keyUp","key":" ","code":"Space",
|
||||
"windowsVirtualKeyCode":32}
|
||||
{"type":"input_mouse","eventType":"mousePressed|mouseReleased|mouseMoved",
|
||||
"x":640,"y":360,"button":"left","clickCount":1}
|
||||
{"type":"input_touch","eventType":"touchStart|touchEnd","touchPoints":[...]}
|
||||
Connect once and run a tight local loop: read frames, send timed input — no
|
||||
per-action process spawn, no round-trip. Works over the extension relay too.
|
||||
|
||||
Global Options:
|
||||
--json Output as JSON
|
||||
--session <name> Use specific session
|
||||
|
||||
Reference in New Issue
Block a user