feat(automation): replay pointer events on the canvas

The new pointer op sends moves (canvas pixels, or a world point projected
through the camera the cursor edits with), button presses/releases and
wheel steps in order, and reports the cursor's world point after each, so
grip drags and picks go through the same cursor path as the mouse.
click now projects through that editing camera too (it used the model
camera inside an entered viewport), and the grips read reports each
grip's constraint axis.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Hakan SevenandClaude Opus 5.5 committed 2026-10-13 12:12:27 +03:00
1 parent 006be8909e
commit df94117fe5
4 files changed
+121 -15

No files matched your search

+2 -1
View File
@@ -32,7 +32,7 @@ const MODERN_PROTOCOL_VERSION: &str = "2026-07-28";
///
/// `block_delete` gained its optional `force` parameter.
#[cfg(test)]
const TOOL_SCHEMA_DIGEST: &str = "96510df2d38cf3f7feb99e50e9491dda6fc1ee9221dc82020303df918292f580";
const TOOL_SCHEMA_DIGEST: &str = "4f193abfb8341acd6dc1a86ef7fe85148bc0bb76d1df700bcd289f79291a1d0f";
const MAX_REQUEST: usize = 1_048_576;
const MAX_RESPONSE: u64 = 16 * 1024 * 1024;
const CACHE_TTL_MS: u64 = 3_600_000;
@@ -85,6 +85,7 @@ const EXECUTE_OPS: &[&str] = &[
"property",
"grip_drag",
"click",
"pointer",
"set_properties",
"action",
"embed_image",