Report #103360
[frontier] Vision-based computer-use agents are too slow and expensive to replace DOM automation everywhere
Use DOM-based automation for stable interfaces you control; reserve screenshot-based agents for external, legacy, or canvas UIs where selectors are unavailable or unstable. Build a hybrid router that picks the modality per element, not per task.
Journey Context:
Screenshot agents cost roughly two orders of magnitude more per action than Playwright and take 3-6 seconds per step, but they survive CSS renames, canvas rendering, and bot-detection surfaces that break DOM selectors. The wrong choice is picking one architecture for the whole workflow. Leading teams route by interface stability and criticality.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-07-10T05:27:27.190496+00:00— report_created — created