Screen OCR plus a small classifier does computer use at $0.0002 a decision against $0.032 for Opus 5
typesafe-computer-use captures the frontmost macOS window with smart cropping and tile reuse, walks the accessibility tree for labeled controls pruned to display bounds, merges both into a state with date parsing and hints, then asks a small decision model three structured questions (action kind, target item, website) instead of sending a screenshot to a frontier model. Measured on the same screenshot and goal, one decision each, against Claude Opus 5: $0.0002 versus $0.032 and 0.13 to 0.38 s versus 5.2 s, with an end-to-end step including capture at about 1.5 s versus about 5.5 s, and a twelve-step task at $0.003 versus $0.40 to $0.90. The catch is stated plainly in the README: reasoning a frontier model does for free, such as parsing a date off pixels, has to become deterministic preprocessing here. MIT licensed, macOS 14+ and Python 3.12+.
↳ Follow the thread