Vibe Coding Discovertypesafe-computer-use
View on GitHubComputer use for about $0.0002 a step: OCR the screen, classify the next action with TypeSafe, click. macOS.
macOS computer-use agent loop that OCRs the screen and the accessibility tree, then asks a TypeSafe classifier for one action per step (~$0.0002, sub-second); a writer LLM is only called for free text. Cheap alternative to screenshot-based frontier-model agents.
Use Cases
Drive a macOS app or browser to a goal from a plain-English promptAutomate multi-step web tasks like finding checkout pages or ticketsLow-cost GUI agent loops (about $0.0002 per decision)Click and type using accessibility elements instead of pixelsOCR-based screen understanding with tile-level capture reuseConfidence-gated action selection with a stop thresholdPress off-screen accessibility controls that OCR cannot seeReplay and debug agent runs offline from a saved image
Built With
- Language
- Python
- Frameworks
- TypeSafe SDK · Anthropic SDK · PyObjC · ocrmac · AppleScript · pytest · ruff · uv
Tags
computer-use · gui-automation · macos · ocr · accessibility-tree · apple-vision · agent-loop · text-classification · cheap-inference · screen-perception · deterministic-state · click-automation · confidence-gating · python