clawdcursor

View on GitHub

clawdcursor compiles whatever's on screen into one UI map — accessibility tree and OCR fused into stable, addressable elements, with a screenshot only when needed — then drives apps through reusable scripts, verifying every action and routing it through a single safety gate.

★ 40259 forksTypeScriptMITAmrDab

Local MCP server giving any tool-calling agent safe control of the real desktop on Windows, macOS, and Linux. It compiles the screen into one UI map (accessibility tree + OCR) with stable element IDs, acts by ID with reaction verification, and routes every action through one safety gate.

Use Cases

Drive native desktop apps that have no API or CLIFill forms and click through legacy GUIsReply to emails and operate the mail clientAutomate browser and canvas-only UIsScreen understanding without a vision model via a11y tree + OCRVerify that an agent action actually changed the UIGate dangerous desktop actions behind a single safety checkpointRun local, private computer-use agents with any tool-calling modelAttach desktop control to Claude Code, Cursor, Windsurf, Zed, or CodexBuild reusable scripts for repeated GUI workflowsFallback execution layer when APIs/CLIs are unavailableHuman-in-the-loop desktop control with a stop banner

Built With

Language: TypeScript
Frameworks: Model Context Protocol · Node.js · TypeScript · nut.js · Tesseract OCR · sharp · Swift Package Manager

clawdcursor

Use Cases

Built With

Tags