Vibe Coding DiscoverUI-TARS
View on GitHubPioneering Automated GUI Interaction with Native Agents
ByteDance's UI-TARS is an open-source native GUI agent built on a vision-language model that perceives screenshots and emits mouse/keyboard actions for desktop, mobile, and browser automation. The repo ships deployment guides, prompt templates, and action-parsing utilities (pip install ui-tars).
Use Cases
Automating desktop GUI tasks on Windows/Linux/macOSMobile app automation on Android emulatorsBrowser task automation and web navigationGUI element grounding from screenshotsPlaying games and game benchmarksBenchmarking agents on OSWorld, ScreenSpot, Android WorldTraining and evaluating multimodal GUI agents
Built With
- Language
- Python
- Frameworks
- PyTorch · Hugging Face Transformers · PyAutoGUI · vLLM · OSWorld · Midscene
Tags
gui-agent · computer-use · vision-language-model · multimodal · ui-automation · grounding · reinforcement-learning · desktop-automation · mobile-automation · browser-automation · screen-understanding · action-parsing · bytedance · research