desktop automation

Native desktop automation CLI for AI agents. It uses OS accessibility trees to control apps with structured JSON output and deterministic element references, and includes both a Rust CLI and C-ABI library for in-process use.

Native desktop automation CLI for AI agents. It controls apps through OS accessibility trees and returns structured JSON with deterministic element references, with extra FFI bindings for embedding in other languages.

Open-source infrastructure for computer-use agents, with sandboxes, SDKs, benchmarks, and desktop automation tooling for macOS, Linux, Windows, and Android. The repo also includes Cua Driver, CuaBot, Cua-Bench, and Lume for VM management.

Local-first coding agent that runs on the user’s machine and can work across the browser, desktop, terminal, and editor surfaces. The repo says it verifies fixes by reproducing failures and re-running assertions, with install instructions for Windows, macOS, and Linux.

Open-source AI-powered desktop automation agent that gives an LLM a fully isolated Ubuntu workspace with browser-based supervision, MCP access, audit logs, and OS-level control.

A Windows desktop agent that lets AI interact with any app through visual GUI control, with a portable single-EXE build and support for ChatGPT, Gemini, and Claude.

Self-hosted voice assistant that can answer aloud, dictate text, run shell commands, control mouse and keyboard, and connect to MCP-enabled services. It can hand off heavier tasks to coding agents and run on local hardware or with hosted providers.

Flue is an open-source Python tool that lets coding agents control desktop software from the command line without MCP servers. The README says it works with apps like Photoshop, Blender, Unity, Houdini, and Microsoft Office, and supports harnesses such as Claude Code, Codex, Gemini CLI, Cursor, and OpenCode.

Open-source agent-first display server for safely running human and AI-driven GUI sessions with per-app isolation and capability-scoped authorization.