FreeLLMVercel Labs Open‑Sources agent‑desktop, a Rust CLI for AI agents to control desktops
The new Apache‑2.0 tool lets AI agents read OS accessibility trees and act on stable references on macOS, Windows and Linux.
It is an open‑source CLI licensed under Apache‑2.0 and written in Rust.
The tool lets any AI agent interact with desktop applications by reading the operating system’s accessibility tree instead of using pixel detection. It returns JSON snapshots with references that agents can click or manipulate safely.
agent‑desktop runs on macOS, Windows and Linux. On macOS it uses a private command host that retains native accessibility objects and exits after 15 minutes of idle time. Windows needs no extra grant, while Linux requires a graphical session with at‑spi2‑core.
Installation is possible via npm (global install) for x64 and ARM64 binaries, or by downloading native archives from GitHub Releases. Homebrew and Scoop packages are being prepared but are not yet in default catalogs.
Users must grant Accessibility permission on macOS and keep temporary directories private. An optional JEV helper can send screen descriptions to a model provider, but snapshots may contain private content and should not be logged publicly.
Qué significa para ti
For developers building AI agents, agent‑desktop provides a reliable way to automate GUI tasks without fragile pixel matching, simplifying error handling. Its open‑source, cross‑platform design helps hobbyists and researchers experiment with free or locally hosted models, complementing the free chat models highlighted on FreeLLMArena.
Fuentes
- Vercel Labs Open-Sources agent-desktop for Agent Computer Use, MarkTechPost, 31 Oct 2026
- GitHub - vercel-labs/agent-desktop: Agent Desktop gives any agent reliable computer use on the desktop. Built with Rust, it sees any app's real UI structure through OS accessibility trees and operates, github.com (official), 31 Oct 2026
Escrito automáticamente a partir de las fuentes anteriores, luego revisado por nuestro código: cada número y nombre debe aparecer en ellas, y ninguna oración puede copiar una fuente. ¿Detectaste un error? Infórmalo.
¿Qué IA puedes usar gratis hoy?
Todos los modelos de IA gratuitos, clasificados por votos humanos ciegos, con la memoria y velocidad que necesita tu GPU.