FreeLLMVercel Labs Open‑Sources agent‑desktop, a Rust CLI for AI agents to control desktops
The new Apache‑2.0 tool lets AI agents read OS accessibility trees and act on stable references on macOS, Windows and Linux.
It is an open‑source CLI licensed under Apache‑2.0 and written in Rust.
The tool lets any AI agent interact with desktop applications by reading the operating system’s accessibility tree instead of using pixel detection. It returns JSON snapshots with references that agents can click or manipulate safely.
agent‑desktop runs on macOS, Windows and Linux. On macOS it uses a private command host that retains native accessibility objects and exits after 15 minutes of idle time. Windows needs no extra grant, while Linux requires a graphical session with at‑spi2‑core.
Installation is possible via npm (global install) for x64 and ARM64 binaries, or by downloading native archives from GitHub Releases. Homebrew and Scoop packages are being prepared but are not yet in default catalogs.
Users must grant Accessibility permission on macOS and keep temporary directories private. An optional JEV helper can send screen descriptions to a model provider, but snapshots may contain private content and should not be logged publicly.
Was das für dich bedeutet
For developers building AI agents, agent‑desktop provides a reliable way to automate GUI tasks without fragile pixel matching, simplifying error handling. Its open‑source, cross‑platform design helps hobbyists and researchers experiment with free or locally hosted models, complementing the free chat models highlighted on FreeLLMArena.
Quellen
- Vercel Labs Open-Sources agent-desktop for Agent Computer Use, MarkTechPost, 31 Oct 2026
- GitHub - vercel-labs/agent-desktop: Agent Desktop gives any agent reliable computer use on the desktop. Built with Rust, it sees any app's real UI structure through OS accessibility trees and operates, github.com (official), 31 Oct 2026
Automatisch aus den obigen Quellen geschrieben, dann von unserem Code geprüft: Jede Zahl und jeder Name muss darin vorkommen, und kein Satz darf eine Quelle kopieren. Einen Fehler entdeckt? Melde ihn.
Welche KI kannst du heute kostenlos nutzen?
Jedes kostenlose KI‑Modell, bewertet durch blinde menschliche Stimmen, mit dem Speicher und der Geschwindigkeit, die es auf deiner eigenen GPU benötigt.