Vercel Labs ships agent-desktop 1.2.0 with MCP so agents can operate desktop apps
Vercel Labs released version 1.2.0 of its open-source tool agent-desktop on November 5. The tool lets AI agents operate desktop software without screenshots: it reads the operating system's accessibility tree, the description of an app's interface that screen readers use, and returns every element an agent can act on as JSON with a reference ID. The agent clicks or types on a reference, then reads the interface again to check the result. It runs on macOS, Windows and Linux, ships as a command-line tool and a C library, and uses the Apache-2.0 license. The new agent-desktop mcp command offers each command as an MCP tool. MCP is an open protocol that lets agents call outside tools, so agents that support it can connect without wrapping the command line.
Why it matters
Agents usually operate desktop apps by reading screenshots. This tool reads the interface structure instead, and MCP support lets many more agents use it directly.
Sources1
Written by dalao.ai: the title, summary and why it matters are our own words. The original reports are the record; their publishers keep authorship.