Now liveThe Skillselion MCP - thousands of ranked skills, loaded into your agent mid-task. No install.Get it →

patrickporto/desktop-agent

1 skill · 3.9k installs · 1 stars · GitHub

Install

npx skills add https://github.com/patrickporto/desktop-agent

Skills in this repo

1Desktop ControlDesktop Control is a PyAutoGUI-based skill that gives AI agents programmatic control over mouse movement, keyboard input, screenshot capture with OCR, and application lifecycle management. Developers invoke commands via the `uvx desktop-agent` CLI with structured JSON responses. Key workflows include form filling, UI element location through image matching or text recognition, application launching and focusing, and desktop state verification. The skill supports cross-platform automation (Windows, macOS, Linux) with safety considerations like fail-safe mechanisms and coordinate validation. Common use cases span RPA, testing automation, screenshot analysis, and interactive workflow orchestration. Five command categories: mouse control, keyboard input, screen capture with OCR, message dialogs, and app lifecycle management. Image-based UI element location via `screen locate-center` with configurable confidence thresholds. OCR text detection: `screen locate-text-coordinates` and `screen read-all-text` for window-specific or full-screen text extraction. Cross-platform application control: open, focus, and list windows with platform-aware app launching. Structured JSON output with error.3.9kinstalls

Five minutes, every Monday - the tools, releases and tactics for developers.

unsubscribe anytime.

patrickporto/desktop-agent · 1 skills · Skillselion