
Windows Ui Automation
Drive Windows desktop apps through UI Automation with audited sessions, permission tiers, and guarded find-and-interact patterns.
Overview
Windows UI Automation is an agent skill most often used in Build (also Ship) that implements secure UIAutomationClient sessions and guarded desktop interactions on Windows.
Install
npx skills add https://github.com/martinholovsky/claude-skills-generator --skill windows-ui-automationWhat is this skill?
- SecureAutomationSession context manager with UUID session ids and explicit cleanup
- Permission tiers (e.g., read-only) plus AutomationGuard limit checks before interactions
- ProcessValidator gate before find_and_interact on a target PID
- UIAuditLogger records session start/end for traceable agent runs
- TimeoutManager paired with CreateObject('UIAutomationClient.CUIAutomation') bootstrap pattern
- SecureAutomationSession pattern includes UUID session ids, AutomationGuard limit checks, and UIAuditLogger start/end eve
Adoption & trust: 1.1k installs on skills.sh; 38 GitHub stars; 2/3 security scanners passed (skills.sh audits).
What problem does it solve?
You need your agent or script to click through a real Windows app without anonymous COM calls, missing audit trails, or unvalidated target processes.
Who is it for?
Indie Windows developers automating in-house Win32/WPF apps, RPA experiments, or agent tools that must log and bound UI actions.
Skip if: macOS or Linux UI automation, pure web Playwright tests, or unattended production bots without reviewing permission tiers and audit requirements.
When should I use this skill?
Automating Windows desktop UI with UIAutomationClient and needing session security, audits, and validated interactions.
What do I get? / Deliverables
You get reusable SecureAutomationSession patterns with permission tiers, audit logging, and validated find-and-interact flows suitable for agent-driven desktop integration.
- SecureAutomationSession-based automation module
- Audit-logged UI interaction flows with permission-tier enforcement
Recommended Skills
Journey fit
Spans multiple journey phases - primary shelf plus alternate fits below.
Desktop automation code lands during Build when you wire agents or scripts to real Windows UI surfaces. Integrations subphase fits COM-based UIAutomationClient hooks and process-targeted actions rather than shipping marketing or infra-only work.
Where it fits
Embed SecureAutomationSession in an agent tool that files data from a legacy Windows accounting app.
Replay audited find-and-interact steps against a release candidate desktop build before shipping to customers.
How it compares
COM UI Automation templates with security wrappers—not a cross-browser MCP or generic screen-scraping skill.
Common Questions / FAQ
Who is windows-ui-automation for?
Solo builders on Windows who automate desktop applications through UIAutomationClient and want session IDs, audits, and validation baked in.
When should I use windows-ui-automation?
During Build integrations when wiring agents to Windows UI trees, and during Ship testing when you need repeatable audited interactions against a specific process.
Is windows-ui-automation safe to install?
The patterns can drive real UI and processes on your machine; review the Security Audits panel on this page and run only with least-privilege tiers in controlled environments.
SKILL.md
READMESKILL.md - Windows Ui Automation
# Windows UI Automation - Advanced Patterns ## Pattern: Secure Automation Session ```python from contextlib import contextmanager import uuid class SecureAutomationSession: """Managed automation session with full security controls.""" def __init__(self, permission_tier: str = 'read-only'): self.session_id = str(uuid.uuid4()) self.permission_tier = permission_tier self.uia = None self.audit_logger = UIAuditLogger() self.timeout_manager = TimeoutManager() self.guard = AutomationGuard() @contextmanager def session(self): """Context manager for safe automation session.""" try: self._initialize() yield self finally: self._cleanup() def _initialize(self): """Initialize automation with security checks.""" self.uia = CreateObject('UIAutomationClient.CUIAutomation') self.audit_logger.log_session_start(self.session_id, self.permission_tier) def _cleanup(self): """Clean up automation session.""" self.audit_logger.log_session_end(self.session_id) self.uia = None def find_and_interact(self, process: str, element_id: str, action: str, **kwargs): """Find element and perform action with full validation.""" # Check limits self.guard.check_limits() # Validate process pid = get_process_pid(process) if not ProcessValidator().validate_process(pid): raise SecurityError(f"Process validation failed: {process}") # Find element with timeout with self.timeout_manager.timeout(30): element = self._find_element(process, element_id) # Perform action based on permission tier if action == 'get_value': return self._get_value(element) elif action == 'click': return self._click(element) elif action == 'send_keys': return self._send_keys(element, kwargs.get('keys', '')) def _find_element(self, process: str, element_id: str): """Find element with caching and validation.""" root = self.uia.GetRootElement() # Implementation details... pass ``` ## Pattern: Hierarchical Element Discovery ```python class ElementDiscovery: """Safe hierarchical element discovery.""" def find_element_path(self, path: list[str]) -> 'UIElement': """Find element by path with validation at each level.""" current = self.uia.GetRootElement() for level, identifier in enumerate(path): # Validate identifier if not validate_element_identifier(identifier): raise ValidationError(f"Invalid identifier: {identifier}") # Find child element child = self._find_child(current, identifier) if not child: raise ElementNotFoundError(f"Element not found: {identifier}") # Validate we can access this element if not self._can_access(child): raise SecurityError(f"Access denied to element: {identifier}") current = child return current ``` ## Pattern: Robust Wait Conditions ```python class WaitConditions: """Wait for UI conditions with timeout and safety.""" def wait_for_element( self, condition: callable, timeout: int = 30, poll_interval: float = 0.5 ) -> 'UIElement': """Wait for element matching condition.""" start = time.time() while time.time() - start < timeout: try: element = condition() if element: return element except Exception: pass time.sleep(poll_interval) raise TimeoutError(f"Element not found within {timeout}s") def wait_for_window(self, title: str, timeout: int = 30): """Wait for window to appear.""" return self.wait_for_element(