Vitest Midscene E2e

Vitest Midscene E2E

Modules

Supported platforms:

• Web — WebTest (Playwright Chromium): ctx.agent + ctx.page
• Android — AndroidTest (ADB + scrcpy): ctx.agent only
• iOS — IOSTest (WebDriverAgent): ctx.agent only

Workflow

Step 1: Clone boilerplate & ensure project ready

bash scripts/clone-boilerplate.sh

The boilerplate at ~/.midscene/boilerplate/vitest-all-platforms-demo/ is the canonical reference for project structure, configs, platform context classes, and test conventions. Compare the current project against it. If anything is missing, ask the user which platform(s) they need (Web / Android / iOS), then fill in what's missing using the boilerplate as the target state. Only include files for the requested platform(s). Do NOT overwrite existing configs or files. Copy .env.example from the boilerplate as .env if it doesn't exist, and prompt the user to fill in the env vars.

Step 2: Read the Midscene Agent API section below before writing tests

It contains mandatory rules for using aiAct — the primary API for all UI operations. Do NOT skip this step.

Step 3: Create, update, or run tests

Use the boilerplate's e2e/ directory and src/context/ as reference for patterns and conventions. Before running tests, ensure dependencies are installed and .env is configured. When debugging failures, check troubleshooting.md.

Midscene Agent API

ctx.agent is a platform-specific agent instance. All methods return Promises.

• Web: PlaywrightAgent from @midscene/web/playwright
• Android: AndroidAgent from @midscene/android
• iOS: IOSAgent from @midscene/ios

All three agents share the same AI methods below.

Mandatory Rule: Use aiAct for User-Described Steps

When the user describes a UI action or state confirmation in natural language, you MUST use `aiAct` to implement it. Do NOT decompose user instructions into aiTap/aiInput/aiAssert or other fine-grained APIs. Pass the user's intent directly to aiAct and let Midscene's AI handle the planning and execution.

// User says: "type iPhone in the search box and click search"

// WRONG — manually decomposing into fine-grained APIs
await ctx.agent.aiInput('search box', { value: 'iPhone' });
await ctx.agent.aiTap('search button');

// CORRECT — pass intent directly to aiAct
await ctx.agent.aiAct('type "iPhone" in the search box, then click the search button');

Assertions, data extraction, and waiting should also be done via aiAct — it handles all of these. Do NOT use aiAssert, aiQuery, aiWaitFor, aiTap, or aiInput separately.

aiAct(taskPrompt, opt?) — Primary API

`aiAct` is the primary API for all UI operations and state confirmations. It accepts natural language instructions and autonomously plans and executes multi-step interactions.

// UI operations
await ctx.agent.aiAct('type "iPhone" in the search box, then click the search button');
await ctx.agent.aiAct('hover over the user avatar in the top right');

// State confirmations / assertions — also use aiAct
await ctx.agent.aiAct('verify the page shows "Login successful"');
await ctx.agent.aiAct('verify the error message is visible');

Phase splitting: If the task prompt is too long or covers multiple distinct stages, split it into separate aiAct calls — one per phase. Each phase should be a self-contained logical step, and all phases combined must match the user's original intent.

// Incorrect — prompt spans multiple pages and too many steps, AI may lose context mid-way
await ctx.agent.aiAct('click the settings button in the top nav, go to settings page, find personal info and click into it, change email to "test@example.com", change phone to "13800000000", click save, wait for success');

// Correct — split by page/stage boundary, each phase stays within one logical context
await ctx.agent.aiAct('click the settings button in the top nav, go to settings page, find personal info and click into it');
await ctx.agent.aiAct('change email to "test@example.com", change phone to "13800000000", click save');
await ctx.agent.aiAct('verify the save success message appears');

aiAction is deprecated. Use aiAct or ai instead.

Common Mistakes

• Vague locators — 'button' is ambiguous; use 'the blue "Submit" button at the top of the page'
• Deprecated `aiAction` — use aiAct instead
• Ambiguous multi-element targets — specify row/position: 'the delete button in the first product row'

Agent Configuration — aiActionContext

aiActionContext is a system prompt string appended to all AI actions performed by the agent. Use it to define the AI's role and expertise.

// Set via agentOptions in setup()
const ctx = WebTest.setup('https://example.com', {
  agentOptions: {
    aiActionContext: 'You are a Web UI testing expert.',
  },
});

Good examples:

• 'You are a Web UI testing expert.'
• 'You are an Android app testing expert who is familiar with Chinese UI.'

Bad examples:

• 'Click the login button.' — specific actions belong in aiAct(), not aiActionContext
• 'The page is in Chinese.' — this is page description, not a system prompt

How to Look Up More

1. In node_modules/@midscene/web, node_modules/@midscene/android, and node_modules/@midscene/ios, find the type definitions for the agent classes 2. If types are not enough, follow the source references in the .d.ts files to read the implementation code in node_modules 3. Download https://midscenejs.com/llms.txt, then use grep to search for the API or concept you need (the file is large, do not read it in full)

Troubleshooting

Timeout / Assertion Too Early

AI-driven tests are slower than selector-based tests. Default timeout is 180s per test. Most timeout and premature assertion failures are caused by not waiting for the page/app to be ready.

Include waiting in the aiAct prompt itself, so Midscene waits for the page/app to be ready before asserting:

// Incorrect — assertion fires before page updates
await ctx.agent.aiAct('click the submit button');
await ctx.agent.aiAct('verify the page shows "Submission successful"');

// Correct — include wait in the action prompt
await ctx.agent.aiAct('click the submit button and wait until the submission completes');
await ctx.agent.aiAct('verify the page shows "Submission successful"');

---

Element Not Found / Wrong Element

Vague or ambiguous descriptions cause AI to match the wrong element or fail entirely.

// Incorrect — too vague, multiple buttons may exist
await ctx.agent.aiAct('click the button');
await ctx.agent.aiAct('click the delete button');

// Correct — include position, context, or visual traits
await ctx.agent.aiAct('click the blue "Submit" button at the top of the page');
await ctx.agent.aiAct('click the delete button in the first product row');

Tips for precise descriptions:

• Include position: top / bottom / first row / left side
• Include text content: "Submit" / "Confirm" / "Delete"
• Include visual traits: blue / large / with icon

---

Device Connection

Android — ensure a device is connected and ADB is configured:

adb devices  # Should list your device

If using a remote device, set MIDSCENE_ADB_REMOTE_HOST and MIDSCENE_ADB_REMOTE_PORT in .env.

iOS — ensure WebDriverAgent is running on the target device/simulator:

curl http://localhost:8100/status

Override the port via IOSTest.setup(uri, { deviceOptions: { wdaPort: 8100 } }).

---

Debugging Tips

• Headed mode (Web): Use WebTest.setup(url, { headless: false }) to see the browser during test execution
• Midscene report: Check the generated report for screenshots and AI decision traces