Browser Max Automation

Browser Max Automation

Browser automation via Playwright MCP, existing-browser CDP, and direct CDP helpers.

When to Use

• ブラウザ自動化、UI 確認、フォーム操作、スクリーンショット取得
• MCP で手順を確立してから Python で一括実行したいとき
• 既存ブラウザのログイン状態を CDP 経由で使いたいとき
• Playwright MCP / connect_over_cdp() が不安定で、raw CDP WebSocket に切り替えたいとき
• モーダルや file chooser など、通常クリックが壊れやすい UI を扱うとき

Not the Best Fit

• API や CLI だけで完結する read/write は、まず該当 domain skill / script を使う
• PowerPoint / Loop / Dynamics 365 の expense entry 画面など専用 skill がある UI は、該当 skill の操作ルールを優先する
• 認証情報、秘密情報、MFA 応答をチャットで受け取らない。必要な入力はブラウザ上でユーザーに処理してもらう

Choose Mode First

既存ブラウザ CDP の起動、profile 確認、port drift、認証 URL encode は references/instructions/cdp-existing-browser.md を参照する。 直接 WebSocket CDP の起動フラグ、websocket-client 接続、SPA hash navigation、virtual scroll 操作は references/instructions/cdp-direct-websocket.instructions.md を参照する。

Quick Reference

Core Loop

1. browser_navigate(url)
2. browser_snapshot で ref を取る
3. browser_click / browser_type で操作する
4. browser_snapshot or screenshot で結果確認する

UI verification では、操作前に期待する state と確認方法を決める。成功 toast やボタン押下だけを成功判定にせず、DOM、URL、永続化された一覧行、API の read 結果、または screenshot / trace などの証跡で確認する。

保存・提出系 UI では、API と DOM の状態が一時的にずれることがある。API が stale / capture failure を返しても、画面上の cell / row の aria-label や status text が PENDING APPROVAL / SUBMITTED 等を示しているなら、その DOM status も正本候補として扱う。API だけで「未提出」「Draft のまま」と断定しない。

Decision Patterns

MCP で続けるか、CLI に切り替えるか

• MCP: 1 件ずつ画面を見ながら UI フロー、セレクタ、待機時間を確立する
• Python CLI: 手順が固まった後に N 件一括処理する

切り替え基準は単純で、操作手順をまだ探っているなら MCP、手順が固まったら CLI。

snapshot で取れない要素をどう扱うか

• browser_snapshot で ref が取れるなら通常操作する
• 画面には見えるのに ref が取れないなら browser_evaluate で DOM 直接操作する
• 画面にも見えていないなら、待機かページ再読込を優先する

snapshot で ref 取得
  ├─ 取れた → click / type
  └─ 取れない → screenshot で可視確認
      ├─ 見えている → evaluate で直接操作
      └─ 見えていない → wait / reload

read-only データ抽出を高速化する

一覧表、残高、ステータス確認など、読み取りだけ の fallback では、巨大な browser_snapshot を何度も解析しない。手順が分かっている画面は browser_evaluate で DOM から必要な行だけを JSON 化して返す。

• ページ到達やログイン状態の判定: browser_snapshot で証跡を残す
• 表・明細・残高などの定型抽出: browser_evaluate で document.querySelectorAll("tr") や document.body.innerText を処理して JSON を返す
• 不可逆操作の直前・直後: snapshot または screenshot を残す

また、helper / probe / cache の JSON artifact を一次ソースにする場合は、内容を見る前に timestamp や対象日を確認する。日付が現在の実行日と合わない artifact は stale とみなし、ok や fast_path_ok が true でも確定情報として使わない。

unsaved editor / draft タブを壊さない

Qiita や CMS の draft editor のように、未保存変更を持つタブへそのまま別 URL を開かせると、beforeunload dialog が出て upload や遷移が壊れることがある。

• 既に目的の editor / draft タブが開いているなら、そのタブを優先して再利用する
• 新しい draft が必要でも、既存タブを別 URL へ飛ばさず、新しいタブを開く
• dialog.accept() で無理に吸収する設計を通常フローにしない。beforeunload は race で失敗しやすい
• file upload の前に、現在タブが本当に editor 本体かを URL と title で確認する
• dirty な form / SPA では page.reload()、location.reload()、API 捕捉目的の reload trigger を使わない。まず保存・キャンセル・画面上の status 読み取りで clean にする。reload 確認や unsaved alert が出たら、追加自動化を止めて手動 Cancel / Stay を優先する

このパターンは、Qiita に限らず「未保存フォームを持つ管理画面」全般で効く。

Azure Portal iframe / OOPIF / trusted event の注意は references/instructions/azure-portal.md を参照する。 Angular Material / mat-select / cdk-overlay / disabled save の注意は references/instructions/angular-material.md を参照する。

iframe と force click

• iframe が多段なら contentFrame() を順に辿る
• SVG オーバーレイなどで塞がれているだけなら force: true を検討する
• ただし force: true は最後の手段で、まず要素の実在確認と可視確認を優先する

file chooser が残ったとき

• browser_file_upload(paths=[]) で空送信して閉じる
• ダメなら別ページへ移動する
• CDP 競合が疑わしいなら Python プロセスや MCP 接続を整理する

hidden file input、evaluate + fetch、API write の UI fallback は references/instructions/ui-fallbacks.md を参照する。

Safety Rules

CDP 排他制御

MCP Playwright と Python スクリプトは 同じ CDP ポートへ同時接続しない。 とくに Python 側も connect_over_cdp() を使う場合、MCP と Playwright セッションが二重になり、ページ操作が競合して「遷移先が想定外」「フォーム送信が効かない」等の不定失敗が起きる。

raw WebSocket を使う場合は、CDP command id で応答をフィルタし、Runtime.enable / Page.enable など必要な domain を先に有効化する。詳細は references/instructions/cdp-direct-websocket.instructions.md を参照する。

CDP recovery、blocking dialog、context/page selection は references/instructions/cdp-recovery-and-context.md を参照する。

破綻しやすい場面

• modal overlay が snapshot に出ない
• file chooser が残って後続操作を塞ぐ
• CDP 二重接続で入力先が混線する
• CDP の別 context/page を使い、未ログイン画面や別アカウントを操作してしまう
• 「見えているがクリックできない」状態を無理に通常 click で押し切る

Subprocess + CDP Stability (Windows)

Windows の PIPE デッドロック、VS Code terminal の SIGINT、JSON status artifact、taskkill tree kill は references/instructions/windows-subprocess-cdp.md を参照する。

Reference Map

Done Criteria

• MCP または CDP 設定が完了している
• 対象ページまで安定して到達できる
• 目的の操作が完了している
• 成功判定を toast だけに頼らず、DOM / URL / API read / screenshot / status artifact のいずれかで確認している
• modal / file chooser / iframe / CDP context のどこで詰まるか説明できる
• 一括処理が必要なら MCP から CLI / API helper へ切り替える判断ができている

Angular Material Automation Notes

Use this for Angular Material forms using mat-select, mat-dialog, cdk-overlay, or reactive form validation.

mat-select

• Do not use native input value setters for mat-select; they do not update Angular form state.
• Click .mat-select-trigger, choose a visible [role="option"], then verify the form state.
• For similar labels, prefer exact text match. Delivery and (ISD only) Delivery are easy to confuse with partial matching.

const options = [...document.querySelectorAll('[role="option"]')].filter(option => option.offsetParent);
const exact = options.find(option => option.innerText.trim() === 'Delivery');
if (exact) exact.click();

Overlays and Dialogs

• cdk-global-overlay-wrapper can remain after selection and block later clicks. Send Escape or click the next trigger directly.
• Scope all form queries to the active visible overlay/dialog. A broad document.querySelector('form') can grab an old dialog.

const overlays = [...document.querySelectorAll('.cdk-overlay-pane, mat-dialog-container')]
  .filter(dialog => dialog.offsetParent);
const activeForm = overlays[overlays.length - 1]?.querySelector('form');

Save Buttons

• A button can be enabled while the form silently fails validation. Verify persisted state after clicking.
• Before bulk page.evaluate actions, reset date/tab/filter selection and close leftover overlays.

Azure Portal Automation Notes

Azure Portal is a hash-routed app with iframe/OOPIF content. Use these notes when deep links or blade operations behave differently from normal pages.

Deep Links

• If a new tab opens a Portal deep link and falls back to login, reuse an existing logged-in Portal tab first.
• Verify arrival with URL, title, and visible body text; URL alone is not enough.
• Subview URLs can land on overview or spin indefinitely. Capture a screenshot before switching paths.

iframe / OOPIF

• Portal content often renders inside sandbox-*.reactblade.portal.azure.net iframes.
• If page.evaluate() only sees the outer frame, search page.frames() and evaluate inside the content frame.
• If Playwright frames do not expose it, CDP Target.getTargets() may show an OOPIF target. Attach with Target.attachToTarget(flatten=true) when needed.

const contentFrame = page.frames().find(frame => frame.url().includes('reactblade.portal.azure.net'));
if (contentFrame) {
  const text = await contentFrame.evaluate(() => document.body.innerText);
}

Trusted Event Boundary

• Some openBlade() transitions require a trusted user event.
• If iframe text and handlers are visible but blade open does not fire from JS, switch only that final action to a real click/manual operation.

Direct CDP WebSocket Automation

Use this reference when Playwright MCP or Playwright connect_over_cdp() is unstable, when an existing Edge/Chrome session must be reused, or when direct Chrome DevTools Protocol (CDP) commands are safer than browser-level automation.

When to Prefer Direct WebSocket CDP

• Existing browser session must be reused without creating another Playwright session.
• Playwright connect_over_cdp() fails because of extension service worker targets or browser-context assertions.
• MCP and Python automation would otherwise compete for the same CDP endpoint.
• The task needs low-level Runtime.evaluate, Page.captureScreenshot, or hash-based SPA navigation.

Browser Launch Flags

Launch Edge or Chrome with a dedicated debugging port.

Start-Process 'C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe' `
  -ArgumentList '--remote-debugging-port=9223', '--remote-allow-origins=*', '--restore-last-session'

If you need --profile-directory=Profile 2 or any other argument whose value contains spaces, pass that flag as one quoted argument. Otherwise PowerShell can split it and launch the wrong profile.

Start-Process 'C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe' `
  -ArgumentList '--remote-debugging-port=9223', '--remote-allow-origins=*', '"--profile-directory=Profile 2"'

Rules:

• --remote-debugging-port=<port> exposes the CDP endpoint.
• --remote-allow-origins=* avoids WebSocket 403 Forbidden in environments that enforce origin checks.
• --restore-last-session is useful when the existing tab/session matters.
• If only one browser profile has the right login state, keep the default browser user data and pin --profile-directory=<known profile> instead of creating an ad-hoc --user-data-dir.
• Use a separate port when Playwright MCP already owns another CDP endpoint.

Direct WebSocket Connection

Use websocket-client for raw CDP access. Avoid adding a second Playwright CDP session when MCP is already attached.

import json
import urllib.request
import websocket

# List page targets.
tabs = json.loads(urllib.request.urlopen('http://localhost:9223/json').read())
page_tabs = [tab for tab in tabs if tab.get('type') == 'page']

target_tab = page_tabs[0]
ws = websocket.create_connection(
    target_tab['webSocketDebuggerUrl'],
    timeout=60,
    suppress_origin=True,
)

If the task depends on an already-open draft/editor tab, do not stop at /json/version. Query /json/list, filter for type == 'page', and pick the tab by URL/title pattern before sending commands.

tabs = json.loads(urllib.request.urlopen('http://localhost:9223/json/list').read())
draft_tabs = [
  tab for tab in tabs
  if tab.get('type') == 'page' and 'qiita.com/drafts/' in (tab.get('url') or '')
]
target_tab = draft_tabs[0]

Command Helper

CDP targets can emit many events. Always wait for the matching command id rather than reading the next message blindly.

import itertools
import json
import time
import websocket

_next_id = itertools.count(1)

def cdp(ws, method, params=None, timeout=30):
    message_id = next(_next_id)
    ws.send(json.dumps({'id': message_id, 'method': method, 'params': params or {}}))
    end = time.time() + timeout
    while time.time() < end:
        ws.settimeout(max(1, end - time.time()))
        try:
            message = json.loads(ws.recv())
            if message.get('id') == message_id:
                return message
        except websocket.WebSocketTimeoutException:
            continue
    raise TimeoutError(f'CDP command timed out: {method}')

Enable required domains before use:

cdp(ws, 'Page.enable')
cdp(ws, 'Runtime.enable')

SPA Navigation

For single-page apps, full Page.navigate can reset client state or trigger a full reload. Prefer in-app navigation when the app supports hash routing.

cdp(ws, 'Runtime.evaluate', {
    'expression': 'window.location.hash = "#/target-route"',
    'returnByValue': True,
})

Some SPAs (e.g., Angular with HashLocationStrategy) do not respond to location.hash assignment alone. If the route change does not take effect, use full location.href assignment instead:

cdp(ws, 'Runtime.evaluate', {
    'expression': 'location.href = "https://app.example.com/#/target-route"',
    'returnByValue': True,
})

Never use Page.navigate for hash-based SPA routes — the SPA router will not fire, and the page may reload to a default/home route.

Do not hammer an authenticated SPA with repeated Page.navigate / Page.reload. Rapid back-to-back reloads can corrupt the session/auth token and surface errors like "Authorization failed" or "Unable to retrieve your profile". Keep it to one navigate, then wait, then confirm render with document.body.innerText.length before the next action. If auth errors appear, stop automating and let the session settle (it usually self-heals) instead of reloading again.

Network Capture to Replicate an API Request

When a UI action persists data through an internal API and you want to drive that API directly (faster and more reliable than clicking), capture the real request first, then replay its shape with urllib / requests.

cdp(ws, 'Network.enable')
# Perform the UI action once (click Save, etc.), then read captured events.
# Filter requestWillBeSent for the target URL + POST/PUT/PATCH,
# then pull the body with getRequestPostData by requestId.
post_data = cdp(ws, 'Network.getRequestPostData', {'requestId': rid})['result']['postData']

Rules:

• Match the captured method, URL path, headers, and exact body shape. Server contracts are picky: a .NET [FromBody] List<T> endpoint wants a bare JSON array [payload], not {"wrapperName": [payload]} — the wrong shape returns 400 Mandatory parameter <name> not provided.
• A 201 with the expected status field (e.g. laborStatus: "Draft") plus a re-read of the resource is the real success signal, not the POST returning without exception.
• If requestWillBeSent carries no body, call getRequestPostData with the requestId; large bodies are not inlined.

Screenshot and Text Extraction

import base64

screenshot = cdp(ws, 'Page.captureScreenshot', {'format': 'png'})
image_bytes = base64.b64decode(screenshot['result']['data'])

text = cdp(ws, 'Runtime.evaluate', {
    'expression': 'document.body.innerText.substring(0, 10000)',
    'returnByValue': True,
})['result']['result']['value']

Hidden File Input Upload

For editors that hide input[type=file] behind custom buttons, raw CDP can upload without Playwright connect_over_cdp() or a visible file chooser.

cdp(ws, 'DOM.enable')
cdp(ws, 'Runtime.enable')

root = cdp(ws, 'DOM.getDocument', {'depth': -1, 'pierce': True})['result']['root']
node_id = cdp(ws, 'DOM.querySelector', {
  'nodeId': root['nodeId'],
  'selector': 'input[type="file"]',
})['result']['nodeId']

before_urls = cdp(ws, 'Runtime.evaluate', {
  'expression': '(document.documentElement.outerHTML.match(/https://qiita-image-store\\.s3[^"\'\\s)]+/g) || [])',
  'returnByValue': True,
})['result']['result']['value']

cdp(ws, 'DOM.setFileInputFiles', {
  'nodeId': node_id,
  'files': [image_path],  # absolute local path resolved by the caller
})
cdp(ws, 'Runtime.evaluate', {
  'expression': '''(() => {
    const input = document.querySelector('input[type="file"]');
    input.dispatchEvent(new Event('input', { bubbles: true }));
    input.dispatchEvent(new Event('change', { bubbles: true }));
  })()''',
  'awaitPromise': True,
})

Rules:

• Reuse the already-open draft/editor tab; do not navigate away from an unsaved page.
• Prefer HTML-diff URL detection after upload instead of assuming the last image on the page is the new one.
• Use this fallback when Playwright connect_over_cdp() reaches <ws connected> and then times out, or when MCP cannot drive the file chooser cleanly.
• If upload still times out on an existing draft/editor tab, open a fresh draft (/drafts/new) in a separate tab and retry there. Some long-lived unsaved drafts do not trigger upload reliably even when DOM.setFileInputFiles succeeds.

Azure Portal OOPIF and Trusted Event Notes

Azure Portal often renders blade content inside sandbox-*.reactblade.portal.azure.net iframe targets. In CDP, these can appear as separate type=iframe targets.

targets = cdp(browser_ws, 'Target.getTargets')['result']['targetInfos']
portal_iframes = [
  t for t in targets
  if t.get('type') == 'iframe' and 'reactblade.portal.azure.net' in (t.get('url') or '')
]

If outer-frame DOM inspection cannot see the content, attach directly to the iframe target:

attach = cdp(browser_ws, 'Target.attachToTarget', {
  'targetId': iframe_target_id,
  'flatten': True,
})
session_id = attach['result']['sessionId']

cdp(browser_ws, 'Runtime.enable', session_id=session_id)

Important:

• Some Azure Portal actions call internal SDK methods such as openBlade().
• You may be able to inspect React handlers or invoke onClick, but blade navigation can still fail because the Portal expects a trusted user event.
• When iframe text is visible but programmatic click / handler invocation does not navigate, treat this as a Portal constraint and switch the last click to a human.

Session Extraction and Headless Handoff

If the browser already holds the valid login state, prefer extracting session material at runtime and moving the bulk operation out of visible UI flows.

• Use CDP to read cookies, local storage, CSRF tokens, or in-page bootstrap state from the live browser session.
• Pass the extracted values directly to a headless HTTP/API helper in the same run.
• Keep the browser UI for login, target verification, and before/after evidence only.
• Do not persist tokens, cookies, or auth headers to tracked files.

Virtual Scroll and Modal Patterns

For Angular/Material or other virtual-scroll UIs, combine scroll, element detection, click, and post-click polling into one async Runtime.evaluate call. Splitting these steps across multiple CDP calls can lose DOM references.

(async () => {
  const findRows = (scope, targetText) =>
    [...scope.querySelectorAll("*")].filter(
      (el) =>
        (el.innerText || "").includes(targetText) &&
        el.querySelectorAll("*").length < 30,
    );

  const pickClickable = (rows) => {
    const candidates = [];
    for (const row of rows) {
      let node = row;
      for (let depth = 0; depth < 8; depth++) {
        if (!node || !node.offsetParent) break;
        if (getComputedStyle(node).cursor === "pointer") candidates.push(node);
        node = node.parentElement;
      }
    }
    return candidates[0] || rows[rows.length - 1];
  };

  const scope = document.body;
  const targetText = "Target text";
  let rows = findRows(scope, targetText);

  if (rows.length === 0) {
    const scroller = scope.querySelector('[style*="overflow-y"]') || scope;
    for (let step = 0; step <= 30; step++) {
      scroller.scrollTop = (scroller.scrollHeight * step) / 30;
      await new Promise((resolve) => setTimeout(resolve, 180));
      rows = findRows(scope, targetText);
      if (rows.length > 0) break;
    }
  }

  if (rows.length === 0) return "not-found";

  const element = pickClickable(rows);
  element.scrollIntoView({ block: "center" });
  await new Promise((resolve) => setTimeout(resolve, 400));
  element.click();

  for (let i = 0; i < 25; i++) {
    await new Promise((resolve) => setTimeout(resolve, 200));
    if (document.body.innerText.includes("Expected next state")) return "ok";
  }
  return "timeout";
})();

Rules:

• Search ancestors for clickable elements; the deepest text node often has no click handler.
• Prefer getComputedStyle(node).cursor === 'pointer' as one signal, but still verify the next UI state.
• For modal chains, close overlays and reopen the modal from a known state between iterations.
• Before looping by date/week, verify that changing the date actually changes modal content.

Encoding for Windows Python Scripts

When Python scripts print Japanese or emoji in Windows terminals, configure UTF-8 on both sides.

chcp 65001 | Out-Null
$env:PYTHONIOENCODING = 'utf-8'

import sys

if sys.stdout.encoding and sys.stdout.encoding.lower() != 'utf-8':
    sys.stdout.reconfigure(encoding='utf-8', errors='replace')
if sys.stderr.encoding and sys.stderr.encoding.lower() != 'utf-8':
    sys.stderr.reconfigure(encoding='utf-8', errors='replace')

Troubleshooting

Existing Browser CDP

Use this when reusing an already-authenticated browser profile through CDP.

Start and Verify

Start Edge with a debugging port only when no suitable CDP endpoint exists:

Start-Process "C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe" `
  -ArgumentList '--remote-debugging-port=9222', '"--profile-directory=Profile 2"'

Before using an endpoint, verify both the port owner and browser identity:

$conn = Get-NetTCPConnection -LocalPort 9222 -ErrorAction SilentlyContinue | Select-Object -First 1
if ($conn) {
  Get-CimInstance Win32_Process -Filter "ProcessId=$($conn.OwningProcess)" |
    Select-Object ProcessId, Name, CommandLine
}
(Invoke-WebRequest "http://localhost:9222/json/version" -UseBasicParsing).Content |
  ConvertFrom-Json | Select-Object Browser

Rules

• Treat "port is open" and "right profile is logged in" as separate checks.
• Do not trust an endpoint value from an environment variable or previous run without /json/version and process command-line verification.
• If Chrome owns the intended Edge port, use another port or close Chrome before starting Edge.
• If the same Edge user-data-dir already has a CDP port, launch another profile without --remote-debugging-port; the window joins the existing process and remains visible through the existing CDP endpoint.
• If Edge is already running without any debug port, a new --remote-debugging-port launch joins the existing portless process and the port never opens (/json/version keeps failing). Close all Edge processes first, then relaunch with the port. Closing all Edge is destructive (drops every open tab), so confirm with the user before Stop-Process -Name msedge.
• When a helper connects from Node, pass http://127.0.0.1:<port> rather than localhost. localhost can resolve to IPv6 ::1 while the CDP endpoint listens on IPv4, so PowerShell reaches it but Node fetch fails with fetch failed.
• Pass the verified CDP URL explicitly to helpers so later scripts do not re-guess a different endpoint.

Authentication Gotchas

• Prefer headful existing profile + CDP over temporary --user-data-dir for sites that rely on cookies or device auth.
• For /json/new?<url>, URL-encode the full target URL. Unencoded &state=... or callback parameters are parsed by the CDP endpoint and disappear from the site URL.
• Close stale auth tabs before retrying expired OAuth or callback flows.
• When the existing Edge is running without a debug port and killing every Edge process is not acceptable (open tabs, dirty editors, other workflows), do not force-close it. Escape hatch: robocopy the target profile (e.g. Default) to a temporary %TEMP%\edge-cdp-<purpose> and launch a separate instance with --user-data-dir=<tmp> --profile-directory=Default --remote-debugging-port=<new-port>. The original Edge stays untouched. The copied profile may still need a fresh login because some cookies, OAuth refresh tokens, or device-bound credentials do not survive the copy — accept manual re-login as part of the flow. After the session, kill the msedge processes bound to the new port and remove the temporary user-data-dir. To keep the copy small (full profiles are often 1–2 GB), exclude transient caches: robocopy <src>\Default <dst>\Default /E /XJ /XD Cache Cache2 CacheData "Code Cache" GPUCache "Service Worker" Crashpad ShaderCache "Default Cache". Cleanup one-liner: Get-NetTCPConnection -State Listen -LocalPort <port> | %{ Stop-Process -Id $_.OwningProcess -Force }; Remove-Item "$env:TEMP\edge-cdp-<purpose>" -Recurse -Force.

CDP Recovery and Context Selection

Disconnection Recovery

When the browser closes or crashes (Target page, context or browser has been closed):

1. Check the port: Invoke-WebRequest -Uri 'http://localhost:<port>/json/version' -TimeoutSec 5. 2. If refused, restart the browser with the same debugging port and profile/user-data-dir. 3. Confirm /json/version returns Browser. 4. Reconnect MCP: browser_close -> browser_navigate. 5. For authenticated sites, release MCP before running Python login helpers, then reconnect for verification.

Unresponsive but Still Connected

If /json/list works but Runtime.evaluate or Page.enable times out, suspect a JS dialog, beforeunload prompt, reload confirmation, or in-page modal.

• Probe with Runtime.evaluate({expression: '1+1'}) and a short timeout.
• Try Page.handleJavaScriptDialog({accept: false}).
• If no native dialog exists, close in-page overlays with DOM/Escape.
• Browser system dialogs may not be controllable by CDP; ask the user to cancel rather than retrying blindly.

Context / Page Selection

connect_over_cdp() can expose multiple browser contexts and profiles. Never assume contexts[0].pages[0] is the right page.

Safe selection:

1. Iterate all contexts. 2. Rank pages by target domain or management URL. 3. Run preflight for login, authorization, and target screen readiness. 4. Use only the first page that passes preflight. 5. If none pass, return compact JSON with URL, title, and failure reason.

Preflight must confirm target domain, no login redirect, and required controls/API visibility.

UI Fallbacks and Fast Paths

Use these patterns after the normal MCP snapshot/click flow has established the page model.

Hidden File Input Upload

• If connectOverCDP() times out but /json/list exposes a page WebSocket URL, use raw CDP.
• For editors with hidden input[type=file], target the existing editor tab and use DOM.setFileInputFiles.
• Do not navigate an unsaved draft tab to a new URL. Open a separate new tab for a fresh draft if needed.
• Verify upload by checking the new file URL in page HTML or editor text.

evaluate + fetch

When a logged-in session exposes a REST API, prefer page.evaluate(() => fetch(...)) for bulk read/write. It avoids navigation instability and uses existing cookies with credentials: 'same-origin'.

Rules:

• Use UI for login, preflight, and before/after evidence.
• Keep business logic in Python or the main script; let JavaScript execute fetch/write only.
• Complete fetch -> decision -> update -> result return in one evaluation when possible.

Minimal UI Write Fallback

If API write fails with stale state, guardrail refusal, or route mismatch:

1. Confirm the UI save path is stable. 2. Use the shortest path: search -> select row -> required fields -> save. 3. Verify with list/detail/status text or a read API after save. 4. Record state precisely, such as saved in UI / pending submit.

Do not change the business classification just because API automation failed. Change the operation path, then verify the intended destination.

Windows Subprocess and CDP Stability

Use this when running Playwright/CDP automation from a CLI helper on Windows.

PIPE Deadlock

subprocess.Popen(stdout=PIPE) + communicate() can deadlock because Playwright's Node/browser process tree keeps pipe handles open. Redirect stdout/stderr to files and poll the process instead.

with open(stdout_path, "w", encoding="utf-8") as fout, open(stderr_path, "w", encoding="utf-8") as ferr:
    proc = subprocess.Popen(cmd, stdout=fout, stderr=ferr, creationflags=subprocess.CREATE_NEW_PROCESS_GROUP)

Terminal Interference

• VS Code shared terminals can send unexpected SIGINT to long-running helpers. Ignore SIGINT in the runner or launch with Start-Process -Wait.
• Prefer JSON status artifacts over terminal text for completion decisions.

result = json.loads(Path(output_json).read_text(encoding="utf-8"))
if result.get("final_status") != "passed":
    raise RuntimeError(f"runner failed: {result.get('final_status')}")

Cleanup

• CDP helpers often spawn Node/browser child trees. proc.kill() may leave grandchildren.
• Use tree kill only for the owned helper PID: taskkill /F /T /PID <pid>.