Stably Sdk Rules

Stably SDK Rules

Quick Rules

1. Prefer raw Playwright for deterministic actions/assertions (faster + cheaper). 2. Prioritize reliability over cost when Playwright becomes brittle. 3. Use agent.act() for canvas, coordinate-based drag/click, or unstable multi-step flows. 4. Use expect(...).aiAssert() for dynamic visual assertions; keep prompts specific. 5. Use page.extract() / locator.extract() when you need visual-to-data extraction. 6. Use page.getLocatorsByAI() when semantic selectors are hard with standard locators. 7. Use Inbox for OTP/magic-link/verification email flows. 8. All prompts must be self-contained; never rely on implicit previous context. 9. Keep agent.act() tasks small; do loops/calculations/conditionals in code. 10. Use fullPage: true only if content outside viewport matters. 11. Always add .describe("...") to locators for trace readability. 12. For email isolation, use unique Inbox.build({ suffix }) per test and clean up.

Setup (Single Block)

npm install -D @playwright/test @stablyai/playwright-test @stablyai/email
export STABLY_API_KEY=YOUR_KEY
export STABLY_PROJECT_ID=YOUR_PROJECT_ID

import { test, expect } from "@stablyai/playwright-test";
import { Inbox } from "@stablyai/email";

Optional: set API key programmatically.

import { setApiKey } from "@stablyai/playwright-test";
setApiKey("YOUR_KEY");

Core Rules

• Locator rule: every locator interaction should use .describe("...").
• Assertion choice:
• Use Playwright assertions first.
• Use aiAssert for dynamic/visual-heavy checks.
• Interaction choice:
• Use Playwright for deterministic steps.
• Use agent.act for brittle or semantic tasks (especially canvas/coordinates).
• Prompt quality:
• Include explicit target, intent, and constraints.
• Pass cross-step data through variables, not vague references.

Minimal Usage Patterns

aiAssert

await expect(page).aiAssert("Shows revenue trend chart and spotlight card");
await expect(page.locator(".header").describe("Header")).aiAssert("Has nav, avatar, and bell icon");

Use fullPage: true only when assertion needs off-screen content.

extract

const orderId = await page.extract("Extract the order ID from the first row");

With schema:

import { z } from "zod";
const Schema = z.object({ revenue: z.string(), users: z.number() });
const metrics = await page.extract("Get revenue and active users", { schema: Schema });

getLocatorsByAI

Requires Playwright >= 1.54.1.

const { locator, count } = await page.getLocatorsByAI("the login button");
expect(count).toBe(1);
await locator.describe("Login button located by AI").click();

agent.act

await agent.act("Find the first pending order and mark it as shipped", { page });

Good pattern: compute values in code, then pass concrete values into the prompt.

Inbox (Email Isolation)

Install: npm install -D @stablyai/email. Requires STABLY_API_KEY and STABLY_PROJECT_ID env vars (or pass to Inbox.build()).

const inbox = await Inbox.build({ suffix: `test-${Date.now()}` });
// inbox.address → "my-org+test-1706621234567@mail.stably.ai"

await page.getByLabel("Email").describe("Email input").fill(inbox.address);

const email = await inbox.waitForEmail({ subject: "verification", timeoutMs: 60_000 });
const { data: otp } = await inbox.extractFromEmail({
  id: email.id,
  prompt: "Extract the 6-digit OTP code",
});

await inbox.deleteAllEmails();

Inbox.build Options

Always use a unique suffix per test for parallel isolation. The inbox automatically filters out emails received before it was created.

waitForEmail

const email = await inbox.waitForEmail({
  from: "noreply@example.com",    // filter by sender
  subject: "verification",         // contains match by default
  subjectMatch: "exact",           // or "contains" (default)
  timeoutMs: 60_000,               // default: 120000 (2 min)
  pollIntervalMs: 5000,            // default: 3000 (3 sec)
});

Throws EmailTimeoutError if no match arrives within the timeout.

extractFromEmail

Returns { data, reason }. Throws EmailExtractionError on failure.

// String extraction
const { data: otp } = await inbox.extractFromEmail({
  id: email.id,
  prompt: "Extract the 6-digit OTP code",
});

// Structured extraction with Zod schema
import { z } from "zod";
const { data } = await inbox.extractFromEmail({
  id: email.id,
  prompt: "Extract the verification URL and expiration time",
  schema: z.object({ url: z.string().url(), expiresIn: z.string() }),
});

Inbox Properties

listEmails

const { emails, nextCursor } = await inbox.listEmails(options?);

Other Methods

const email = await inbox.getEmail(id);                          // get by ID
await inbox.deleteEmail(email.id);                               // delete single
await inbox.deleteAllEmails();                                   // delete all (this inbox only)

Email Object Properties

Playwright Fixture Pattern

import { test as base } from "@stablyai/playwright-test";
import { Inbox } from "@stablyai/email";

const test = base.extend<{ inbox: Inbox }>({
  inbox: async ({}, use, testInfo) => {
    const inbox = await Inbox.build({ suffix: `test-${testInfo.testId}` });
    await use(inbox);
    await inbox.deleteAllEmails();
  },
});

test("signup flow", async ({ page, inbox }) => {
  await page.fill("#email", inbox.address);
  await page.click("#signup");
  const email = await inbox.waitForEmail({ subject: "Welcome" });
  // ...
});

Finding Your Organization's Email Address

Your email address is visible in the Stably dashboard:

• Settings > Email Inbox: Displays the full address with a copy button
• In code: inbox.address after calling Inbox.build() returns your full address

The pattern is {org-name}@mail.stably.ai. If the user needs to allowlist, they should add mail.stably.ai to their email provider's allowlist.

Direct users to the dashboard Settings > Email Inbox to find their specific address.

Auth Flows (Google)

Use the helper instead of custom popup scripting:

import { authWithGoogle } from "@stablyai/playwright-test/auth";

await authWithGoogle({
  context,
  email: process.env.GOOGLE_AUTH_EMAIL!,
  password: process.env.GOOGLE_AUTH_PASSWORD!,
  otpSecret: process.env.GOOGLE_AUTH_OTP_SECRET!,
});

Required env vars:

• GOOGLE_AUTH_EMAIL
• GOOGLE_AUTH_PASSWORD
• GOOGLE_AUTH_OTP_SECRET

Use a dedicated test Google account only.

Troubleshooting (Short)

• aiAssert is slow/flaky: scope to a locator, tighten prompt, avoid unnecessary fullPage: true.
• agent.act fails: split into smaller tasks, pass explicit constraints, raise maxCycles only when needed.
• Email timeout: verify subject/from filter and use unique inbox suffixes.

Full References

• Stably docs: https://docs.stably.ai
• SDK setup skill: skills/stably-sdk-setup/SKILL.md
• Package docs: https://www.npmjs.com/package/@stablyai/playwright-test
• Email package docs: https://www.npmjs.com/package/@stablyai/email
• Local overview: skills/stably-sdk-rules/README.md

stably-sdk-rules

AI rules for writing tests with Stably Playwright SDK.

Installation

npx skills add stablyai/agent-skills --skill stably-sdk-rules

When to Use

• Writing or modifying Playwright tests with Stably AI features
• Using aiAssert, agent.act(), page.extract(), or page.getLocatorsByAI()
• Testing email-dependent flows (OTP, magic links, confirmations) with @stablyai/email
• Deciding between Playwright vs Stably SDK methods

Key Features

AI Assertions

await expect(page).aiAssert(
  "Shows revenue trend chart and spotlight card"
);

AI Extraction

const data = await page.extract("Get the order ID");

AI Agent

await agent.act("Complete the checkout process", { page });

AI Locator Finding

const { locator, count } = await page.getLocatorsByAI("the login button");
await locator.click();

Email Inbox Testing

import { Inbox } from "@stablyai/email";

const inbox = await Inbox.build({ suffix: `test-${Date.now()}` });
const email = await inbox.waitForEmail({ subject: "verification" });
const { data: otp } = await inbox.extractFromEmail({
  id: email.id,
  prompt: "Extract the 6-digit OTP code",
});

Related

• stably-sdk-setup - Setup assistant for Stably SDK