Mastra Smoke Test

Mastra Smoke Test

Comprehensive smoke testing for Mastra projects.

Release smoke workflows

Use progressive disclosure: stay in this file until the workflow branches, then read only the reference for the branch you are on. references/release-smoke.md is a short index if you need the full map.

Alpha release branch point

Before alpha smoke testing, identify the alpha versioning PR state. Prefer the standard Changesets release branch:

gh pr view changeset-release/main \
  --json number,title,state,url,headRefName,baseRefName,isDraft,mergeable,reviewDecision,updatedAt,mergedAt,mergeCommit

Expected shape:

title: chore: version packages (alpha)
head: changeset-release/main
base: main

If that branch lookup fails, search open and recently merged PRs:

gh pr list --state open --search 'version packages alpha in:title' --limit 20
gh pr list --state merged --search 'version packages alpha in:title' --limit 20

Then branch:

• If the versioning PR is open, read references/alpha-versioning-pr.md.
• If the versioning PR is merged, read references/alpha-publish.md.
• If no versioning PR exists, report that and wait for the scheduled alpha versioning flow or user direction.

Do not create the alpha smoke-test project until the automatic alpha publish workflow has completed and the intended packages are installable.

Stable release branch point

If the user is running the stable/full release workflow, read references/stable-release-smoke.md. If that workflow fails after some packages publish, switch to references/stable-partial-publish-recovery.md.

Scope and targeted checks

After the release package is published and before running smoke tests, read references/release-scope-discovery.md to identify changed features. Use the default generated project for the baseline checklist, then add targeted checks for changed features the generated project does not exercise.

When scope discovery identifies a branch:

• For general changed-feature coverage, read references/targeted-feature-smoke.md.
• For storage/provider schema or migration changes, read references/storage-provider-migration-smoke.md.

⚠️ Mandatory Test Checklist

Use `task_write` to track progress. Run ALL tests unless --test specifies otherwise.

Do not skip tests unless you hit an actual blocker. "Seemed complex" or "wasn't sure" are not valid reasons. Attempt everything - only stop a test when you literally cannot proceed. Report what you tried and what blocked you.

Execution Flow

1. Read the reference file for each test you're about to run 2. Execute the steps in that reference file 3. Mark the test complete before moving to the next

Partial Testing (--test)

If --test is provided:

1. Always run Setup (step 1) 2. Run only the specified test(s) 3. Skip other tests

Example: --test agents,traces → Run steps 1, 2, and 5 only.

Local Studio Browser Smoke

For local release smoke tests, do both API/curl checks and a Studio browser pass unless --skip-browser is explicitly requested or browser access is genuinely blocked. API checks prove runtime endpoints work; browser checks prove the Playground/Studio UI can load, submit forms, and display results.

Before opening the browser:

1. Confirm the dev server is alive on the expected port:

   curl -s -o /dev/null -w '%{http_code}\n' http://localhost:4111
   lsof -i :4111 || true

2. If the process died, restart it from the generated project and wait for readiness:

   cd "$SMOKE_DIR/smoke-project"
   pnpm run dev > "$SMOKE_DIR/logs/dev-server-browser.log" 2>&1 &

   for i in {1..60}; do
     code=$(curl -s -o /dev/null -w '%{http_code}' http://localhost:4111 || true)
     [ "$code" = 200 ] && break
     sleep 1
   done

3. Use browser tools to navigate to http://localhost:4111. If networkidle times out but domcontentloaded succeeds and the UI is usable, continue and note the timeout.

Recommended browser task list:

1. Verify Studio shell loads
2. Smoke test agent chat UI
3. Smoke test tools UI
4. Smoke test workflows UI
5. Smoke test observability, scorers, and MCP pages
6. Report browser smoke results

Run these page checks:

If a browser interaction does not expose enough text in the accessibility snapshot, inspect document.body.innerText or take a screenshot, then record the visible evidence. Do not rely only on API output for browser smoke.

Append browser results to $SMOKE_DIR/smoke-report.md with a separate section, for example:

## Studio Browser Smoke Results

| Area         | Result | Evidence                                                          |
| ------------ | ------ | ----------------------------------------------------------------- |
| Studio shell | PASS   | Browser loaded localhost:4111; sidebar/nav visible; version shown |
| Agents UI    | PASS   | Weather Agent chat returned Tokyo weather and displayed tool call |
| Tools UI     | PASS   | get-weather form returned Paris weather JSON                      |
| Workflows UI | PASS   | weather-workflow Berlin run completed as success                  |
| Traces UI    | PASS   | Recent agent/workflow traces listed                               |
| Scorers UI   | PASS   | Expected scorers listed                                           |
| MCP UI       | PASS   | Expected empty MCP state shown                                    |

Call out separately whether browser smoke was local Studio only or cloud Studio/deployed server.

---

Usage

# Full smoke test
smoke test --env local --existing-project ~/my-app
smoke test --env staging -d ~/projects -n test-app

# Partial testing
smoke test --env local --existing-project ~/my-app --test agents
smoke test --env production --existing-project ~/my-app --test studio,server,traces

# Multi-environment: same project, different targets
smoke test --env staging --existing-project ~/my-app   # Uses .mastra-project-staging.json
smoke test --env production --existing-project ~/my-app # Uses .mastra-project.json

Multi-Environment Support

One project can target all environments using separate config files:

See references/tests/setup.md for setup details.

Parameters

\* Either --directory + --name OR --existing-project required

Test Options (--test)

Prerequisites

All environments:

• Node.js + package manager
• LLM API key in env or .env

Local (`--env local`):

• Browser tools enabled (/browser on)

Cloud (`--env staging/production`):

• Mastra platform account

Quick Start Flow

1. Setup      → Read references/tests/setup.md, create/verify project
2. Start      → `pnpm run dev` (local) or deploy (cloud)
3. Test       → For each test, read its reference file and execute
4. Verify     → Check all items in reference file's checklist
5. Report     → Summarize pass/fail for each test

References

Platform Dashboards

• Production: https://projects.mastra.ai
• Staging: https://projects.staging.mastra.ai

For Gateway API testing (memory, threads, BYOK via gateway), use platform-smoke-test.

Result Reporting

After testing, provide:

## Smoke Test Results

**Environment**: local/staging/production
**Project**: <name>

| Test   | Status | Notes |
| ------ | ------ | ----- |
| Setup  | ✅/❌  |       |
| Agents | ✅/❌  |       |
| Tools  | ✅/❌  |       |
| ...    |        |       |

**Issues Found**: (list any)
**Warnings**: (list any deploy/runtime warnings)
**Skipped Tests**: (list with reason - e.g., "Server Deploy - not applicable in local environment")

Alpha Publish Verification

Use this after the alpha versioning PR has merged.

Confirm the automatic alpha publish workflow

Merging the alpha versioning PR to main automatically kicks off the Publish to npm workflow on a push event. Do not advise manually starting the alpha publish workflow.

Check for the automatic run:

gh run list --workflow "Publish to npm" --branch main --limit 5

Inspect the newest run. The alpha path should be the prerelease job, with snapshot, stable, and enter_prerelease skipped:

gh run view <run-id> --json name,event,status,conclusion,workflowName,headBranch,headSha,jobs,url --jq .

Watch it:

gh run watch <run-id>

Offer to open the prerelease run in the user's browser when helpful:

gh run view <run-id> --web

Use gh run view --web instead of browser automation because it opens the page in the user's normal browser/session.

If no automatic publish run starts after the merge, report that as a release automation issue and ask the user before taking any recovery action.

Confirm alpha is published

Before smoke testing, confirm the alpha package is installable:

npm view @mastra/core@alpha version
npm view mastra@alpha version
npm view create-mastra@alpha version

Only then create the smoke-test project with the alpha tag/version.

Next step

After alpha packages are published, continue with release scope discovery in references/release-scope-discovery.md, then run the mandatory local checklist from SKILL.md plus any targeted checks.

Alpha Versioning PR Readiness

Use this only when the alpha versioning PR is still open.

Open the PR for the user

Offer to open the open versioning PR in the user's browser when helpful:

gh pr view <pr-number> --web

Use gh pr view --web instead of browser automation because it opens the page in the user's normal browser/session.

Check readiness

gh pr view <pr-number> --json number,title,isDraft,mergeable,reviewDecision,url
gh pr checks <pr-number> --watch=false

If checks are still running, wait:

gh pr checks <pr-number> --watch --interval 30

Spot check the versioning diff

Before telling the user it is ready to merge, spot check the versioning diff yourself and advise the user to spot check it too. Prefer the helper script:

.claude/skills/mastra-smoke-test/scripts/check-versioning-pr.sh <pr-number> --workspace "$SMOKE_DIR"

Or inspect manually:

gh pr diff <pr-number> --name-only
gh pr view <pr-number> --json files --jq '.files[].path' \
  | rg '(package\.json|CHANGELOG\.md|\.changeset/)'
gh pr diff <pr-number>

Check:

• package versions look intentional
• there are no unintended major version bumps or breaking-change releases
• changelog entries match the PRs expected in the alpha
• CI is green and the PR is not a draft

Summarize your spot-check findings for the user before they approve/merge.

Merge guidance

The user must review, approve, and merge the versioning PR. The agent may check readiness and advise, but should not merge the PR without explicit user instruction.

If the PR is ready, tell the user to approve and merge it in GitHub, or ask whether they want you to merge it. If branch protection rejects the merge because review is required, stop and ask for the required approval.

After the PR merges, continue with references/alpha-publish.md.

Architecture Overview

For detailed internal architecture documentation, see the Notion page: [Architecture Overview](https://www.notion.so/33bebffbc9f8816280b8eb09c72fed4f)

High-Level Summary

When you deploy a Studio or Server to Mastra platform:

1. Deploy → CLI sends your project to platform services 2. Token → Platform signs a JWT for your deployment 3. Traces → Your deployment sends traces using that token 4. Storage → Traces are stored and queryable in Studio UI

Key Points for Testing

• Both Studio and Server deploys generate traces
• Traces should appear in Studio's Observability tab
• If traces don't appear, check deploy logs for warnings

Troubleshooting

If you have GCP access and need to debug infrastructure issues, see the internal docs in Notion.

Cloud Advanced Testing

Advanced test flows for --env staging and --env production.

Note: For account creation, team invites, and RBAC testing, use the platform-smoke-test skill.

Those features are part of the platform dashboard (projects.mastra.ai / gateway.mastra.ai), not deployed Studio/Server projects.

BYOK Testing (--byok)

Tests bring-your-own-key functionality for deployed servers.

This tests passing your own API key to your deployed Mastra server (not the Gateway API).

Via HTTP Header

# Test with OpenAI key via header
curl -X POST https://<project>.server.mastra.cloud/api/agents/weather-agent/generate \
  -H "Content-Type: application/json" \
  -H "x-openai-api-key: sk-your-openai-key" \
  -d '{"messages": [{"role": "user", "content": "What is the weather in Tokyo?"}]}'

# For staging environment
curl -X POST https://<project>.server.staging.mastra.cloud/api/agents/weather-agent/generate \
  -H "Content-Type: application/json" \
  -H "x-openai-api-key: sk-your-openai-key" \
  -d '{"messages": [{"role": "user", "content": "What is the weather in Tokyo?"}]}'

Supported headers:

• x-openai-api-key - OpenAI
• x-anthropic-api-key - Anthropic
• x-google-api-key - Google

Via Project Settings

1. Navigate to deployed Studio → Settings → API Keys 2. Add OpenAI/Anthropic/Google API key 3. Verify agents use the configured key instead of default

Storage Backend Testing (--db)

Tests that the project works with the selected database backend.

LibSQL (Default)

# No additional setup required
# Uses local SQLite file in development

PostgreSQL (--db pg)

Requires DATABASE_URL environment variable:

export DATABASE_URL="postgresql://user:pass@host:5432/db"

Turso (--db turso)

Requires Turso credentials:

export TURSO_DATABASE_URL="libsql://your-db.turso.io"
export TURSO_AUTH_TOKEN="your-token"

Extended Test Verification Checklist

Cloud Deployment Setup

Instructions specific to --env staging and --env production testing.

Prerequisites

• Mastra platform account with deploy access
• pnpx (or npx) available
• For debugging: GCP Console access (see gcp-debugging.md)

Multi-Environment Config

Deploy to staging and production from the same project using separate config files:

Each environment gets its own project ID, so they don't interfere.

Environment Setup

Set the platform URL based on target environment:

# For production
export MASTRA_PLATFORM_API_URL=https://platform.mastra.ai

# For staging
export MASTRA_PLATFORM_API_URL=https://platform.staging.mastra.ai

LLM API Key

Ensure .env has the required API key:

Authenticate with Platform

Check Existing Credentials

Before triggering a browser login, check if credentials exist and are valid:

# Check if credentials file exists
cat ~/.mastra/credentials.json | jq '{email: .user.email, organizationId}'

# Verify token is still valid
TOKEN=$(jq -r '.token' ~/.mastra/credentials.json)
ORG_ID=$(jq -r '.currentOrgId // .organizationId' ~/.mastra/credentials.json)
curl -s "$MASTRA_PLATFORM_API_URL/v1/auth/verify" \
  -H "Authorization: Bearer $TOKEN" \
  -H "x-organization-id: $ORG_ID" | jq '.user.email'

Token Refresh (if expired)

WorkOS tokens expire in 5 minutes, but can be refreshed without re-login:

REFRESH_TOKEN=$(jq -r '.refreshToken' ~/.mastra/credentials.json)
curl -s "$MASTRA_PLATFORM_API_URL/v1/auth/refresh-token" \
  -X POST \
  -H "Content-Type: application/json" \
  -d "{\"refreshToken\": \"$REFRESH_TOKEN\"}"

See references/tests/traces.md for the full get_valid_token helper function.

Login (only if refresh fails)

⚠️ Always warn the user before running this — it opens a browser:

# Logout first for clean state (optional)
pnpx mastra@latest auth logout

# Login to target environment
pnpx mastra@latest auth login

This opens a browser for OAuth. Complete the login flow.

Verify Organization

The browser may default to a different account/org. Always verify after login:

cat ~/.mastra/credentials.json | jq '{email: .user.email, organizationId}'

Deploy Studio

# Production (uses .mastra-project.json by default)
pnpx mastra@latest studio deploy -y

# Staging (specify config file)
pnpx mastra@latest studio deploy --config .mastra-project-staging.json -y

Wait for deployment. Note the URL from output:

• Production: https://<project>.studio.mastra.cloud
• Staging: https://<project>.studio.staging.mastra.cloud

Verify: Open URL, sign in, confirm Studio UI loads.

Deploy Server

# Production
pnpx mastra@latest server deploy -y

# Staging
pnpx mastra@latest server deploy --config .mastra-project-staging.json -y

The -y flag auto-confirms settings.

Note the URL from output:

• Production: https://<project>.server.mastra.cloud
• Staging: https://<project>.server.staging.mastra.cloud

Verify health:

# Staging
curl https://<project>.server.staging.mastra.cloud/health

# Production (no environment subdomain)
curl https://<project>.server.mastra.cloud/health

# Expected: {"success":true}

Test Server API

Use the helper script:

.claude/skills/mastra-smoke-test/scripts/test-server.sh <server-url> [agent-id] [message]

# Examples
.claude/skills/mastra-smoke-test/scripts/test-server.sh https://my-app.server.staging.mastra.cloud
.claude/skills/mastra-smoke-test/scripts/test-server.sh https://my-app.server.mastra.cloud weather-agent "Weather in Tokyo?"

The script:

1. Checks /health endpoint 2. Calls agent's /generate endpoint 3. Parses and displays response 4. Exits with error if checks fail

Verify Server Traces in Studio

Critical step — verifies the full trace pipeline works:

1. Make a Server API call (using script or curl) 2. Return to Studio UI → Observability → Traces 3. Refresh the page 4. Verify traces from Server API call appear

If traces don't appear, see gcp-debugging.md.

Server Trace Verification

Both should appear in the Studio's Traces page. If only Studio traces appear, there's a trace pipeline issue.

Testing Custom API Routes (Deployed)

After deploying a server with custom routes:

# Staging
curl https://<project>.server.staging.mastra.cloud/hello

# Production
curl https://<project>.server.mastra.cloud/hello

# Expected: {"message":"Hello from custom route!"}

Browser Agent (Deployed)

When testing browser agents in deployed environments:

• Set headless: true in the browser config
• Browser runs server-side in the deployed container

Quick Commands Reference

# === Environment ===
export MASTRA_PLATFORM_API_URL=https://platform.mastra.ai          # production
export MASTRA_PLATFORM_API_URL=https://platform.staging.mastra.ai  # staging

# === Auth (warn user before login - opens browser!) ===
pnpx mastra@latest auth login
pnpx mastra@latest auth logout

# === Deploy (production) ===
pnpx mastra@latest studio deploy -y
pnpx mastra@latest server deploy -y

# === Deploy (staging) ===
pnpx mastra@latest studio deploy --config .mastra-project-staging.json -y
pnpx mastra@latest server deploy --config .mastra-project-staging.json -y

# === Test (use actual URLs from deploy output) ===
curl https://<project>.server.mastra.cloud/health           # production
curl https://<project>.server.staging.mastra.cloud/health   # staging

# === Agent call ===
curl -X POST <server-url>/api/agents/<agent-id>/generate \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Hello"}]}'

# === Check traces directly ===
TOKEN=$(jq -r '.token' ~/.mastra/credentials.json)
PROJECT_ID=$(jq -r '.projectId' .mastra-project.json)
ORG_ID=$(jq -r '.organizationId' .mastra-project.json)
curl -s "https://mobs-query-vgvrl5lbxq-uc.a.run.app/api/observability/traces?resourceId=$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" \
  -H "x-organization-id: $ORG_ID" | jq '.traces | length'

Troubleshooting

Wrong Organization on Login

The browser may default to a different account/org. Always verify after login:

cat ~/.mastra/credentials.json | jq '{email: .user.email, organizationId}'

Token Expired (401 errors)

WorkOS tokens expire after 5 minutes. Check token validity before re-logging in — use the refresh token first. See the get_valid_token helper in references/tests/traces.md.

"Session expired" errors in Studio

Known issue with cookie domain mismatch. The Studio may need re-authentication periodically.

Custom Routes Not Working

Custom routes must use apiRoutes (not routes) in the server config:

server: {
  apiRoutes: [helloRoute],  // ✅ Correct
  // routes: [helloRoute],  // ❌ Wrong - silently fails
}

This is a common typo that causes routes to silently not register.

CORS Errors

Server deploys inject CORS config via SERVER_WRAPPER. If you see CORS errors:

1. Check MASTRA_CORS_ORIGIN env var is set correctly on the deploy 2. Verify the origin domain matches the studio domain pattern

Server traces not appearing

1. Check mobs-collector logs (GCP Console)

• POST 200 = traces received
• POST 401 = JWT auth failed
• POST 404 = wrong endpoint

2. If 401 invalid signature: JWT_SECRET mismatch between services

3. If "mastra-cloud-observability-exporter disabled" in deploy logs:

• JWT_SECRET not configured on platform-api
• Server can't get MASTRA_CLOUD_ACCESS_TOKEN

See gcp-debugging.md for detailed debugging steps.

Deploy fails with auth error

pnpx mastra@latest auth logout
pnpx mastra@latest auth login

Then retry deploy.

Common Errors and Fixes

For detailed error documentation with infrastructure context, see the Notion page: [Common Errors and Fixes](https://www.notion.so/33bebffbc9f881da8415c12fae971872)

Quick Troubleshooting

Traces Not Appearing

Deploy Issues

When to Escalate

Contact the platform team if:

• Redeploy doesn't fix trace issues
• You see 401 or 404 errors in deploy logs
• Issues persist across multiple projects

For infrastructure debugging (requires GCP access), see the detailed Notion docs.

Environment Variables Reference

For detailed internal infrastructure variables, see the Notion page: [Environment Variables Reference](https://www.notion.so/33bebffbc9f881e694f5f8af9df85a4a)

Variables You Need to Set

For Smoke Testing

Environment Values

Staging:

export MASTRA_PLATFORM_API_URL=https://platform.staging.mastra.ai

Production (default):

export MASTRA_PLATFORM_API_URL=https://platform.mastra.ai
# Or just don't set it — production is the default

Variables Set Automatically

These are injected by the platform during deployment — you don't need to set them:

• MASTRA_CLOUD_ACCESS_TOKEN — JWT for trace authentication
• MASTRA_CLOUD_TRACES_ENDPOINT — Where traces are sent

Checking Your Environment

# Verify which environment you're targeting
echo $MASTRA_PLATFORM_API_URL

# Check if you're authenticated
mastra auth status

GCP Debugging Guide

Internal: This guide requires GCP Console access. Contact a team member with infrastructure access if you need help.

For detailed GCP debugging instructions, see the Notion page: [GCP Debugging Guide](https://www.notion.so/33bebffbc9f881758e0bd10be1ef7e84)

Quick Reference

When to Check GCP Logs

• Server traces not appearing in Studio
• Deploy failing silently
• Authentication/session issues

What You'll Need

• GCP Console access to the appropriate project (staging or production)
• Knowledge of which service to check based on the symptom

Who to Contact

If you don't have GCP access, reach out to the platform team for help debugging infrastructure issues.

Local Development Setup

Instructions specific to --env local testing.

Prerequisites

• Browser tools enabled via /browser on
• Works with either Stagehand (AI-powered) or AgentBrowser (deterministic) providers

If browser tools are not available, run /browser to configure.

Environment Variables

Based on the selected LLM provider, ensure the API key is available:

Check order:

1. Global environment: echo $<ENV_VAR_NAME> 2. Project .env file 3. Ask user only if not found

Start Development Server

1. Check for a zombie on :4111 first

mastra dev auto-increments the port if :4111 is already in use (e.g. :4112, :4113). If you don't notice, your subsequent curls will hit the wrong project (any earlier test session left running). Always check:

lsof -i :4111
# If a node process is listening, kill it before starting:
kill $(lsof -ti :4111) 2>/dev/null

2. Start

cd <project-directory>
<pm> run dev

Server starts on http://localhost:4111. Wait for "Mastra API running" in the output and confirm the printed URL before running tests:

curl -s -o /dev/null -w "HTTP %{http_code}\n" http://localhost:4111/api/agents
# HTTP 200 → dev server up

If the dev server prints `url: "http://localhost:4112/api"` instead of `:4111`, port 4111 was already taken. Stop, kill the zombie, and restart, or pass --port 4111 if supported — otherwise all the curl examples in the test references will target the wrong server.

Local Observability Setup

Verify observability is configured before testing traces:

1. Check src/mastra/index.ts

create-mastra now scaffolds PinoLogger + Observability (not the older createLogger / OtelConfig), and wires a MastraCompositeStore with a default LibSQL store plus a DuckDB store for the observability domain:

import { Mastra } from '@mastra/core/mastra';
import { PinoLogger } from '@mastra/loggers';
import { LibSQLStore } from '@mastra/libsql';
import { DuckDBStore } from '@mastra/duckdb';
import { MastraCompositeStore } from '@mastra/core/storage';
import {
  Observability,
  MastraStorageExporter,
  MastraPlatformExporter,
  SensitiveDataFilter,
} from '@mastra/observability';

export const mastra = new Mastra({
  // ... agents, workflows, scorers
  storage: new MastraCompositeStore({
    id: 'composite-storage',
    default: new LibSQLStore({ id: 'mastra-storage', url: 'file:./mastra.db' }),
    domains: {
      observability: await new DuckDBStore().getStore('observability'),
    },
  }),
  logger: new PinoLogger({ name: 'Mastra', level: 'info' }),
  observability: new Observability({
    configs: {
      default: {
        serviceName: 'mastra',
        exporters: [new MastraStorageExporter(), new MastraPlatformExporter()],
        spanOutputProcessors: [new SensitiveDataFilter()],
      },
    },
  }),
});

2. Check Dependencies

Verify package.json includes:

• @mastra/observability
• @mastra/loggers
• @mastra/libsql (default store) and @mastra/duckdb (observability domain)

3. Check Dev Server Output

When starting the dev server, look for the version banner and Studio URL, e.g. mastra 1.13.0-alpha.4 ready in … followed by Studio at http://localhost:4111 and the API at http://localhost:4111/api. Should NOT see MASTRA_CLOUD_ACCESS_TOKEN not set (that's for cloud only).

Troubleshooting Local Traces

If traces are missing:

1. Verify telemetry config — Check telemetry is configured in Mastra instance 2. Restart dev server — Config changes require restart 3. Check browser console — Look for OTel export errors 4. Check dependencies — Ensure @mastra/observability is installed

Testing Custom API Routes

After adding a custom route (see main SKILL.md):

curl http://localhost:4111/hello
# Expected: {"message":"Hello from custom route!"}

Browser Agent Testing (Local)

When testing browser agents locally, you'll experience "browserception" — your MastraCode browser watching the project's agent browser.

Ensure Playwright browsers are installed:

<pm> exec playwright install chromium

Notes

• Local traces are stored in-memory by default
• Traces persist only while the dev server is running
• For persistent traces, configure a storage backend

Release Scope Discovery

Before running release smoke tests, determine what changed since the last release. Do not rely only on the default smoke-test checklist; use the release diff to decide whether targeted checks are needed.

0. Create a dated smoke-test workspace

Create a date-scoped workspace before collecting release artifacts. Keep the generated Mastra project, PR list, logs, and smoke reports together.

SMOKE_DATE=$(date +%F)
SMOKE_DIR="$HOME/mastra-smoke-tests/$SMOKE_DATE"
mkdir -p "$SMOKE_DIR/logs"

If $HOME/mastra-smoke-tests is outside the repo sandbox, request filesystem access before writing there.

Expected layout:

~/mastra-smoke-tests/YYYY-MM-DD/
  merged-prs.tsv
  smoke-scope.md
  smoke-report.md
  logs/
  smoke-project/
  stable-smoke-project/

Prefer the helper script when available:

.claude/skills/mastra-smoke-test/scripts/discover-release-scope.sh --release-tag '@mastra/core@1.28.0'

The script writes merged-prs.tsv and a starter smoke-scope.md to the workspace.

1. Identify the last release baseline

Find the previous stable release tag and timestamp. Prefer the tag/release that corresponds to the package being released, usually @mastra/core@<version> for monorepo releases.

gh release list --limit 20
gh release view '@mastra/core@<version>' --json tagName,publishedAt,createdAt,targetCommitish,url
git show -s --format='%H %cI %s' '@mastra/core@<version>'

Use the tag commit date or release createdAt as the cutoff for merged PR discovery. State which cutoff you used.

2. List merged PRs since the cutoff

Export merged PRs since the baseline into the dated workspace. Exclude Dependabot unless the release specifically needs dependency smoke coverage.

gh pr list \
  --state merged \
  --search 'merged:>=YYYY-MM-DDTHH:MM:SSZ -author:app/dependabot' \
  --limit 200 \
  --json number,title,author,mergedAt,labels,url \
  --jq '.[] | [.number, .mergedAt, .author.login, .title, .url] | @tsv' \
  > "$SMOKE_DIR/merged-prs.tsv"

If the release has more than 200 PRs, paginate or narrow by date ranges until the full set is captured. Cross-check with first-parent merge commits when needed:

git log --first-parent --oneline '@mastra/core@<previous>'..origin/main

3. Categorize the PRs

Group PRs by runtime surface area and smoke implication:

4. Convert categories into a smoke plan

Use full smoke as the baseline, but do not stop at the default generated-project happy path. The goal is to prove the actual changed feature or bug fix works in the published package, not just that a nearby happy path still works.

For every material PR, ask:

1. What user-visible behavior, API behavior, persistence behavior, or integration path changed? 2. Does the generated smoke project execute that exact path? 3. If not, what is the smallest targeted check that proves the changed behavior? 4. What evidence will show the fix worked, not merely that the app did not crash?

If the default project does not exercise the changed feature, add a targeted check or explicitly record why it cannot be tested in this environment.

For targeted check patterns, read references/targeted-feature-smoke.md. For storage/provider schema or migration changes, read references/storage-provider-migration-smoke.md.

Add a Coverage vs Changes table to $SMOKE_DIR/smoke-scope.md before testing:

Write the scope analysis to $SMOKE_DIR/smoke-scope.md before running tests so it is not trapped in terminal output. Include:

• release baseline tag and cutoff timestamp
• command used to collect PRs
• categorized PR table with PR number, title, author, merged time, and category
• targeted smoke plan derived from the categories
• coverage-vs-changes table showing what the generated project covers naturally and what needs targeted checks
• any omitted PRs/commits and why, including Dependabot, direct non-PR commits, cloud-only features, missing credentials, or product areas outside create-mastra

Report the category summary and the coverage-vs-changes summary before running tests so the user can see why each targeted check is included and where the default smoke project is insufficient.

Release Smoke Index

This file is intentionally short. Use the smallest reference needed for the branch you are on.

Alpha release

Start in the top-level SKILL.md under Release smoke workflows. It includes the commands to identify whether the alpha versioning PR is open, merged, or missing.

• If the alpha versioning PR is open, read references/alpha-versioning-pr.md.
• If the alpha versioning PR is merged, read references/alpha-publish.md.
• After alpha packages publish, read references/release-scope-discovery.md.

Stable release

• For normal stable publish monitoring and final smoke, read references/stable-release-smoke.md.
• If stable publish partially fails, read references/stable-partial-publish-recovery.md.

Targeted checks

• For changed features not covered by the generated project, read references/targeted-feature-smoke.md.
• For storage/provider schema or migration changes, read references/storage-provider-migration-smoke.md.

Stable Partial Publish Recovery

Use this only when the stable publish fails after some packages have already published.

Treat it as a partial release. Do not create a new versioning PR or bump versions. First identify the failed step and whether npm latest is split across old and new versions.

Recovery flow

1. Record which packages already reached npm latest and which are still old. 2. Rerun the same stable Publish to npm workflow from the same release commit on main. 3. After the rerun succeeds, verify all intended package versions are on npm latest. 4. Verify release git tags were created after the successful publish. 5. Only then run final stable smoke against create-mastra@latest.

Tag behavior

The Add tags step runs after publish:

pnpm changeset-cli tag
git push origin --tags

It creates and pushes git tags only for packages Changesets considers part of the current release/version bump, not every package in the monorepo. The tags look like @mastra/core@1.29.0, mastra@1.7.0, and create-mastra@1.7.0. If the first publish attempt fails before Add tags, the rerun should create tags for the full changed-package release set after npm publish completes.

Verify changed-package tags, not every workspace package:

git fetch --tags
# Spot-check release-critical tags
git tag -l '@mastra/core@<version>'
git tag -l 'mastra@<version>'
git tag -l 'create-mastra@<version>'
git tag -l '@mastra/server@<version>'
git tag -l '@mastra/playground-ui@<version>'

If a package version is on npm latest but its expected release tag is missing after a successful rerun, stop and report it before smoke testing.

Stable Release Smoke

Use this after the stable/full release workflow starts or completes.

After the stable/full release publishes, run smoke again against the published stable packages. Do not rely on alpha smoke as final stable-release signoff because the stable publish path, npm dist-tags, generated project install path, and package versions are distinct release surfaces.

1. Confirm the stable publish workflow completed

Inspect the full release run. The stable path should complete successfully, with snapshot/prerelease jobs skipped when this is a normal full release.

gh run view <run-id> --json name,event,status,conclusion,workflowName,headBranch,headSha,jobs,url --jq .
gh run watch <run-id>

If the run fails or is cancelled, stop and report the failed job/step before creating a fresh smoke project.

If it fails after some packages published, use references/stable-partial-publish-recovery.md before smoke testing.

2. Confirm stable packages are installable

Check the published latest versions and make sure they match the intended stable release versions:

npm view @mastra/core@latest version
npm view mastra@latest version
npm view create-mastra@latest version

If npm returns the previous stable version, wait for publish/registry propagation and retry. Do not run final stable smoke against stale latest packages.

3. Create a fresh stable smoke project

Use the same dated workspace, but create a separate project from the alpha project so dependency resolution and generated files prove the stable release path independently.

# Reuse the existing SMOKE_DIR from the alpha/release-scope run.
: "${SMOKE_DIR:?Set SMOKE_DIR via references/release-scope-discovery.md before stable smoke}"
mkdir -p "$SMOKE_DIR/logs"

cd "$SMOKE_DIR"
pnpm create mastra@latest stable-smoke-project -c agents,tools,workflows,scorers -l openai -e
cd stable-smoke-project
pnpm run dev

If an existing dev server is holding port 4111 or a DuckDB lock, stop that process before starting the stable project. Do not run alpha and stable smoke projects against the same generated project directory or storage file.

4. Rerun the required smoke coverage

For stable release signoff, rerun at least:

• mandatory local checklist: setup, agents, tools, workflows, traces, scorers, memory, MCP, errors
• Local Studio browser smoke: shell/version, agent chat, tool execution, workflow run, traces, scorers, MCP
• targeted release-scope checks identified from the PR categorization, especially any checks added because the generated project does not cover changed features

Append stable results to the dated smoke-report.md in a separate section from alpha results and clearly record the package versions tested.

Storage/Provider Migration Smoke Pattern

Use this when a release includes storage/provider schema, init, or migration changes. The goal is to prove the released provider package works against a real backend and repairs the old/broken schema state.

Pattern

1. Start the affected backend locally when feasible.

   docker run -d --name mastra-smoke-pg \
     -e POSTGRES_PASSWORD=postgres \
     -e POSTGRES_DB=mastra \
     -p 5544:5432 \
     postgres:16

2. Add the released provider package to the generated smoke project.

   pnpm add @mastra/pg@alpha pg
   # or, for stable smoke:
   pnpm add @mastra/pg@latest pg

3. Configure the smoke project to use the provider through public Mastra config. 4. Enable the feature that depends on the changed schema, such as observational memory. 5. Create or mutate backend state to mimic the pre-fix schema. 6. Restart the Mastra dev server so provider init/migration runs from the released package. 7. Verify the missing column/table/index is restored. 8. Run a real API/UI flow that writes and reads through the affected provider path.

Example evidence: Postgres observational-memory migration

For a Postgres observational-memory migration fix, collect evidence such as:

• information_schema.columns shows the missing column exists after restart, for example reflectedObservationLineCount
• an agent call using PG-backed memory succeeds
• a second call recalls the test phrase/thread context
• memory messages and observational memory rows exist in Postgres

Record the backend image/version, package version, schema mutation, verification query, and API result in smoke-report.md.

Targeted Feature Smoke Pattern

Use this when release scope discovery shows that the generated smoke project does not exercise a changed runtime feature.

When a PR changes a specific feature, smoke the smallest real scenario that proves that feature. Avoid vague substitutes like "agent chat works" for a streaming fix, "tools list loads" for a tool approval fix, or "Studio loads" for a persistence/save bug.

Pattern

1. Identify the exact changed path from the PR title, files, changelog, and tests. 2. Configure or modify the smoke project so that path is reachable with the released package. 3. Trigger the behavior through the public API or UI a user would use. 4. Assert the before/after condition that would have failed before the fix. 5. Capture concrete evidence: response fields, stream chunks, persisted rows, saved config after reload, trace/span contents, or visible UI text. 6. If the feature requires cloud auth, external credentials, or a separate product like Mastra Code, either run that targeted environment or mark it PARTIAL/NOT COVERED with the exact reason.

Targeted additions by category

• CLI/create-mastra changed: create a brand-new project at $SMOKE_DIR/smoke-project with the release tag and run pnpm run dev.
• Server/adapters/API changed: add curl checks for /health, /api/agents, /generate, /stream if applicable, tool execute, workflow start, custom routes, and invalid routes. If route prefixing changed, add or use a custom route and verify built-in /api/* routes remain reserved.
• Agent streaming changed: test /stream, resume-stream, streamUntilIdle, abort/length/error behavior, or another endpoint that actually uses the changed streaming path.
• Tools changed: test the exact changed tool behavior, such as dynamic tools, approval functions, requireApproval, programmatic tool calls, or preserved args. A single static weather tool call is not enough for dynamic/approval/tool-merge changes.
• Workflows changed: test the exact changed behavior, such as suspend/resume, background task progress, long-running runs, dataset/experiment workflows, or start-async output shape.
• Memory changed: test thread/resource isolation plus the changed memory mode, such as current-thread recall defaults, observational memory boundaries, or agent network incompatibility. A basic two-call memory check is necessary but may not be sufficient. If the change only affects memory under a specific storage backend, configure the smoke project to use that backend.
• Memory storage migrations changed: run an end-to-end migration smoke against the affected backend. Use references/storage-provider-migration-smoke.md.
• Forked subagents changed: create or use a Mastra Code/subagent scenario that proves parent thread/resource inheritance and prompt cache prefix behavior. Do not assume a normal agent run covers forked subagents.
• Studio/Playground changed: run browser smoke for the affected pages, especially observability traces/logs/metrics and theme/layout changes.
• Auth/permissions/Agent Builder changed: prefer staging/production cloud smoke with an authenticated user and targeted permission flows. Local create-mastra usually does not cover stored agents/skills, starring, visibility, avatar upload, or server-side session refresh.
• MCP/A2A changed: default empty MCP state is only a baseline. For SDK/client/server or schema-validator changes, run a targeted MCP/A2A integration check with a configured server/client when feasible.
• Storage/provider packages changed: at minimum verify package installation/import. If the changed provider can run locally in Docker, run a provider-backed smoke against the released package rather than stopping at import.
• Mastra Code changed: run a separate Mastra Code/TUI smoke path; do not assume standard create-mastra smoke covers it.
• Docs/examples changed: run docs validation or example-specific checks; do not replace runtime smoke with docs-only checks.

Examples

• For streaming fixes: call /stream or resume-stream, inspect event chunks and final response shape.
• For tool approval/dynamic tool fixes: configure the affected tool mode, run an agent that triggers it, and verify approval/rejection or dynamic resolution behavior.
• For workflow fixes: run the specific workflow mode changed, such as suspend/resume, background progress, or long-running output shape.
• For Studio persistence fixes: change the affected form field, save, reload/refetch, and verify the value persisted.
• For observability fixes: generate the affected run type and verify trace/span/scores/logs include the corrected data.
• For CLI/create fixes: create a fresh project with the published CLI and verify generated files/dependencies/scripts match the intended output.

Agents Testing (--test agents)

Purpose

Verify agents page loads and agent chat functionality works.

Steps

1. Navigate to Agents Page

• [ ] Open /agents in Studio
• [ ] Note if agents list loads and any errors displayed
• [ ] Record which agents appear (e.g., "Weather Agent")

2. Open Agent Chat

• [ ] Click on an agent (e.g., Weather Agent)
• [ ] Note if chat interface loads
• [ ] Record whether input field is visible

3. Send Test Message

• [ ] Enter: What's the weather in Tokyo?
• [ ] Click Send or press Enter
• [ ] Wait for response (may take 5-30 seconds)

4. Observe Response

• [ ] Record the agent's response content
• [ ] Note if response is coherent and relevant
• [ ] Record any error messages displayed

5. Test Follow-up (Memory Check)

• [ ] Send: What about London?
• [ ] Note if agent references the previous question
• [ ] Record whether context appears to be maintained

Observations to Report

Common Issues

Browser Actions

Navigate to: /agents
Click: First agent in list
Type in chat: "What's the weather in Tokyo?"
Click: Send button
Wait: For response
Type in chat: "What about London?"
Click: Send button
Wait: For response

Curl / API (for --skip-browser)

`<agentKey>` is the key used in `Mastra({ agents: { weatherAgent } })`, not the agent's `id` field. A template where agents: { weatherAgent } and the agent's id: 'weather-agent' is addressed as /api/agents/weatherAgent/..., not /api/agents/weather-agent/....

List agents:

curl -s http://localhost:4111/api/agents

Generate (single call, no memory):

curl -s -X POST "http://localhost:4111/api/agents/<agentKey>/generate" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"What is the weather in Tokyo?"}]}'

Generate with memory (see memory.md for the two-call persistence check):

curl -s -X POST "http://localhost:4111/api/agents/<agentKey>/generate" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"..."}],"memory":{"thread":"<tid>","resource":"<rid>"}}'

Pass criteria:

• /api/agents returns a JSON object keyed by agent key
• /generate returns HTTP 200 with a text field containing a coherent

response (and, if the agent has a tool, toolCalls / toolResults in steps)

Common mistake: sending threadId / resourceId at the top level to /generate — these are silently discarded. Use memory: { thread, resource }. Top-level threadId / resourceId are only read by the deprecated /generate-legacy route.

Browser agents (browser: new StagehandBrowser(...)) require memory: { thread, resource } on every call — without it the auto-attached BrowserContextProcessor throws computeStateSignal requires Mastra memory with an active resourceId and threadId. See tests/setup.md → "Runtime requirement".

Error Handling Testing (--test errors)

Purpose

Verify the application handles errors gracefully with user-friendly messages.

Steps

1. Test Agent Error Handling

• [ ] Navigate to /agents
• [ ] Select an agent
• [ ] Send intentionally problematic input:
• Empty message
• Very long message (10000+ chars)
• Special characters only: @#$%^&*()
• [ ] Record the error message displayed (note if stack trace or user-friendly)

2. Test Tool Error Handling

• [ ] Navigate to /tools
• [ ] Select a tool
• [ ] Submit with invalid input:
• Empty required fields
• Wrong data type (text for number field)
• Invalid format
• [ ] Record the error message displayed

3. Test API Error Handling (Cloud)

For --env staging or --env production:

Replace <server-url> with your environment URL, <agent-id> with an agent from your setup, and <your-api-key> from the platform dashboard.

# Invalid agent
curl -X POST <server-url>/api/agents/nonexistent-agent/generate \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"test"}]}'

# Invalid JSON
curl -X POST <server-url>/api/agents/<agent-id>/generate \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d 'not valid json'

# Missing required fields
curl -X POST <server-url>/api/agents/<agent-id>/generate \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{}'

• [ ] Record HTTP status codes returned
• [ ] Record error message content
• [ ] Note if stack traces appear in response

4. Test Navigation Errors

• [ ] Navigate to invalid route: /nonexistent-page
• [ ] Record what page/behavior appears
• [ ] Navigate to invalid agent: /agents/fake-agent-id
• [ ] Record the error handling behavior

5. Test Network Error Recovery

• [ ] Start a long-running operation
• [ ] Briefly disconnect network (if possible)
• [ ] Record error-handling behavior
• [ ] Note if retry or recovery options appear

Observations to Report

Error Message Quality

Note these aspects of error messages:

• Explain what went wrong
• Suggest how to fix it
• Not expose internal details
• Be readable by non-developers

Bad: TypeError: Cannot read property 'x' of undefined Good: Unable to process your request. Please try again.

Common Issues

Browser Actions

# Agent error test
Navigate to: /agents
Click: Select agent
Type: "@#$%^&*()"
Send: Message
Verify: Error is user-friendly

# Tool error test
Navigate to: /tools
Click: Select tool
Clear: All inputs
Click: Submit
Verify: Validation error shown

# 404 test
Navigate to: /this-page-does-not-exist
Verify: 404 or redirect, not crash

Curl / API (for --skip-browser)

Same curls for local and cloud; cloud needs Authorization: Bearer <api-key>.

# Unknown agent
curl -sw "\nHTTP %{http_code}\n" -X POST \
  http://localhost:4111/api/agents/nonexistent/generate \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"hi"}]}'

# Workflow with missing required input field
curl -sw "\nHTTP %{http_code}\n" -X POST \
  http://localhost:4111/api/workflows/<workflowId>/start-async \
  -H "Content-Type: application/json" \
  -d '{"inputData":{}}'

# Tool with missing required input field
curl -sw "\nHTTP %{http_code}\n" -X POST \
  http://localhost:4111/api/tools/<toolId>/execute \
  -H "Content-Type: application/json" \
  -d '{"data":{}}'

# Unknown tool
curl -sw "\nHTTP %{http_code}\n" -X POST \
  http://localhost:4111/api/tools/nonexistent/execute \
  -H "Content-Type: application/json" \
  -d '{"data":{}}'

Expected behavior

The current server returns the following. These are the values to assert against — flag any deviation as a regression.

Known inconsistencies (document if you observe, don't treat as failures unless they change):

• Workflow invalid input returns 500 (arguably should be 400)
• Tool invalid input returns 200 with error: true in the body

(inconsistent with HTTP semantics vs. workflow/agent error responses)

Pass criteria

• Every error response includes a readable error field (or

validationErrors for tools)

• No stack traces leak into the response body
• HTTP status codes match the table above (or are documented deviations)

MCP Servers Testing (--test mcp)

Purpose

Verify MCP (Model Context Protocol) servers page loads and connections work.

Steps

1. Navigate to MCP Page

• [ ] Open /mcps in Studio
• [ ] Note if page loads and any errors displayed
• [ ] Record what MCP servers list shows

2. Observe Empty State

If no MCP servers configured:

• [ ] Record the empty state message shown
• [ ] Note any errors displayed
• [ ] Record if instructions for adding servers appear

3. Observe Configured Servers

If MCP servers are configured:

• [ ] Record which servers appear in list
• [ ] Note connection status shown (connected/disconnected)
• [ ] Record server names and types visible

4. Test Server Connection

For each configured server:

• [ ] Record connection status
• [ ] Note available tools from server
• [ ] Record which tools are discoverable

5. Test MCP Tool (if available)

• [ ] Navigate to /tools
• [ ] Find MCP-provided tool
• [ ] Execute tool
• [ ] Record the result and whether it calls external server

Observations to Report

MCP Configuration

Servers are typically configured in project code:

import { MCPConfiguration } from '@mastra/core/mcp';

const mcp = new MCPConfiguration({
  servers: {
    myServer: {
      command: 'node',
      args: ['path/to/server.js'],
    },
  },
});

Common Issues

Notes

• MCP is optional - empty state is acceptable
• External MCP servers may require separate processes
• Connection issues may be transient

Browser Actions

Navigate to: /mcps
Wait: For page to load
Verify: Page loads without errors
Verify: Server list OR empty state visible

# If servers configured:
Click: On server in list
Verify: Connection status shown
Verify: Available tools listed

Curl / API (for --skip-browser)

The server exposes MCP endpoints under /api/mcp/v0/... (not /api/mcps — that's the Studio route). The MCP here refers to Mastra hosting MCP servers for external clients; it does not list external MCP clients the project consumes.

1. List MCP servers this Mastra instance exposes

curl -s "http://localhost:4111/api/mcp/v0/servers" | jq '.'

Response shape: { servers: [...], total_count: N, next: null }. An empty array is a valid pass if the project declares no MCP servers (the default create-mastra template does not).

2. Get one server's metadata

curl -s "http://localhost:4111/api/mcp/v0/servers/<serverId>" | jq '.'

3. List tools a server exposes, and execute one

# List tools
curl -s "http://localhost:4111/api/mcp/<serverId>/tools" | jq '.'

# Inspect one tool
curl -s "http://localhost:4111/api/mcp/<serverId>/tools/<toolId>" | jq '.'

# Execute the tool
curl -s -X POST "http://localhost:4111/api/mcp/<serverId>/tools/<toolId>/execute" \
  -H "Content-Type: application/json" \
  -d '{"data":{ ... }}'

Pass criteria:

• GET /api/mcp/v0/servers returns HTTP 200 with the expected shape

(empty servers array is OK)

• If servers are declared: each shows up in the list and

GET /api/mcp/v0/servers/:id returns a non-null object

• If tools are exposed: GET /api/mcp/:serverId/tools lists them and

POST /.../execute returns a successful result

Common mistakes:

• Hitting /api/mcps or /api/mcp/servers — neither exists server-side
• Treating an empty servers list as failure on the default template —

it's expected

Memory Testing (--test memory)

Purpose

Verify conversation memory persists and context is maintained.

Prerequisites

• Agent with memory configured
• Completed at least one agent chat

Steps

1. Start Fresh Conversation

• [ ] Navigate to /agents
• [ ] Select an agent (e.g., Weather Agent)
• [ ] Send: What's the weather in Tokyo?
• [ ] Wait for response and record it

2. Test Context Retention

• [ ] Send follow-up: What about comparing it to London?
• [ ] Note if agent references Tokyo in response
• [ ] Record whether agent understands "it" refers to weather

3. Test Navigation Persistence

• [ ] Navigate away (e.g., to /tools)
• [ ] Navigate back to /agents → same agent
• [ ] Note if conversation history is visible
• [ ] Record which previous messages are displayed

4. Test Cross-Session (if applicable)

• [ ] Note the current thread/conversation
• [ ] Refresh the page (F5)
• [ ] Navigate back to the same agent
• [ ] Record whether history persists

5. Test New Thread

• [ ] Start a new conversation (if UI supports)
• [ ] Note if new thread has no history
• [ ] Record whether old thread is still accessible

Observations to Report

Memory Configurations

Common Issues

Browser Actions

Navigate to: /agents
Click: Select agent
Type: "What's the weather in Tokyo?"
Send: Message
Wait: For response
Type: "What about comparing it to London?"
Send: Message
Verify: Response references Tokyo

Navigate to: /tools
Navigate to: /agents
Click: Same agent
Verify: Previous messages visible

Refresh: Page (F5)
Navigate to: /agents
Click: Same agent
Verify: History still visible (if persistent storage)

Curl / API (for --skip-browser)

The current /agents/:agentId/generate route expects thread/resource under a memory object. Top-level threadId / resourceId are only read by the deprecated /generate-legacy route — sending them to /generate silently discards them and the agent will appear to "forget" context.

Correct request shape:

{
  "messages": [{ "role": "user", "content": "..." }],
  "memory": { "thread": "<thread-id>", "resource": "<resource-id>" }
}

Two-call persistence check:

TID="smoke-memory-$(date +%s)"
RID="smoke-user"

# Call 1: seed context
curl -s -X POST "http://localhost:4111/api/agents/<agentKey>/generate" \
  -H "Content-Type: application/json" \
  -d "{\"messages\":[{\"role\":\"user\",\"content\":\"Remember: my name is Abhi.\"}],\"memory\":{\"thread\":\"$TID\",\"resource\":\"$RID\"}}"

# Call 2: same thread, verify recall
curl -s -X POST "http://localhost:4111/api/agents/<agentKey>/generate" \
  -H "Content-Type: application/json" \
  -d "{\"messages\":[{\"role\":\"user\",\"content\":\"What is my name?\"}],\"memory\":{\"thread\":\"$TID\",\"resource\":\"$RID\"}}"

# Assert: thread exists in storage
curl -s "http://localhost:4111/api/memory/threads?resourceId=$RID" | \
  jq '{total, ids: (.threads | map(.id))}'

Response shape: GET /api/memory/threads returns { threads: [...], total, page, perPage, hasMore } — not a bare array. Each entry has { id, resourceId, title, metadata, createdAt, updatedAt }.

Query params: resourceId is case-sensitive (capital I). Lowercase resourceid is silently ignored and returns all threads, which can make a broken test look like it passed. agentId is optional.

Pass criteria:

• Call 2 response references "Abhi"
• GET /api/memory/threads?resourceId=<rid> returns .total >= 1 with a

thread whose id matches $TID

• To harden: seed a second thread under a different resourceId and

confirm the filter excludes it

If call 2 forgets context: check you sent memory: { thread, resource } (not top-level threadId / resourceId) and that <agentKey> matches the key used in the Mastra({ agents }) config, not the agent's id field.

If `/memory/threads` returns threads from other resources: you typed resourceid instead of resourceId — the unknown param is dropped and no filter is applied.

Scorers Testing (--test scorers)

Purpose

Verify evaluation scorers page loads and displays available scorers.

Steps

1. Navigate to Scorers Page

• [ ] Open /evaluation?tab=scorers in Studio
• [ ] Note if page loads and any errors displayed
• [ ] Record what scorers list shows

2. Observe Scorers Display

• [ ] Record which scorers are listed
• [ ] Note what information is shown for each scorer (name, description)
• [ ] Record any error messages

3. Check Scorer Details (if available)

• [ ] Click on a scorer to view details
• [ ] Record what configuration is visible
• [ ] Note any run history shown

Observations to Report

Notes

• Scorers are optional - empty state is OK if none configured
• Default project may include example scorers
• Scorer runs appear in traces as scorer run: <name>

Common Issues

Browser Actions

Navigate to: /evaluation?tab=scorers
Wait: For page to load
Verify: Page loads without errors
Verify: Scorers list visible (may be empty)

Curl / API (for --skip-browser)

There is no "execute scorer" HTTP endpoint. Scorers run automatically as part of agent / workflow execution (live scoring) and record score rows. To smoke-test scorers over the API you verify (1) they are registered, and (2) they emit scores when an agent / workflow runs.

1. List registered scorers

curl -s http://localhost:4111/api/scores/scorers | jq 'keys'

Pass: returns an object keyed by scorer id (weather-scorer, translation-quality-scorer, etc.) for the scorers declared in the project. Empty {} is only acceptable if the project genuinely declares none.

2. Get a single scorer's config

curl -s http://localhost:4111/api/scores/scorers/<scorerId> | jq '.'

Pass: returns a non-null object with config and the agents/workflows it is attached to. null means the id is wrong or the scorer is not registered.

3. Trigger scoring by running an agent / workflow, then read scores

# Run the workflow (or agent) that has scorers attached
curl -s -X POST "http://localhost:4111/api/workflows/<workflowId>/start-async" \
  -H "Content-Type: application/json" \
  -d '{"inputData":{"city":"Tokyo"}}' | jq '.traceId'

# Scores recorded for that run
curl -s "http://localhost:4111/api/observability/scores?page=0&perPage=20" \
  | jq '.scores | map({scorerId, score, reason})'

Pass criteria:

• /api/scores/scorers lists every scorer the template declares
• After a workflow/agent run completes, /api/observability/scores has

new entries with the expected scorerIds and numeric score values

• Each recorded score has a runId / traceId tying it back to the

invoking run

Common mistakes:

• Treating POST /api/scores/scorers/<id>/execute as real — it does not

exist. If you see agent output that claims a score without a corresponding row in /observability/scores, you hallucinated the result.

• Expecting scorers to run on ad-hoc text input — they need a live

agent/workflow run.

Server Deploy Testing (--test server)

Cloud only: For --env staging or --env production.

Purpose

Verify Server deployment works and API is accessible.

Prerequisites

• Mastra platform account
• Project with at least one agent
• Authenticated via mastra auth login
• Studio deployed first (recommended)

Steps

1. Set Environment

# For staging
export MASTRA_PLATFORM_API_URL=https://platform.staging.mastra.ai

# For production (default)
unset MASTRA_PLATFORM_API_URL

2. Authenticate (if not already)

pnpx mastra@latest auth login

3. Deploy Server

pnpx mastra@latest server deploy -y

Watch for:

• [ ] Build starts
• [ ] Build completes (note any warnings)
• [ ] Deploy starts
• [ ] Capture Server URL from output

Critical warnings to note:

• mastra-cloud-observability-exporter disabled - traces won't work
• CLOUD_EXPORTER_FAILED_TO_BATCH_UPLOAD_LOGS - trace endpoint issue

4. Test Health Endpoint

curl <server-url>/health

• [ ] Record HTTP status code returned
• [ ] Record response body content

5. Test Agent API

curl -X POST <server-url>/api/agents/weather-agent/generate \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Weather in Tokyo?"}]}'

• [ ] Record HTTP status code returned
• [ ] Record response content

6. Use Test Script

.claude/skills/mastra-smoke-test/scripts/test-server.sh <server-url>

• [ ] Record health check result
• [ ] Record agent call result
• [ ] Note script exit code

7. Check Traces in Studio

• [ ] Open Studio /observability
• [ ] Refresh page
• [ ] Note if trace from Server API call appears
• [ ] Record how long until trace appears (if at all)

Observations to Report

Deploy URLs

API Endpoints

Common Issues

Notes

• Server cold starts may take 10-30 seconds
• First request after deploy may be slow
• Traces may take up to 30 seconds to appear
• Redeploy if traces consistently missing

Project Setup

Purpose

Set up or verify a Mastra project for smoke testing.

Multi-Environment Config

One project can target multiple environments using separate config files:

• Local = Running your Mastra project with pnpm dev (no cloud deploy)
• Staging/Production = Deploying to Mastra platform

Setting Up Multi-Environment

After creating a project:

# Deploy to staging (creates .mastra-project-staging.json)
export MASTRA_PLATFORM_API_URL=https://platform.staging.mastra.ai
pnpx mastra@latest auth login
pnpx mastra@latest studio deploy --config .mastra-project-staging.json -y
pnpx mastra@latest server deploy --config .mastra-project-staging.json -y

# Deploy to production (creates .mastra-project.json)
export MASTRA_PLATFORM_API_URL=https://platform.mastra.ai
pnpx mastra@latest auth login
pnpx mastra@latest studio deploy -y
pnpx mastra@latest server deploy -y

Each deploy creates a separate project ID in its config file, so staging and production don't interfere.

Note: Always warn user before running auth login as it opens a browser.

---

Option A: Create New Project

1. Navigate to Directory

cd <directory>
# Default: ~/mastra-smoke-tests

2. Create Project

<pm> create mastra@<tag> <project-name> -c agents,tools,workflows,scorers -l <llm> -e

3. Enter Project

cd <project-name>

4. Record Structure

• [ ] Note if package.json exists
• [ ] Note if src/mastra/index.ts exists
• [ ] Record agents found in src/mastra/agents/
• [ ] Record tools found in src/mastra/tools/

Option B: Use Existing Project

1. Navigate to Project

cd <existing-project-path>

2. Record Requirements

• [ ] Note if package.json contains @mastra/core
• [ ] Note if src/mastra/index.ts has Mastra instance
• [ ] Record which agents are configured

3. Update Dependencies (if --tag provided)

# Update ALL @mastra/* packages to avoid version drift
<pm> add @mastra/core@<tag> @mastra/memory@<tag> mastra@<tag>

# Also update any adapters in package.json:
# @mastra/libsql, @mastra/pg, @mastra/turso, @mastra/duckdb
# @mastra/evals, @mastra/observability, @mastra/stagehand

Important: Check package.json first — only update packages that exist.

Storage Backend (--db)

Install Non-Default Backend

<pm> add @mastra/<backend>

Configure in src/mastra/index.ts

import { LibSQLStore } from '@mastra/libsql'; // or PgStore, TursoStore

export const mastra = new Mastra({
  // ...
  storage: new LibSQLStore({
    /* config */
  }),
});

Browser Agent (--browser-agent)

1. Install Packages

<pm> add @mastra/stagehand @mastra/memory

2. Create Agent

Create src/mastra/agents/browser-agent.ts:

import { Agent } from '@mastra/core/agent';
import { Memory } from '@mastra/memory';
import { StagehandBrowser } from '@mastra/stagehand';

export const browserAgent = new Agent({
  id: 'browser-agent',
  name: 'Browser Agent',
  instructions: `You are a helpful assistant that can browse the web.`,
  model: '<provider>/<model>',
  memory: new Memory(),
  browser: new StagehandBrowser({
    headless: false, // true for cloud deploys
  }),
});

3. Register Agent

Update src/mastra/index.ts:

import { browserAgent } from './agents/browser-agent';

export const mastra = new Mastra({
  agents: { weatherAgent, browserAgent },
  // ...
});

4. Install Playwright

<pm> exec playwright install chromium

5. Runtime requirement: pass thread + resource on every call

Passing browser: new StagehandBrowser(...) to Agent auto-attaches the BrowserContextProcessor input processor. That processor reads/writes browser state via Mastra memory, so every call to the browser agent must provide both a thread and a resource id, or the processor throws:

[Processor:browser-context] computeStateSignal requires Mastra memory with an active resourceId and threadId

Use the memory: { thread, resource } payload shape:

curl -s -X POST 'http://localhost:4111/api/agents/browser-agent/generate' \
  -H 'Content-Type: application/json' \
  -d '{
    "messages":[{"role":"user","content":"Navigate to https://example.com and tell me the page title."}],
    "memory":{"thread":"<tid>","resource":"<rid>"}
  }'

Top-level threadId / resourceId are silently discarded (same as other agents). The Studio chat works without explicit IDs because the chat UI allocates them for you.

Custom API Routes

To add custom API routes:

1. Create Route

Create src/mastra/routes/hello.ts:

import { registerApiRoute } from '@mastra/core/server';

export const helloRoute = registerApiRoute('/hello', {
  method: 'GET',
  requiresAuth: false, // Set to true if auth required
  handler: async c => {
    return c.json({ message: 'Hello from custom route!' });
  },
});

2. Register Route

Update src/mastra/index.ts:

import { helloRoute } from './routes/hello';

export const mastra = new Mastra({
  // ...
  server: {
    apiRoutes: [helloRoute], // ⚠️ Must be "apiRoutes", not "routes"
  },
});

Common mistake: Using routes instead of apiRoutes - this will silently fail.

3. Verify Locally

# Start dev server
<pm> run dev

# Test route
curl http://localhost:4111/hello

Environment Variables

Check/Set LLM API Key

# Check if set
echo $OPENAI_API_KEY  # or ANTHROPIC_API_KEY, etc.

# Or check .env file
cat .env | grep API_KEY

If not set, add to .env:

OPENAI_API_KEY=sk-...

Platform URL (Cloud Only)

# Staging
export MASTRA_PLATFORM_API_URL=https://platform.staging.mastra.ai

# Production (default - can be unset)
unset MASTRA_PLATFORM_API_URL

Verification Checklist

TypeScript Check (Required)

Always run `tsc --noEmit` after modifying config files. This catches most config mistakes that mastra build silently ignores:

• Wrong property names (routes vs apiRoutes)
• Type mismatches (timeout: "30" vs timeout: 30)
• Missing imports
• Unknown properties

<pm> tsc --noEmit
# Record any errors that appear

If errors appear, fix them before proceeding. Don't rely on mastra build or pnpm dev to catch these.

Common Issues

Studio Deploy Testing (--test studio)

Cloud only: For --env staging or --env production.

Purpose

Verify Studio deployment works and UI is accessible.

Prerequisites

• Mastra platform account
• Project with at least one agent
• Authenticated via mastra auth login

Steps

1. Set Environment

# For staging
export MASTRA_PLATFORM_API_URL=https://platform.staging.mastra.ai

# For production (default)
unset MASTRA_PLATFORM_API_URL

2. Authenticate

pnpx mastra@latest auth login

• [ ] Note if browser opens for OAuth
• [ ] Record login flow completion
• [ ] Record CLI authentication confirmation

3. Deploy Studio

pnpx mastra@latest studio deploy -y

Record:

• [ ] Note if build starts
• [ ] Record build completion and any warnings
• [ ] Note if deploy starts
• [ ] Capture Studio URL from output

4. Handle Deploy Output

5. Observe Studio Access

• [ ] Open Studio URL in browser
• [ ] Note if sign-in is prompted
• [ ] Record whether Studio UI loads
• [ ] Record which agents appear in list

6. Test Basic Functionality

• [ ] Navigate to /agents
• [ ] Click on an agent
• [ ] Send a test message
• [ ] Record the response

Observations to Report

Deploy URLs

Common Issues

Studio vs Server: When to Use Each

Typical flow:

1. Deploy Studio first (for UI access) 2. Deploy Server (for API access) 3. Test via both UI and API 4. Check if Server traces appear in Studio

You can deploy one without the other, but:

• Studio-only: No API access, can't test server traces
• Server-only: No UI, must use curl/API clients

Notes

• First deploy may take longer (2-5 minutes)
• Subsequent deploys are faster
• Studio URL persists across deploys
• Check projects.mastra.ai to view all deployments

Tools Testing (--test tools)

Purpose

Verify tools page loads and tool execution works.

Steps

1. Navigate to Tools Page

• [ ] Open /tools in Studio
• [ ] Note if tools list loads and any errors displayed
• [ ] Record which tools appear (e.g., "get-weather")

2. Select a Tool

• [ ] Click on a tool (e.g., get-weather)
• [ ] Note if tool details panel opens
• [ ] Record which input fields are visible

3. Execute Tool

• [ ] Enter test input (e.g., "London" for city field)
• [ ] Click "Submit" or "Run"
• [ ] Wait for execution

4. Observe Output

• [ ] Record the output format (JSON, text, etc.)
• [ ] Record the output content
• [ ] Note any error messages

5. Test Error Handling

• [ ] Enter invalid input (e.g., empty or special characters)
• [ ] Record the error message displayed
• [ ] Note if tool crashes or handles gracefully

Observations to Report

Common Issues

Browser Actions

Navigate to: /tools
Click: First tool in list (e.g., get-weather)
Type in input field: "London"
Click: Submit button
Wait: For output
Verify: JSON output appears

Curl / API (for --skip-browser)

`<toolId>` is the tool's `.id` property, not the export name. A weatherTool export with id: 'get-weather' is addressed as /api/tools/get-weather/execute, not /api/tools/weatherTool/execute.

List tools:

curl -s http://localhost:4111/api/tools

Execute a tool:

curl -s -X POST "http://localhost:4111/api/tools/<toolId>/execute" \
  -H "Content-Type: application/json" \
  -d '{"data":{"location":"San Francisco"}}'

Note the data wrapper — the tool's input schema fields go inside data, not at the top level.

Pass criteria:

• /api/tools returns a JSON object keyed by tool id
• /execute with valid input returns HTTP 200 with the tool's result object
• /execute with invalid input returns HTTP 200 with

{ error: true, validationErrors: {...} } (see errors.md)

Common mistakes:

• Using the export name (e.g. weatherTool) instead of the tool's id

(e.g. get-weather) → 404 "Tool not found"

• Sending input fields at the top level instead of under data → validation

error on every field

• External API failures surface as HTTP 500 with upstream error content.

Retry with a different input (e.g. a well-known city) before concluding the tool itself is broken.

Traces Testing (--test traces)

Purpose

Verify observability traces are being collected and displayed.

Prerequisites

• Must have run agent/tool/workflow tests first (to generate traces)
• For cloud: Both Studio and Server need to be deployed

Steps

1. Navigate to Observability

• [ ] Open /observability in Studio
• [ ] Note if page loads and any errors displayed
• [ ] Record existing traces shown

2. Observe Studio-Originated Traces

• [ ] Look for traces from previous tests (agent chat, tool runs)
• [ ] Record what information traces show (name, timestamp, duration, status)
• [ ] Click on a trace to expand details

3. Check Trace Details

• [ ] Record what input/output is shown
• [ ] Note timing information displayed
• [ ] Record any error states shown

4. Generate Server Trace (Cloud Only)

For --env staging or --env production:

curl -X POST <server-url>/api/agents/weather-agent/generate \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Weather in Paris?"}]}'

• [ ] Execute the curl command
• [ ] Record the response

5. Check for Server Trace

• [ ] Refresh /observability page
• [ ] Note if new trace from Server API call appears
• [ ] Record how long until trace appears (if at all)

Observations to Report

Trace Sources

Troubleshooting

Local vs Cloud

Local (`--env local`):

• Traces stored in-memory
• Only persist while dev server runs
• Check @mastra/observability is installed

Cloud (`--env staging/production`):

• Traces sent to cloud collector
• Persist across sessions
• Note if both Studio and Server traces appear

Curl / API (for --skip-browser, local)

The same /api/observability/traces endpoint works locally (no auth needed):

# List recent spans (response shape: { pagination, spans })
curl -s "http://localhost:4111/api/observability/traces?page=0&perPage=20" | jq '.'

# Get a specific trace by id (captured from a prior agent/workflow response)
curl -s "http://localhost:4111/api/observability/traces/<traceId>" | jq '.'

Response shape: GET /api/observability/traces returns { pagination: { total, page, perPage, hasMore }, spans: [...] } — not a bare array and not a traces key. Each entry in spans has spanType (agent_run, tool_call, workflow_run, scorer_run), traceId, timestamps, and payload.

Quick pass check:

curl -s "http://localhost:4111/api/observability/traces?page=0&perPage=100" | \
  jq '{total: .pagination.total, byType: ([.spans[].spanType] | group_by(.) | map({t: .[0], n: length}))}'

Pass criteria (local):

• After running agent / tool / workflow tests, .pagination.total > 0
• .spans contains the expected spanTypes: agent_run, workflow_run,

scorer_run (and tool_call if the agent invoked a tool)

• traceId values returned in earlier generate/workflow responses resolve

via /observability/traces/:traceId

Note: local traces are in-memory only (MastraStorageExporter). They disappear on dev server restart. Run the agent/tool/workflow tests in the same dev server session as the traces test.

Direct Trace API (Cloud Only)

If UI traces aren't appearing but you need to verify the trace pipeline:

Mobs-Query URLs

Get Auth Token

Credentials are stored in ~/.mastra/credentials.json after mastra auth login:

{
  "token": "eyJhbG...", // Access token (5 min expiry)
  "refreshToken": "eyJhbG...", // Refresh token (long-lived)
  "user": { "id": "...", "email": "..." },
  "organizationId": "org_01KN...",
  "currentOrgId": "org_01KN..."
}

Token Refresh Helper

WorkOS tokens expire in 5 minutes. Use this helper to auto-refresh:

get_valid_token() {
  local PLATFORM_URL="${1:-https://platform.mastra.ai}"
  local TOKEN=$(jq -r '.token' ~/.mastra/credentials.json)
  local ORG_ID=$(jq -r '.currentOrgId // .organizationId' ~/.mastra/credentials.json)

  # Try current token
  local VERIFY=$(curl -s "$PLATFORM_URL/v1/auth/verify" \
    -H "Authorization: Bearer $TOKEN" \
    -H "x-organization-id: $ORG_ID")

  if echo "$VERIFY" | jq -e '.user' > /dev/null 2>&1; then
    echo "$TOKEN"
    return 0
  fi

  # Token expired — try refresh
  local REFRESH_TOKEN=$(jq -r '.refreshToken' ~/.mastra/credentials.json)
  if [ -z "$REFRESH_TOKEN" ] || [ "$REFRESH_TOKEN" = "null" ]; then
    echo "No refresh token. Re-login required." >&2
    return 1
  fi

  local REFRESH_RESULT=$(curl -s "$PLATFORM_URL/v1/auth/refresh-token" \
    -X POST \
    -H "Content-Type: application/json" \
    -d "{\"refreshToken\": \"$REFRESH_TOKEN\"}")

  if echo "$REFRESH_RESULT" | jq -e '.accessToken' > /dev/null 2>&1; then
    local NEW_TOKEN=$(echo "$REFRESH_RESULT" | jq -r '.accessToken')
    local NEW_REFRESH=$(echo "$REFRESH_RESULT" | jq -r '.refreshToken')

    # Update credentials file
    jq --arg t "$NEW_TOKEN" --arg r "$NEW_REFRESH" \
      '.token = $t | .refreshToken = $r' \
      ~/.mastra/credentials.json > ~/.mastra/credentials.json.tmp \
      && mv ~/.mastra/credentials.json.tmp ~/.mastra/credentials.json

    echo "$NEW_TOKEN"
    return 0
  fi

  echo "Refresh failed. Re-login required." >&2
  return 1
}

# Usage
TOKEN=$(get_valid_token "https://platform.mastra.ai") || exit 1

Query Traces

# Get project info from config file
PROJECT_ID=$(jq -r '.projectId' .mastra-project.json)  # or .mastra-project-staging.json
ORG_ID=$(jq -r '.organizationId' .mastra-project.json)
TOKEN=$(get_valid_token "https://platform.mastra.ai")

# Production
curl -s "https://mobs-query-vgvrl5lbxq-uc.a.run.app/api/observability/traces?page=0&perPage=10&resourceId=$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" \
  -H "x-organization-id: $ORG_ID" | jq '.'

# Staging
TOKEN=$(get_valid_token "https://platform.staging.mastra.ai")
curl -s "https://mobs-query-pvyw2kfhjq-uc.a.run.app/api/observability/traces?page=0&perPage=10&resourceId=$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" \
  -H "x-organization-id: $ORG_ID" | jq '.'

Trace Response Structure

{
  "pagination": { "total": 10, "page": 0, "perPage": 10, "hasMore": false },
  "traces": [
    {
      "traceId": "37f0d68d760887e994135c984ebd7b89",
      "name": "agent run: 'weather-agent'",
      "spanType": "agent_run",
      "startedAt": "2026-04-08T15:29:27.123Z",
      "endedAt": "2026-04-08T15:29:30.456Z",
      "metadata": { "buildId": "...", "runId": "..." },
      "requestContext": { "user": { "id": "...", "email": "..." } },
      "status": "success"
    }
  ]
}

Filter Traces

# By time range (URL-encoded JSON)
curl -s "...?startedAt=%7B%22start%22%3A%222026-04-08T15%3A00%3A00.000Z%22%7D" ...

# By resource ID (project)
curl -s "...?resourceId=$PROJECT_ID" ...

# By run ID
curl -s "...?runId=$RUN_ID" ...

Browser Actions

Navigate to: /observability
Wait: For traces to load
Verify: At least one trace visible
Click: On a trace row
Verify: Details panel shows input/output

# For cloud only:
Execute: curl command to server
Navigate to: /observability
Click: Refresh or wait
Verify: New server trace appears

Workflows Testing (--test workflows)

Purpose

Verify workflows page loads and workflow execution works.

Steps

1. Navigate to Workflows Page

• [ ] Open /workflows in Studio
• [ ] Note if workflows list loads and any errors displayed
• [ ] Record which workflows appear

2. Select a Workflow

• [ ] Click on a workflow (e.g., weather-workflow)
• [ ] Note if workflow details/run panel opens
• [ ] Record which input fields are visible (if any)

3. Execute Workflow

• [ ] Enter the required input (e.g., "Berlin" for city)
• [ ] Click "Run" or "Execute"
• [ ] Wait for workflow to complete

4. Observe Execution

• [ ] Record the workflow state shown (Running, etc.)
• [ ] Note completion status (success, failure, steps shown)
• [ ] Record output/result displayed

5. Check Workflow Steps

• [ ] Record which individual steps executed
• [ ] Note step-by-step output if available
• [ ] Record the final result

Observations to Report

Common Issues

Browser Actions

Navigate to: /workflows
Click: First workflow in list
Type in input (if required): "Berlin"
Click: Run button
Wait: For completion
Verify: Success state and output

Curl / API (for --skip-browser)

`<workflowId>` is the workflow's registered id (the key used in Mastra({ workflows: { weatherWorkflow } }) or the workflow's .id, depending on how it was registered).

List workflows:

curl -s http://localhost:4111/api/workflows

Run a workflow synchronously:

curl -s -X POST "http://localhost:4111/api/workflows/<workflowId>/start-async" \
  -H "Content-Type: application/json" \
  -d '{"inputData":{"city":"Tokyo"}}'

Note the inputData wrapper — the workflow's input schema fields go inside inputData, not at the top level.

Pass criteria:

• /api/workflows returns a JSON object keyed by workflow id
• /start-async returns HTTP 200 with a result object containing the final

workflow output and a run id

• Response typically includes a traceId — capture it to cross-reference in

traces.md verification

Common mistakes:

• Sending input fields at the top level instead of under inputData →

HTTP 500 "Invalid input data: <field> expected <type>, received undefined"

• Using the workflow's display name instead of its registered id → 404

"Workflow not found"