Positron E2e Tests

Positron Playwright E2E Testing

Purpose

Provides specialized knowledge and patterns for writing correct, reliable Playwright e2e tests that follow Positron's established conventions and avoid common mistakes.

When to Use This Skill

Load this skill when:

• Creating new e2e test files
• Adding test cases to existing test files
• Debugging flaky or failing tests
• Understanding the test fixture system
• Working with page objects
• Choosing correct selectors and assertions

Critical: Test File Structure

Every test file MUST follow this structure:

import { test, expect, tags } from '../_test.setup';

// REQUIRED: Each test file needs a unique suiteId
test.use({
	suiteId: __filename
});

test.describe('Feature Name', {
	tag: [tags.WEB, tags.WIN, tags.CRITICAL, tags.FEATURE_TAG]
}, () => {

	test.beforeEach(async function ({ app }) {
		// Optional setup for each test
	});

	test.afterEach(async function ({ app, hotKeys }) {
		// Cleanup after each test
		await hotKeys.closeAllEditors();
	});

	test('Test description', async function ({ app, python }) {
		// Test implementation
	});
});

MANDATORY REQUIREMENTS: 1. Import from ../_test.setup - NOT from @playwright/test 2. Set suiteId: __filename - Required for app isolation 3. Use function syntax for tests (not arrow functions) - Required for fixtures 4. Add appropriate tags for platform filtering

Quick Reference: Available Fixtures

See references/fixtures.md for complete fixture documentation.

Quick Reference: Page Objects

Access via app.workbench.*:

const { console, variables, dataExplorer, plots, notebooks, sessions } = app.workbench;

// Execute code
await console.executeCode('Python', 'x = 1');

// Wait for content
await console.waitForConsoleContents('expected text');

// Variable interaction
await variables.doubleClickVariableRow('df');

// Data explorer
await dataExplorer.grid.verifyTableData([{ col: 'value' }]);

See references/page-objects.md for complete page object documentation.

Quick Reference: Assertions

// Visibility with timeout
await expect(locator).toBeVisible({ timeout: 30000 });

// Text content
await expect(locator).toHaveText('expected');
await expect(locator).toContainText('partial');

// Count
await expect(locator).toHaveCount(3, { timeout: 15000 });

// Retry pattern for flaky operations
await expect(async () => {
	await someAction();
	await expect(resultLocator).toBeVisible();
}).toPass({ timeout: 15000 });

See references/assertions.md for complete assertion patterns.

Quick Reference: Test Tags

Feature tags (what the test covers):

• tags.CONSOLE, tags.DATA_EXPLORER, tags.NOTEBOOKS, tags.PLOTS, tags.VARIABLES
• tags.CRITICAL - High priority tests

Platform tags (where the test runs):

• tags.WEB - Enable web browser testing
• tags.WIN - Enable Windows testing
• Default: Linux/Electron only

test.describe('Console Tests', {
	tag: [tags.WEB, tags.WIN, tags.CRITICAL, tags.CONSOLE]
}, () => { ... });

Common Mistakes to Avoid

Critical (will break tests): 1. Wrong imports - use ../_test.setup, not @playwright/test 2. Missing `suiteId` - must have test.use({ suiteId: __filename }) 3. Arrow functions - use function syntax, not async ({ app }) => 4. Missing platform tags - add tags.WEB, tags.WIN for cross-platform

Quality issues: 5. No timeout on assertions - use { timeout: 30000 } for async operations 6. No `test.step()` - wrap complex multi-action sequences for better reports

See references/common-mistakes.md for 26 detailed gotchas with code examples.

Running Tests

# Run specific test file
npx playwright test <test-name>.test.ts --project e2e-electron

# Run all tests in a category
npx playwright test test/e2e/tests/<category>/

# Run with specific tags
npx playwright test --grep @:critical

# Run in headed mode (see browser)
npx playwright test --headed

# Run with debug mode
npx playwright test --debug

# Show test report
npx playwright show-report

Progressive Documentation

For detailed information, read the bundled reference docs:

• `references/test-structure.md` - Complete test file structure and organization
• `references/fixtures.md` - All available fixtures and their usage
• `references/page-objects.md` - Page object patterns and available POMs
• `references/assertions.md` - Assertion patterns and waiting strategies
• `references/common-mistakes.md` - Comprehensive list of gotchas to avoid

Key Architecture Principles

1. Worker-scoped app - One app instance per test file (suite) 2. Test-scoped fixtures - page, sessions, etc. fresh per test 3. Page Object Model - UI interactions wrapped in POMs via app.workbench.* 4. Tag-based filtering - Tests tagged for platform and feature filtering 5. Automatic cleanup - Tracing, screenshots attached on failure

Getting Help

1. Look at existing tests in test/e2e/tests/<feature>/ for patterns 2. Check page object source in test/e2e/pages/ for available methods 3. Read test/e2e/tests/_test.setup.ts for fixture definitions 4. Use --debug flag to step through tests interactively

Assertions and Waiting Patterns

Complete guide to assertions, waits, and reliability patterns in Positron e2e tests.

Basic Assertions

Visibility Assertions

// Element visible
await expect(locator).toBeVisible();
await expect(locator).toBeVisible({ timeout: 30000 });

// Element hidden/not visible
await expect(locator).toBeHidden();
await expect(locator).not.toBeVisible();
await expect(locator).toBeHidden({ timeout: 15000 });

Text Assertions

// Exact text match
await expect(locator).toHaveText('exact text');
await expect(locator).toHaveText('exact text', { timeout: 15000 });

// Contains text
await expect(locator).toContainText('partial');
await expect(locator).toContainText(/regex pattern/);

// Multiple elements text
await expect(locator).toHaveText(['item1', 'item2', 'item3']);

Count Assertions

// Exact count
await expect(locator).toHaveCount(3);
await expect(locator).toHaveCount(3, { timeout: 15000 });

// At least one
await expect(locator).toHaveCount(1);

// None (element doesn't exist)
await expect(locator).toHaveCount(0);

Attribute Assertions

// Has attribute with value
await expect(locator).toHaveAttribute('aria-label', 'Expected Label');
await expect(locator).toHaveAttribute('data-state', 'active');

// Attribute matches regex
await expect(locator).toHaveAttribute('aria-label', /Python/);

// Has class
await expect(locator).toHaveClass(/selected/);
await expect(locator).toHaveClass('my-class');

Value Assertions

// Input value
await expect(input).toHaveValue('expected value');
await expect(input).toHaveValue(/partial/);

// Checkbox/Radio checked
await expect(checkbox).toBeChecked();
await expect(checkbox).not.toBeChecked();

// Element enabled/disabled
await expect(button).toBeEnabled();
await expect(button).toBeDisabled();

Waiting Patterns

Wait for Condition with toPass

The most important pattern for handling timing issues:

// Retry until assertion passes
await expect(async () => {
	await someAction();
	await expect(result).toBeVisible();
}).toPass({ timeout: 15000 });

// With multiple assertions
await expect(async () => {
	const count = await locator.count();
	expect(count).toBeGreaterThan(0);

	const text = await locator.first().textContent();
	expect(text).toContain('expected');
}).toPass({ timeout: 30000 });

When to use `toPass`:

• Operations that may need retries (clicking, typing)
• Actions that have race conditions
• Cleanup operations that may not work first time
• Any operation where timing is unpredictable

expect.poll

Poll a function until condition is met:

// Poll for count
await expect.poll(async () => {
	return (await locator.all()).length;
}).toBeGreaterThan(2);

// Poll for value
await expect.poll(async () => {
	return await getValue();
}, { timeout: 30000 }).toBe('expected');

// Poll with intervals
await expect.poll(async () => {
	return await checkStatus();
}, {
	timeout: 60000,
	intervals: [1000, 2000, 5000]  // Check at 1s, 2s, 5s intervals
}).toBe('ready');

Wait for Network/State

// Wait for navigation
await page.waitForURL('**/expected-path');

// Wait for load state
await page.waitForLoadState('domcontentloaded');
await page.waitForLoadState('networkidle');

// Wait for response
await page.waitForResponse(response =>
	response.url().includes('/api/data') && response.status() === 200
);

Wait for Element State

// Wait for element to exist
await locator.waitFor();
await locator.waitFor({ state: 'attached' });

// Wait for element to be visible
await locator.waitFor({ state: 'visible', timeout: 30000 });

// Wait for element to be hidden
await locator.waitFor({ state: 'hidden' });

// Wait for element to be detached from DOM
await locator.waitFor({ state: 'detached' });

Timeout Guidelines

Default Timeouts

• Assertion timeout: 15 seconds (Playwright default)
• Action timeout: 30 seconds (in page objects)
• Test timeout: 2 minutes (configured in playwright.config.ts)

Recommended Timeout Values

Setting Timeouts

// Per-assertion timeout
await expect(locator).toBeVisible({ timeout: 30000 });

// Per-action timeout
await locator.click({ timeout: 10000 });

// toPass timeout
await expect(async () => {
	// ...
}).toPass({ timeout: 15000 });

Locator Strategies

Preferred Selectors (Most to Least Reliable)

1. Test IDs (most stable)

page.getByTestId('restart-session')
page.getByTestId('data-grid-cell-0-0')

2. Accessible Labels

page.getByLabel('Clear console')
page.getByRole('button', { name: 'Execute' })
page.getByRole('tab', { name: 'Console', exact: true })

3. Text Content

page.getByText('Python')
page.getByText(/started/)
locator.filter({ hasText: 'expected' })

4. CSS Selectors (less stable, but sometimes necessary)

page.locator('.monaco-workbench')
page.locator('[id="workbench.panel.positronSession"]')

Combining Locators

// Filter by text
page.locator('.console-instance').filter({ hasText: 'Python' })

// Chain locators
page.locator('.variable-item').locator('.name-column')

// Has descendant
page.locator('.cell').filter({ has: page.getByText('output') })

// Nth element
page.locator('.item').nth(0)
page.locator('.item').first()
page.locator('.item').last()

Frame Locators (Webviews)

// Single frame
page.frameLocator('.webview').locator('.content')

// Nested frames
page.frameLocator('.webview').frameLocator('#active-frame').locator('.output')

Common Assertion Patterns

Verify Console Output

// Wait for specific text
await app.workbench.console.waitForConsoleContents('expected output');

// Wait for regex match
await app.workbench.console.waitForConsoleContents(/Python.*started/);

// Wait for exact count of matches
await app.workbench.console.waitForConsoleContents('success', {
	expectedCount: 2,
	timeout: 30000
});

// Verify text does NOT appear
await app.workbench.console.waitForConsoleContents('error', {
	expectedCount: 0,
	timeout: 5000
});

Verify Data Explorer Contents

await app.workbench.dataExplorer.grid.verifyTableData([
	{ 'Name': 'Alice', 'Age': '30', 'City': 'NYC' },
	{ 'Name': 'Bob', 'Age': '25', 'City': 'LA' }
], 60000);

Verify Variable Exists

await app.workbench.variables.waitForVariable('df');
await app.workbench.variables.waitForVariableValue('x', '42');

Verify Editor Tab

await app.workbench.editors.verifyTab('Data: df', { isVisible: true });
await app.workbench.editors.verifyTab('script.py', { isActive: true });

Verify Files Created

await app.workbench.explorer.verifyExplorerFilesExist([
	'output.csv',
	'plot.png'
]);

Retry Patterns for Flaky Operations

Click with Retry

await expect(async () => {
	await button.click();
	await expect(dialog).toBeVisible();
}).toPass({ timeout: 10000 });

Clear with Retry (for plots, editors, etc.)

await expect(async () => {
	await hotKeys.clearPlots();
	await app.workbench.plots.waitForNoPlots({ timeout: 3000 });
}).toPass({ timeout: 15000 });

Menu Interaction with Retry

await expect(async () => {
	await menuTrigger.click();
	await expect(menuItem).toBeVisible();
}).toPass({ timeout: 5000 });

await menuItem.click();

Dialog with Retry

await expect(async () => {
	if (!await dialog.isVisible()) {
		await triggerButton.click();
	}
	await expect(dialog).toBeVisible();
}).toPass({ timeout: 5000 });

Negative Assertions

Element Should NOT Appear

// Check element doesn't exist
await expect(locator).toHaveCount(0);

// Check element hidden
await expect(locator).toBeHidden({ timeout: 5000 });

// Wait for disappearance
await locator.waitFor({ state: 'hidden', timeout: 10000 });

Error Should NOT Occur

// Verify no error toast
await app.workbench.toasts.verifyNoToasts();

// Verify no console error
await app.workbench.console.waitForConsoleContents('Error', {
	expectedCount: 0,
	timeout: 5000
});

Soft Assertions

For non-critical checks that shouldn't fail the test:

// Soft assertion (continues even if fails)
await expect.soft(locator).toBeVisible();
await expect.soft(locator).toHaveText('expected');

// Check if any soft assertions failed
expect(test.info().errors).toHaveLength(0);

Debugging Failed Assertions

Add Context to Assertions

// Custom error message
await expect(locator, 'Dialog should be visible after clicking button').toBeVisible();

// Using test.step for context
await test.step('Open settings dialog', async () => {
	await settingsButton.click();
	await expect(settingsDialog).toBeVisible();
});

Screenshots on Failure

Automatic - Playwright captures screenshots on assertion failure.

Manual Debugging

// Pause execution
await page.pause();

// Log locator info
console.log(await locator.count());
console.log(await locator.textContent());
console.log(await locator.isVisible());

// Take manual screenshot
await page.screenshot({ path: 'debug.png' });

Common Mistakes and Gotchas

Comprehensive list of mistakes to avoid when writing Positron e2e tests.

Critical Mistakes

1. Wrong Import Source

WRONG:

import { test, expect } from '@playwright/test';

CORRECT:

import { test, expect, tags } from '../_test.setup';

The custom _test.setup provides all Positron fixtures. Using the raw Playwright import will cause fixture errors.

2. Missing suiteId

WRONG:

test.describe('Console Tests', () => {
	test('my test', async function ({ app }) {
		// ...
	});
});

CORRECT:

test.use({
	suiteId: __filename
});

test.describe('Console Tests', () => {
	test('my test', async function ({ app }) {
		// ...
	});
});

Without suiteId:

• Tests may share app instances incorrectly
• Logs won't be organized by test file
• beforeAll/afterAll won't work as expected

3. Arrow Functions Instead of Function Syntax

WRONG:

test('my test', async ({ app, python }) => {
	// ...
});

test.beforeEach(async ({ app }) => {
	// ...
});

CORRECT:

test('my test', async function ({ app, python }) {
	// ...
});

test.beforeEach(async function ({ app }) {
	// ...
});

The codebase consistently uses function syntax. While arrow functions sometimes work, they can cause issues with fixture access and this-binding.

4. Forgetting Tags for Cross-Platform Tests

WRONG:

test.describe('Console Tests', () => {
	// Only runs on Linux/Electron
});

CORRECT:

test.describe('Console Tests', {
	tag: [tags.WEB, tags.WIN, tags.CONSOLE]
}, () => {
	// Runs on web, Windows, and Linux/Electron
});

Without platform tags:

• tags.WEB - test won't run in web browser mode
• tags.WIN - test won't run on Windows

Fixture Mistakes

5. Using python/r Fixture Without Understanding Scope

WRONG (misunderstanding):

test('test 1', async function ({ python }) {
	// Python starts
});

test('test 2', async function ({ app }) {
	// Assuming Python is still running - IT'S NOT GUARANTEED
	await app.workbench.console.executeCode('Python', 'x = 1');
});

CORRECT:

test('test 2', async function ({ app, python }) {
	// python fixture ensures Python is running
	await app.workbench.console.executeCode('Python', 'x = 1');
});

// Or use sessions for manual control
test('test 2', async function ({ app, sessions }) {
	await sessions.start('python', { reuse: true });
	await app.workbench.console.executeCode('Python', 'x = 1');
});

6. Wrong Settings Fixture Scope

WRONG:

test('my test', async function ({ settings }) {
	await settings.set({ 'key': 'value' });  // Settings is worker-scoped!
});

CORRECT:

test.beforeAll(async ({ settings }) => {
	await settings.set({ 'key': 'value' });
});

settings is worker-scoped (shared across tests in a file). Setting it per-test can cause unexpected behavior.

7. Mixing Up Fixture Dependencies

WRONG:

test('test', async function ({ page, sessions }) {
	await sessions.start('python');
	// page is derived from app, but you might not have app in scope
});

CORRECT:

test('test', async function ({ app, sessions }) {
	await sessions.start('python');
	const page = app.code.driver.page;  // Access page from app
});

// Or use page fixture directly
test('test', async function ({ page, sessions }) {
	await sessions.start('python');
	await expect(page.getByText('Python')).toBeVisible();
});

Assertion Mistakes

8. Missing Timeouts on Async Assertions

WRONG:

await expect(locator).toBeVisible();  // Uses default 5s timeout

CORRECT:

await expect(locator).toBeVisible({ timeout: 30000 });

Default timeout is often too short for:

• Interpreter startup
• Code execution
• Data loading
• Network operations

9. Not Using toPass for Flaky Operations

WRONG:

await hotKeys.clearPlots();
await app.workbench.plots.waitForNoPlots();
// May fail if clear didn't work first time

CORRECT:

await expect(async () => {
	await hotKeys.clearPlots();
	await app.workbench.plots.waitForNoPlots({ timeout: 3000 });
}).toPass({ timeout: 15000 });

Use toPass for:

• Clear/cleanup operations
• Menu interactions
• Dialog triggers
• Any operation that may need retry

10. Wrong Element Count Assertion

WRONG:

// Checking if element exists
await expect(locator).toBeVisible();  // Fails if multiple match

// Checking if element doesn't exist
await expect(locator).not.toBeVisible();  // May pass if one is hidden

CORRECT:

// Element should exist (one or more)
await expect(locator).toHaveCount(1, { timeout: 15000 });

// Element should not exist
await expect(locator).toHaveCount(0, { timeout: 5000 });

Locator Mistakes

11. Using Unstable Selectors

WRONG:

page.locator('.monaco-list-row:nth-child(3)')
page.locator('div > div > span.text')
page.locator('[style*="z-index: 1"]')

CORRECT:

page.getByTestId('specific-element')
page.getByLabel('Button Name')
page.getByRole('button', { name: 'Submit' })
page.locator('.well-named-class').filter({ hasText: 'Expected' })

Prefer (in order): 1. Test IDs 2. Accessible roles/labels 3. Text content 4. Stable class names

12. Not Scoping Locators

WRONG:

// Finds ALL buttons on page
await page.getByRole('button', { name: 'OK' }).click();

CORRECT:

// Scoped to specific dialog
const dialog = page.locator('.my-dialog');
await dialog.getByRole('button', { name: 'OK' }).click();

// Or use filter
await page.getByRole('button', { name: 'OK' })
	.filter({ has: page.locator('.dialog-footer') })
	.click();

13. Forgetting Exact Match

WRONG:

page.getByText('Console')  // Matches "Console", "Console Tab", "Active Console"

CORRECT:

page.getByText('Console', { exact: true })
page.getByRole('tab', { name: 'Console', exact: true })

Test Structure Mistakes

14. No Cleanup in afterEach

WRONG:

test.describe('Tests', () => {
	test('test 1', async function ({ app }) {
		await app.workbench.variables.doubleClickVariableRow('df');
		// Opens data explorer tab
	});

	test('test 2', async function ({ app }) {
		// Data explorer tab still open - may interfere
	});
});

CORRECT:

test.describe('Tests', () => {
	test.afterEach(async function ({ hotKeys }) {
		await hotKeys.closeAllEditors();
	});

	test('test 1', async function ({ app }) {
		await app.workbench.variables.doubleClickVariableRow('df');
	});

	test('test 2', async function ({ app }) {
		// Clean slate
	});
});

15. Tests Depending on Order

WRONG:

test('step 1 - create file', async function ({ app }) {
	// Creates file
});

test('step 2 - use file', async function ({ app }) {
	// Assumes file from test 1 exists - BAD
});

CORRECT:

test('complete workflow', async function ({ app }) {
	await test.step('Create file', async () => {
		// Create file
	});

	await test.step('Use file', async () => {
		// Use file
	});
});

// Or use beforeEach to set up state
test.beforeEach(async function ({ app }) {
	// Create file for each test
});

16. Not Using test.step for Complex Tests

WRONG:

test('full workflow', async function ({ app, python }) {
	await app.workbench.console.executeCode('Python', 'x = 1');
	await app.workbench.console.executeCode('Python', 'y = 2');
	await app.workbench.console.executeCode('Python', 'df = create_df()');
	await app.workbench.variables.doubleClickVariableRow('df');
	await app.workbench.dataExplorer.grid.verifyTableData(expected);
});

CORRECT:

test('full workflow', async function ({ app, python }) {
	await test.step('Set up variables', async () => {
		await app.workbench.console.executeCode('Python', 'x = 1');
		await app.workbench.console.executeCode('Python', 'y = 2');
	});

	await test.step('Create dataframe', async () => {
		await app.workbench.console.executeCode('Python', 'df = create_df()');
	});

	await test.step('Open in data explorer', async () => {
		await app.workbench.variables.doubleClickVariableRow('df');
	});

	await test.step('Verify data', async () => {
		await app.workbench.dataExplorer.grid.verifyTableData(expected);
	});
});

Timing Mistakes

17. Hard-Coded Waits

WRONG:

await page.waitForTimeout(5000);  // Just waiting...
await button.click();

CORRECT:

await expect(button).toBeEnabled({ timeout: 5000 });
await button.click();

// Or wait for specific state
await page.waitForLoadState('networkidle');
await button.click();

18. Not Waiting for Console Ready

WRONG:

test('execute code', async function ({ sessions, app }) {
	await sessions.start('python');
	await app.workbench.console.executeCode('Python', 'x = 1');
	// May fail if console not ready
});

CORRECT:

test('execute code', async function ({ python, app }) {
	// python fixture waits for ready state
	await app.workbench.console.executeCode('Python', 'x = 1');
});

// Or manually wait
test('execute code', async function ({ sessions, app }) {
	await sessions.start('python');
	await app.workbench.console.waitForReady('>>>');
	await app.workbench.console.executeCode('Python', 'x = 1');
});

19. Race Conditions with UI State

WRONG:

await triggerButton.click();
await dialogContent.textContent();  // Dialog may not be open yet

CORRECT:

await triggerButton.click();
await expect(dialog).toBeVisible({ timeout: 5000 });
const content = await dialogContent.textContent();

Environment Mistakes

20. Hardcoding Interpreter Versions

WRONG:

await notebooks.selectInterpreter('Python', 'Python 3.11.5');

CORRECT:

await notebooks.selectInterpreter('Python', process.env.POSITRON_PY_VER_SEL!);
await notebooks.selectInterpreter('R', process.env.POSITRON_R_VER_SEL!);

Environment variables ensure tests work across different CI environments.

21. Platform-Specific Code Without Guards

WRONG:

await page.keyboard.press('Meta+C');  // Only works on macOS

CORRECT:

if (process.platform === 'darwin') {
	await page.keyboard.press('Meta+C');
} else {
	await page.keyboard.press('Control+C');
}

// Or use hotKeys which handles this
await hotKeys.copy();

22. Headless-Only Operations

WRONG:

await app.workbench.plots.copyCurrentPlotToClipboard();
// Clipboard doesn't work in some headless environments

CORRECT:

const headless = process.env.HEADLESS === 'true';
if (!headless) {
	await app.workbench.plots.copyCurrentPlotToClipboard();
}

Page Object Mistakes

23. Direct Page Manipulation Instead of POM

WRONG:

await page.locator('.console-input').click();
await page.keyboard.type('print("hello")');
await page.keyboard.press('Enter');

CORRECT:

await app.workbench.console.pasteCodeToConsole('print("hello")', true);

// Or
await app.workbench.console.executeCode('Python', 'print("hello")');

Page objects encapsulate:

• Correct selectors
• Wait states
• Retry logic
• Platform handling

24. Not Checking Page Object Methods First

Before writing custom locator code, check if a page object method exists:

// Check test/e2e/pages/*.ts for available methods
// Most common operations are already implemented

// Instead of custom code, use:
await app.workbench.console.executeCode(...)
await app.workbench.variables.doubleClickVariableRow(...)
await app.workbench.dataExplorer.grid.verifyTableData(...)
await app.workbench.plots.waitForCurrentPlot()
await app.workbench.notebooks.selectInterpreter(...)

Debugging Mistakes

25. Not Using --headed or --debug

When tests fail, always try:

# See what's happening
npx playwright test my-test.test.ts --headed

# Step through interactively
npx playwright test my-test.test.ts --debug

26. Not Checking Test Reports

After failures:

npx playwright show-report

Reports include:

• Screenshots at failure
• Traces (if enabled)
• Step-by-step execution
• Error messages with context

Summary: Pre-Submit Checklist

Before submitting a test, verify:

• [ ] Imports from ../_test.setup
• [ ] Has test.use({ suiteId: __filename })
• [ ] Uses function syntax (not arrow functions)
• [ ] Has appropriate tags (tags.WEB, tags.WIN, feature tag)
• [ ] All assertions have explicit timeouts for async operations
• [ ] Uses toPass for potentially flaky operations
• [ ] Has cleanup in afterEach
• [ ] Uses environment variables for interpreter versions
• [ ] Uses page object methods instead of raw locators where possible
• [ ] Test steps are wrapped in test.step() for complex tests
• [ ] Test is independent (doesn't rely on other tests)

Test Fixtures

Complete documentation of all available fixtures in Positron's e2e test system.

Fixture Scopes

Fixtures have two scopes:

• Worker-scoped: One instance per test file (shared across tests)
• Test-scoped: Fresh instance for each test

Core Fixtures

app (Worker-scoped)

The main application instance. Provides access to all page objects.

test('example', async function ({ app }) {
	// Access workbench page objects
	await app.workbench.console.executeCode('Python', 'x = 1');
	await app.workbench.variables.doubleClickVariableRow('x');

	// Access workspace path
	const workspacePath = app.workspacePathOrFolder;

	// Access page directly
	const page = app.code.driver.page;
});

page (Test-scoped)

Shorthand for app.code.driver.page. Direct Playwright Page access.

test('example', async function ({ page }) {
	// Direct locator access
	await page.getByLabel('Start Interpreter').click();
	await expect(page.getByText('Python')).toBeVisible();
});

sessions (Test-scoped)

Session/interpreter management.

test('example', async function ({ sessions }) {
	// Start specific interpreter
	await sessions.start('python');
	await sessions.start('r');
	await sessions.start('pythonAlt');  // Alternate Python version
	await sessions.start('rAlt');       // Alternate R version

	// Start with options
	await sessions.start('python', { reuse: true });

	// Wait for all sessions to be ready
	await sessions.expectAllSessionsToBeReady({ timeout: 15000 });

	// Check session status
	await sessions.expectStatusToBe('session-id', 'idle');
});

python (Test-scoped)

Auto-starts Python interpreter before test runs.

test('Python test', async function ({ app, python }) {
	// Python interpreter already started
	// Console is ready with >>> prompt
	await app.workbench.console.executeCode('Python', 'print("hello")');
});

r (Test-scoped)

Auto-starts R interpreter before test runs.

test('R test', async function ({ app, r }) {
	// R interpreter already started
	// Console is ready with > prompt
	await app.workbench.console.executeCode('R', 'print("hello")');
});

File Operation Fixtures

openFile (Test-scoped)

Opens a file from the workspace.

test('example', async function ({ openFile }) {
	// Path relative to qa-example-content
	await openFile('workspaces/basic-rmd-file/basicRmd.rmd');

	// With wait option
	await openFile('workspaces/test/file.py', true);  // Wait for focus
});

openDataFile (Test-scoped)

Opens a data file (for data explorer).

test('example', async function ({ openDataFile }) {
	await openDataFile('workspaces/large_r_notebook/spotify.ipynb');
});

openFolder (Test-scoped)

Opens a folder.

test('example', async function ({ openFolder }) {
	await openFolder('qa-example-content/workspaces/r_testing');
});

Code Execution Fixtures

executeCode (Test-scoped)

Execute code in the console.

test('example', async function ({ executeCode }) {
	// Basic execution
	await executeCode('Python', 'print("hello")');
	await executeCode('R', 'print("world")');

	// With options
	await executeCode('Python', 'long_running()', {
		timeout: 60000,
		waitForReady: true,
		maximizeConsole: true
	});
});

runCommand (Test-scoped)

Run a VS Code command via quick access.

test('example', async function ({ runCommand }) {
	// Run command
	await runCommand('workbench.action.files.save');

	// With options
	await runCommand('some.command', { keepOpen: true });
});

Settings Fixtures

settings (Worker-scoped)

Manage user settings.

test.beforeAll(async ({ settings }) => {
	// Set settings
	await settings.set({
		'editor.fontSize': 14,
		'files.autoSave': 'off',
		'positron.notebook.enabled': true
	});

	// With options
	await settings.set({ 'key': 'value' }, {
		reload: true,       // Reload window after setting
		waitMs: 1000,       // Wait after setting
		waitForReady: true, // Wait for app ready
		keepOpen: false     // Close settings UI
	});

	// Clear all custom settings
	await settings.clear();

	// Remove specific settings
	await settings.remove(['editor.fontSize', 'files.autoSave']);
});

settingsFile (Worker-scoped)

Direct settings file access. Use for settings that need to be set before app starts.

test.beforeAll(async ({ settingsFile }) => {
	// Write settings directly to file
	await settingsFile.write({
		'positron.notebook.enabled': true
	});
});

vsCodeSettings (Worker-scoped)

Access VS Code's settings file (separate from user data dir settings).

test.beforeAll(async ({ vsCodeSettings }) => {
	await vsCodeSettings.write({ 'key': 'value' });
});

Utility Fixtures

hotKeys (Test-scoped)

Keyboard shortcuts and UI actions.

test('example', async function ({ hotKeys }) {
	// Editor actions
	await hotKeys.copy();
	await hotKeys.paste();
	await hotKeys.selectAll();
	await hotKeys.closeAllEditors();

	// Layout actions
	await hotKeys.stackedLayout();
	await hotKeys.notebookLayout();
	await hotKeys.fullSizeSecondarySidebar();

	// Sidebar actions
	await hotKeys.showSecondarySidebar();
	await hotKeys.closeSecondarySidebar();
	await hotKeys.showPrimarySidebar();
	await hotKeys.closePrimarySidebar();

	// Panel actions
	await hotKeys.toggleBottomPanel();
	await hotKeys.focusConsole();

	// Execution control
	await hotKeys.sendInterrupt();

	// Plots
	await hotKeys.clearPlots();
});

packages (Test-scoped)

Package management utilities.

test('example', async function ({ packages }) {
	// Install package
	await packages.manage('snowflake', 'install');

	// Uninstall package
	await packages.manage('renv', 'uninstall');
});

cleanup (Test-scoped)

Test cleanup utilities.

test.afterAll(async function ({ cleanup }) {
	// Remove files created during test
	await cleanup.removeTestFiles(['output.txt', 'generated.csv']);
});

devTools (Test-scoped)

Opens DevTools before test.

test('debug test', async function ({ devTools, app }) {
	// DevTools already open
	// Useful for debugging
});

restartApp (Test-scoped)

Restarts the app before test runs.

test('fresh app test', async function ({ restartApp: app }) {
	// App has been restarted
	// Fresh state
});

metric (Test-scoped)

Record performance metrics.

test('performance test', async function ({ metric, app }) {
	await metric.record('operation-name', async () => {
		// Operation to measure
		await app.workbench.console.executeCode('Python', code);
	});
});

logger (Worker-scoped)

Logging utilities.

test('example', async function ({ app, logger }) {
	logger.log('Starting test operation');
	await app.workbench.console.executeCode('Python', 'x = 1');
	logger.log('Operation completed');
});

Docker Fixtures (Workbench/Remote only)

runDockerCommand

Execute commands in Docker container. Only available in e2e-workbench and e2e-remote-ssh projects.

test('docker test', async function ({ runDockerCommand }) {
	const result = await runDockerCommand('ls -la', 'List files');
	// result.stdout, result.stderr, result.exitCode
});

Fixture Dependencies

Some fixtures depend on others:

app
 ├── page (derived from app.code.driver.page)
 ├── sessions (derived from app.workbench.sessions)
 ├── hotKeys (derived from app.workbench.hotKeys)
 ├── executeCode (uses app.workbench.console)
 └── openFile/openDataFile/openFolder (use app)

python, r → sessions → app
settings → app
settingsFile → userDataDir → options

Custom Test Setup Files

Some test categories have their own _test.setup.ts that extends base fixtures:

Example: `test/e2e/tests/notebooks-positron/_test.setup.ts`

import { test as base, expect, tags } from '../_test.setup';

// Extend base test with notebook-specific settings
export const test = base.extend({
	beforeApp: async ({ settingsFile }, use) => {
		// Enable Positron notebooks before app starts
		await settingsFile.write({
			'positron.notebook.enabled': true,
			'workbench.editorAssociations': {
				'*.ipynb': 'workbench.editor.positronNotebook'
			}
		});
		await use();
	}
});

export { expect, tags };

Use the local _test.setup when testing specific features:

// For Positron notebook tests
import { test, expect, tags } from './_test.setup';

// For general tests
import { test, expect, tags } from '../_test.setup';

Best Practices

Use Appropriate Scope

// Worker-scoped for expensive operations (app startup, settings)
test.beforeAll(async ({ settings }) => {
	await settings.set({ 'key': 'value' });
});

// Test-scoped for per-test setup
test.beforeEach(async function ({ app }) => {
	await app.workbench.layouts.enterLayout('stacked');
});

Combine Related Fixtures

test('complete workflow', async function ({ app, python, hotKeys, executeCode }) {
	await executeCode('Python', 'df = pd.DataFrame(...)');
	await app.workbench.variables.doubleClickVariableRow('df');
	await hotKeys.closeSecondarySidebar();
	await app.workbench.dataExplorer.grid.verifyTableData([...]);
});

Use Interpreter Fixtures for Interpreter-Dependent Tests

// Prefer this - interpreter auto-started
test('Python test', async function ({ app, python }) {
	// Ready to execute code
});

// Over this - manual start
test('Python test', async function ({ app, sessions }) {
	await sessions.start('python');  // Extra step
});

Page Objects

Complete documentation of page objects available via app.workbench.*.

Page Object Architecture

Page objects encapsulate UI interactions. Access them through the app.workbench property:

test('example', async function ({ app }) {
	const { console, variables, dataExplorer, plots } = app.workbench;

	await console.executeCode('Python', 'x = 1');
	await variables.doubleClickVariableRow('x');
});

Console (app.workbench.console)

REPL/Console interactions.

Key Methods

// Execute code (opens quick input, selects language, runs code)
await console.executeCode('Python', 'print("hello")');
await console.executeCode('R', 'print("world")');
await console.executeCode('Python', code, {
	timeout: 60000,      // Wait timeout
	waitForReady: true,  // Wait for prompt after execution
	maximizeConsole: true // Maximize console panel
});

// Type directly to console
await console.typeToConsole('x = 1');
await console.typeToConsole('x = 1', true);  // Press enter after

// Paste code to console
await console.pasteCodeToConsole('multi\nline\ncode');
await console.pasteCodeToConsole('code', true);  // Press enter after

// Wait for ready state
await console.waitForReady('>>>');  // Python prompt
await console.waitForReady('>');    // R prompt
await console.waitForReady('>>>', 30000);  // With timeout

// Wait for content
await console.waitForConsoleContents('expected text');
await console.waitForConsoleContents(/regex pattern/);
await console.waitForConsoleContents('text', { timeout: 30000 });
await console.waitForConsoleContents('text', { expectedCount: 2 });
await console.waitForConsoleContents('should not appear', { expectedCount: 0 });

// Wait for execution states
await console.waitForExecutionStarted();
await console.waitForExecutionComplete();

// Actions
await console.sendEnterKey();
await console.clearInput();
await console.sendInterrupt();
await console.interruptExecution();
await console.focus();
await console.maximizeConsole();

// Session buttons
await console.restartButton.click();
await console.clearButton.click();
await console.trashButton.click();

Locators

console.activeConsole      // The active console instance
console.suggestionList     // Autocomplete suggestions
console.emptyConsole       // Empty console message

Variables (app.workbench.variables)

Variables pane interactions.

// Click/select variable
await variables.clickVariableRow('df');
await variables.doubleClickVariableRow('df');  // Opens in data explorer

// Expand/collapse
await variables.expandVariable('my_list');
await variables.collapseVariable('my_list');

// Get data
const value = await variables.getVariableValue('x');
await variables.waitForVariableValue('x', 'expected_value');

// Verify variable exists
await variables.waitForVariable('df');
await variables.verifyVariableExists('df', { timeout: 15000 });

Data Explorer (app.workbench.dataExplorer)

Data viewer interactions.

Grid Operations (dataExplorer.grid)

// Click cells
await dataExplorer.grid.clickCell(0, 0);  // Row 0, Column 0
await dataExplorer.grid.clickCell(0, 0, true);  // With Shift

// Get data
const cellValue = await dataExplorer.grid.getCellValue(0, 0);
const tableData = await dataExplorer.grid.getData();

// Verify data
await dataExplorer.grid.verifyTableData([
	{ 'Name': 'Alice', 'Age': '30' },
	{ 'Name': 'Bob', 'Age': '25' }
]);
await dataExplorer.grid.verifyTableData(expected, 60000);  // With timeout

// Column operations
await dataExplorer.grid.clickColumnHeader('Name');
await dataExplorer.grid.sortByColumn('Age', 'ascending');
await dataExplorer.grid.sortByColumn('Age', 'descending');

Filters (dataExplorer.filters)

// Add filter
await dataExplorer.filters.addTextFilter('Name', 'contains', 'Alice');
await dataExplorer.filters.addNumericFilter('Age', 'greater than', 25);

// Clear filters
await dataExplorer.filters.clearAll();
await dataExplorer.filters.removeFilter('Name');

Summary Panel (dataExplorer.summaryPanel)

await dataExplorer.summaryPanel.open();
await dataExplorer.summaryPanel.close();
await dataExplorer.summaryPanel.verifyColumnStats('Age', { mean: 27.5 });

Plots (app.workbench.plots)

Plots pane interactions.

// Wait for plot
await plots.waitForCurrentPlot();
await plots.waitForCurrentPlot({ timeout: 30000 });
await plots.waitForNoPlots();
await plots.waitForNoPlots({ timeout: 5000 });

// Plot count
await plots.waitForPlotCount(3);
const count = await plots.getPlotCount();

// Navigation
await plots.nextPlot();
await plots.previousPlot();
await plots.goToPlot(2);

// Actions
await plots.savePlotFromPlotsPane({ name: 'my-plot', format: 'PNG' });
await plots.savePlotFromPlotsPane({ name: 'plot', format: 'JPEG' });
await plots.copyCurrentPlotToClipboard();
await plots.openPlotIn('editor');
await plots.openPlotIn('newWindow');

// Editor operations
await plots.waitForPlotInEditor();
await plots.savePlotFromEditor({ name: 'editor-plot', format: 'PNG' });

Notebooks (app.workbench.notebooks)

Shared notebook operations (works with both VS Code and Positron notebooks).

// Open notebook
await notebooks.openNotebook(path);
await notebooks.openNotebook(join(app.workspacePathOrFolder, 'workspaces', 'notebook.ipynb'));

// Select interpreter
await notebooks.selectInterpreter('Python', process.env.POSITRON_PY_VER_SEL!);
await notebooks.selectInterpreter('R', process.env.POSITRON_R_VER_SEL!);

// Cell selection
await notebooks.selectCellAtIndex(0);
await notebooks.clickCell(0);

// Execution
await notebooks.executeActiveCell();
await notebooks.executeAllCells();
await notebooks.runAllCells();

// Cell content
await notebooks.addCell('code', 'print("hello")');
await notebooks.addCell('markdown', '# Header');
await notebooks.editCell(0, 'new code');

// Output
const output = await notebooks.getCellOutput(0);
await notebooks.waitForCellOutput(0, 'expected');

// Navigation
await notebooks.scrollToCell(5);

Sessions (app.workbench.sessions)

Session management.

// Start sessions
await sessions.start('python');
await sessions.start('r');
await sessions.start('python', { reuse: true });

// Wait for ready
await sessions.expectAllSessionsToBeReady();
await sessions.expectAllSessionsToBeReady({ timeout: 30000 });
await sessions.expectStatusToBe('session-id', 'idle');
await sessions.expectNoStartUpMessaging();

// Session info
const activeSession = await sessions.getActiveSession();
const allSessions = await sessions.getAllSessions();

Quick Access (app.workbench.quickaccess)

Command palette and quick access operations.

// Run commands
await quickaccess.runCommand('workbench.action.files.save');
await quickaccess.runCommand('command.id', { keepOpen: true });
await quickaccess.runCommand('command.id', { exactMatch: true });

// Quick open
await quickaccess.openFile('file.py');
await quickaccess.openFileQuickAccessAndWait('file.py');

Quick Input (app.workbench.quickInput)

Quick input dialog interactions.

// Wait for open/close
await quickInput.waitForQuickInputOpened();
await quickInput.waitForQuickInputClosed();

// Type and select
await quickInput.type('search text');
await quickInput.selectQuickInputElement(0);
await quickInput.waitForQuickInputElements(elements => elements.length > 0);

// Close
await quickInput.closeQuickInput();

Editors (app.workbench.editors)

Editor/tab management.

// Verify tabs
await editors.verifyTab('Data: df', { isVisible: true });
await editors.verifyTab('file.py', { isActive: true });
await editors.verifyTabCount(3);

// Tab actions
await editors.selectTab('file.py');
await editors.closeTab('file.py');
await editors.closeAllTabs();

Explorer (app.workbench.explorer)

File explorer interactions.

// Navigate
await explorer.openFile('src/main.py');
await explorer.expandFolder('src');
await explorer.collapseFolder('src');

// Verify files
await explorer.verifyExplorerFilesExist(['file1.py', 'file2.py']);
await explorer.waitForFile('output.txt');

Layouts (app.workbench.layouts)

Layout management.

// Predefined layouts
await layouts.enterLayout('stacked');
await layouts.enterLayout('fullSizedPanel');
await layouts.enterLayout('fullSizedAuxBar');
await layouts.enterLayout('notebook');

HotKeys (app.workbench.hotKeys)

Keyboard shortcuts. Also available as hotKeys fixture - see references/fixtures.md.

Standard: copy(), paste(), selectAll() Editor: closeAllEditors(), closeCurrentEditor() Layout: stackedLayout(), notebookLayout(), fullSizeSecondarySidebar() Sidebar: showSecondarySidebar(), closeSecondarySidebar(), showPrimarySidebar(), closePrimarySidebar() Panel: toggleBottomPanel(), focusConsole() Execution: sendInterrupt() Plots: clearPlots()

Modals (app.workbench.modals)

Modal dialog interactions.

await modals.waitForModalToOpen();
await modals.waitForModalToClose();
await modals.clickButton('OK');
await modals.clickButton('Cancel');

Toasts (app.workbench.toasts)

Toast notification interactions.

await toasts.waitForToast('Success message');
await toasts.dismissToast();
await toasts.verifyNoToasts();

Context Menu (app.workbench.contextMenu)

Context menu interactions.

await contextMenu.triggerAndClick({
	menuTrigger: someLocator,
	menuItemLabel: 'Menu Item'
});

await contextMenu.triggerAndVerifyMenuItems({
	menuTrigger: someLocator,
	menuItemStates: [
		{ label: 'Item 1', enabled: true },
		{ label: 'Item 2', enabled: false }
	]
});

Other Page Objects

Page Object Pattern

All page objects follow this pattern:

export class MyPageObject {
	// Locators as class properties
	someButton: Locator;
	someList: Locator;

	constructor(private code: Code, ...) {
		// Initialize locators in constructor
		this.someButton = this.code.driver.page.getByTestId('some-button');
		this.someList = this.code.driver.page.locator('.some-list');
	}

	// Actions wrapped in test.step
	async doSomething(): Promise<void> {
		return test.step('Do something', async () => {
			await this.someButton.click();
		});
	}

	// Verifications with expect
	async verifySomething(expected: string): Promise<void> {
		await test.step(`Verify something is ${expected}`, async () => {
			await expect(this.someList).toContainText(expected);
		});
	}
}

Finding Available Methods

Check source files in test/e2e/pages/ or use IDE autocomplete on app.workbench.<pageObject>.

Test File Structure

Complete guide to structuring Playwright e2e test files in Positron.

Test File Location

Tests are organized by feature in test/e2e/tests/:

test/e2e/tests/
  ├── _test.setup.ts       # Core setup - ALWAYS import from here
  ├── _global.setup.ts     # Global setup (runs once)
  ├── console/             # Console/REPL tests
  ├── data-explorer/       # Data Explorer tests
  ├── notebook/            # Notebook tests
  ├── plots/               # Plots tests
  ├── variables/           # Variables pane tests
  └── ...

Complete Test File Template

/*---------------------------------------------------------------------------------------------
 *  Copyright (C) 2024-2025 Posit Software, PBC. All rights reserved.
 *  Licensed under the Elastic License 2.0. See LICENSE.txt for license information.
 *--------------------------------------------------------------------------------------------*/

import { join } from 'path';
import { test, expect, tags } from '../_test.setup';

// REQUIRED: Unique suite ID for app isolation
test.use({
	suiteId: __filename
});

test.describe('Feature Name - Subsection', {
	tag: [tags.WEB, tags.WIN, tags.CRITICAL, tags.FEATURE_TAG]
}, () => {

	// Worker-scoped setup (runs once before all tests in file)
	test.beforeAll(async ({ settings }) => {
		await settings.set({
			'some.setting': true
		});
	});

	// Test-scoped setup (runs before each test)
	test.beforeEach(async function ({ app }) {
		await app.workbench.layouts.enterLayout('fullSizedPanel');
	});

	// Test-scoped cleanup (runs after each test)
	test.afterEach(async function ({ app, hotKeys }) {
		await app.workbench.dataExplorer.filters.clearAll();
		await hotKeys.closeAllEditors();
	});

	// Worker-scoped cleanup (runs after all tests in file)
	test.afterAll(async function ({ cleanup }) {
		await cleanup.removeTestFiles(['generated-file.txt']);
	});

	// Test with auto-started interpreter
	test('Test with Python', async function ({ app, python }) {
		// Python interpreter automatically started before this runs
		await app.workbench.console.executeCode('Python', 'print("hello")');
		await app.workbench.console.waitForConsoleContents('hello');
	});

	// Test with manual session management
	test('Test with manual session', async function ({ app, sessions }) {
		await sessions.start('python');
		// ... test logic
	});

	// Test with per-test tags
	test('Specific platform test', {
		tag: [tags.WIN],  // Only on Windows
		annotation: [{ type: 'issue', description: 'https://github.com/posit-dev/positron/issues/1234' }]
	}, async function ({ app, r }) {
		// R-specific test
	});
});

Import Rules

Always Import from _test.setup

// CORRECT
import { test, expect, tags } from '../_test.setup';

// WRONG - Do not import from @playwright/test
import { test, expect } from '@playwright/test';

The _test.setup provides:

• Custom test object with all Positron fixtures
• Re-exported expect from Playwright
• tags enum for test filtering

Other Common Imports

import { join } from 'path';  // For file paths

suiteId Requirement

MANDATORY: Every test file MUST set suiteId:

test.use({
	suiteId: __filename
});

This ensures:

• Each test file gets a fresh app instance
• Logs are organized by test file
• beforeAll/afterAll hooks run correctly per file

Test Organization

Describe Blocks

Use test.describe to group related tests:

test.describe('Console Input', {
	tag: [tags.WEB, tags.WIN, tags.CONSOLE]
}, () => {
	// Tests for console input functionality
});

// Nested describes for sub-features
test.describe('Console History', () => {
	test.describe('Navigation', () => {
		// History navigation tests
	});

	test.describe('Search', () => {
		// History search tests
	});
});

Test Naming

Use descriptive names that indicate: 1. The interpreter/language (if applicable) 2. What is being tested 3. Expected outcome

// Good names
test('Python - Can execute multi-line code in console');
test('R - Verify plot renders with correct dimensions');
test('Verify data explorer filters work with numeric columns');

// Bad names
test('test1');
test('console works');
test('execute code');

Hook Scopes

Worker-Scoped (beforeAll/afterAll)

Run once per test file. Use for:

• Setting up user settings
• Creating shared resources
• Final cleanup

test.beforeAll(async ({ settings }) => {
	// Runs once before all tests in this file
	await settings.set({ 'editor.fontSize': 14 });
});

test.afterAll(async ({ cleanup }) => {
	// Runs once after all tests in this file
	await cleanup.removeTestFiles(['output.txt']);
});

Test-Scoped (beforeEach/afterEach)

Run before/after each test. Use for:

• UI state reset
• Closing editors
• Clearing state

test.beforeEach(async function ({ app }) {
	await app.workbench.layouts.enterLayout('fullSizedPanel');
});

test.afterEach(async function ({ hotKeys }) {
	await hotKeys.closeAllEditors();
});

Function Syntax Requirement

IMPORTANT: Use function syntax (not arrow functions) for tests and hooks:

// CORRECT - function syntax
test('my test', async function ({ app, python }) {
	// ...
});

test.beforeEach(async function ({ app }) {
	// ...
});

// INCORRECT - arrow function
test('my test', async ({ app, python }) => {
	// ...
});

While arrow functions often work, function syntax is the established pattern in the codebase and ensures proper fixture access.

Test Tags

Feature Tags

Indicate what feature the test covers:

tags.CONSOLE          // Console/REPL functionality
tags.DATA_EXPLORER    // Data Explorer
tags.NOTEBOOKS        // Jupyter notebooks
tags.PLOTS            // Plotting
tags.VARIABLES        // Variables pane
tags.CONNECTIONS      // Database connections
tags.HELP             // Help system
tags.INTERPRETER      // Interpreter management
tags.CRITICAL         // Critical path tests (high priority)

Platform Tags

Control which platforms/projects run the test:

tags.WEB      // Enable web browser testing
tags.WIN      // Enable Windows testing
tags.WORKBENCH // Enable Posit Workbench testing

Default behavior: Tests without platform tags only run on Linux/Electron.

Applying Tags

// Describe-level tags (apply to all tests in block)
test.describe('Console', {
	tag: [tags.WEB, tags.WIN, tags.CRITICAL, tags.CONSOLE]
}, () => { ... });

// Per-test tags (override or add to describe tags)
test('Special test', {
	tag: [tags.WIN]  // Only Windows
}, async function ({ app }) { ... });

Test Annotations

Add metadata to tests for tracking:

test('Flaky test', {
	annotation: [
		{ type: 'issue', description: 'https://github.com/posit-dev/positron/issues/1234' },
		{ type: 'fixme', description: 'Flaky on CI - timing issue' }
	]
}, async function ({ app }) { ... });

Using test.step

Wrap logical groups of actions in test.step for better reporting:

test('Complete workflow', async function ({ app, python }) {
	await test.step('Create dataframe', async () => {
		await app.workbench.console.executeCode('Python', 'df = pd.DataFrame(...)');
	});

	await test.step('Open in data explorer', async () => {
		await app.workbench.variables.doubleClickVariableRow('df');
		await app.workbench.editors.verifyTab('Data: df', { isVisible: true });
	});

	await test.step('Verify data', async () => {
		await app.workbench.dataExplorer.grid.verifyTableData([...]);
	});
});

Benefits:

• Test report shows each step
• Easier to identify where failures occur
• Self-documenting test structure

Parallel Test Considerations

Tests in the same file share an app instance. Ensure:

• Tests don't depend on order
• Cleanup properly in afterEach
• Don't leave state that affects other tests

test.afterEach(async function ({ hotKeys, app }) {
	// Reset UI state
	await hotKeys.closeAllEditors();
	await app.workbench.layouts.enterLayout('stacked');
});