Testing

LobeHub Testing Guide

Quick Reference

Commands:

# Run specific test file
bunx vitest run --silent='passed-only' '[file-path]'

# Database package (client-db, PGlite — default, skips BM25/pg_search)
cd packages/database && bunx vitest run --silent='passed-only' '[file]'

# Database package (server-db, Postgres — BM25/pgvector parity, what CI measures coverage in)
cd packages/database && TEST_SERVER_DB=1 bunx vitest run --silent='passed-only' '[file]'

Never run bun run test - it runs all 3000+ tests (\~10 minutes).

Database models/repositories: every new file under packages/database/src/models/**

or src/repositories/** ships with a sibling __tests__/<name>.test.ts in the same PR.

Use the real DB via getTestDB() (integration style), guard BM25/full-text-search blocks

with describe.skipIf(!isServerDB), and always test user-isolation. See

references/db-model-test.md for setup, schema gotchas, and the client-vs-server-db split.

Test Categories

Core Principles

1. Prefer `vi.spyOn` over `vi.mock` - More targeted, easier to maintain 2. Tests must pass type check - Run bun run type-check after writing tests 3. After 1-2 failed fix attempts, stop and ask for help 4. Test behavior, not implementation details 5. Regression tests for bug fixes - After fixing a bug, add a regression test that fails before the fix and passes after, to prevent recurrence 6. No new component tests - Only update existing React component tests. Complex logic should be extracted into hooks and tested there instead 7. All source changes before any test changes - Complete all source file edits first, then update tests in a separate pass. Interleaving disrupts reasoning about the source changes, especially across many files

Basic Test Structure

import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';

beforeEach(() => {
  vi.clearAllMocks();
});

afterEach(() => {
  vi.restoreAllMocks();
});

describe('ModuleName', () => {
  describe('functionName', () => {
    it('should handle normal case', () => {
      // Arrange → Act → Assert
    });
  });
});

Mock Patterns

// ✅ Spy on direct dependencies
vi.spyOn(messageService, 'createMessage').mockResolvedValue('id');

// ✅ Use vi.stubGlobal for browser APIs
vi.stubGlobal('Image', mockImage);
vi.spyOn(URL, 'createObjectURL').mockReturnValue('blob:mock');

// ❌ Avoid mocking entire modules globally
vi.mock('@/services/chat'); // Too broad

Detailed Guides

See references/ for specific testing scenarios:

• Database Model testing: references/db-model-test.md
• Electron IPC testing: references/electron-ipc-test.md
• Zustand Store Action testing: references/zustand-store-action-test.md
• Agent Runtime E2E testing: references/agent-runtime-e2e.md
• Desktop Controller testing: references/desktop-controller-test.md

Fixing Failing Tests — Optimize or Delete?

When tests fail due to implementation changes (not bugs), evaluate before blindly fixing:

Keep & Fix (update test data/assertions)

• Behavior tests: Tests that verify _what_ the code does (output, side effects, user-visible behavior). Just update mock data formats or expected values.
• Example: Tool data structure changed from { name } to { function: { name } } → update mock data
• Example: Output format changed from Current date: YYYY-MM-DD to Current date: YYYY-MM-DD (TZ) → update expected string

Delete (over-specified, low value)

• Param-forwarding tests: Tests that assert exact internal function call arguments (e.g., expect(internalFn).toHaveBeenCalledWith(expect.objectContaining({ exact params }))) — these break on every refactor and duplicate what behavior tests already cover.
• Implementation-coupled tests: Tests that verify _how_ the code works internally rather than _what_ it produces. If a higher-level test already covers the same behavior, the low-level test adds maintenance cost without coverage gain.

Decision Checklist

1. Does the test verify externally observable behavior (API response, DB write, rendered output)? → Keep 2. Does the test only verify internal wiring (which function receives which params)? → Check if a behavior test already covers it. If yes → Delete 3. Is the same behavior already tested at a higher integration level? → Delete the lower-level duplicate 4. Would the test break again on the next routine refactor? → Consider raising to integration level or deleting

When Writing New Tests

• Prefer integration-level assertions (verify final output) over white-box assertions (verify internal calls)
• Use expect.objectContaining only for stable, public-facing contracts — not for internal param shapes that change with refactors
• Mock at boundaries (DB, network, external services), not between internal modules

Common Issues

1. Module pollution: Use vi.resetModules() when tests fail mysteriously 2. Mock not working: Check setup position and use vi.clearAllMocks() in beforeEach 3. Test data pollution: Clean database state in beforeEach/afterEach 4. Async issues: Wrap state changes in act() for React hooks

Agent Runtime E2E Testing Guide

Core Principles

Minimal Mock Principle

Only mock three external dependencies:

NOT mocked:

• model-bank - Uses real model config
• Mecha (AgentToolsEngine, ContextEngineering)
• AgentRuntimeService
• AgentRuntimeCoordinator

Use vi.spyOn, not vi.mock

Different tests need different LLM responses. vi.spyOn provides:

• Flexible return values per test
• Easy testing of different scenarios
• Better test isolation

Default Model: gpt-5

• Always available in model-bank
• Stable across model updates

Technical Implementation

Database Setup

import { LobeChatDatabase } from '@lobechat/database';
import { getTestDB } from '@lobechat/database/test-utils';

let testDB: LobeChatDatabase;

beforeEach(async () => {
  testDB = await getTestDB();
});

OpenAI Stream Response Helper

export const createOpenAIStreamResponse = (options: {
  content?: string;
  toolCalls?: Array<{ id: string; name: string; arguments: string }>;
  finishReason?: 'stop' | 'tool_calls';
}) => {
  const { content, toolCalls, finishReason = 'stop' } = options;

  return new Response(
    new ReadableStream({
      start(controller) {
        const encoder = new TextEncoder();

        if (content) {
          const chunk = {
            id: 'chatcmpl-mock',
            object: 'chat.completion.chunk',
            model: 'gpt-5',
            choices: [{ index: 0, delta: { content }, finish_reason: null }],
          };
          controller.enqueue(encoder.encode(`data: ${JSON.stringify(chunk)}\n\n`));
        }

        // ... tool_calls handling
        // ... finish chunk
        controller.enqueue(encoder.encode('data: [DONE]\n\n'));
        controller.close();
      },
    }),
    { headers: { 'content-type': 'text/event-stream' } },
  );
};

State Management

import {
  InMemoryAgentStateManager,
  InMemoryStreamEventManager,
} from '@/server/modules/AgentRuntime';

const stateManager = new InMemoryAgentStateManager();
const streamEventManager = new InMemoryStreamEventManager();

const service = new AgentRuntimeService(serverDB, userId, {
  coordinatorOptions: { stateManager, streamEventManager },
  queueService: null,
  streamEventManager,
});

Mock OpenAI API

const fetchSpy = vi.spyOn(globalThis, 'fetch');

it('should handle text response', async () => {
  fetchSpy.mockResolvedValueOnce(createOpenAIStreamResponse({ content: 'Response text' }));
  // ... execute test
});

it('should handle tool calls', async () => {
  fetchSpy.mockResolvedValueOnce(
    createOpenAIStreamResponse({
      toolCalls: [
        {
          id: 'call_123',
          name: 'lobe-web-browsing____search',
          arguments: JSON.stringify({ query: 'weather' }),
        },
      ],
      finishReason: 'tool_calls',
    }),
  );
  // ... execute test
});

Notes

1. Test isolation: Clean InMemoryAgentStateManager and InMemoryStreamEventManager after each test 2. Timeout: E2E tests may need longer timeouts 3. Debug: Use DEBUG=lobe-server:* for detailed logs

Database Model Testing Guide

Test the packages/database Model and Repository layers.

Rule: every new Model or Repository ships with a sibling test in the same PR.

A new file under src/models/** or src/repositories/** must have a matching

__tests__/<name>.test.ts. Coverage runs in server-db mode in CI and the patch

gate will not always catch a brand-new untested file (a small new file barely

moves the project total) — so this is a convention, not something CI guarantees.

Start from the template: packages/database/src/models/__tests__/_test_template.ts.

Two test environments: client-db vs server-db

getTestDB() (src/core/getTestDB.ts) returns different engines based on the TEST_SERVER_DB env var:

# 1. Client environment (fast, default — what most local runs use)
cd packages/database && bunx vitest run --silent='passed-only' '[file]'

# 2. Server environment (BM25 / pg_search / pgvector parity, needs DATABASE_TEST_URL)
cd packages/database && TEST_SERVER_DB=1 bunx vitest run --silent='passed-only' '[file]'

Implication: client-db coverage under-counts any code that needs BM25 (e.g. repositories/search/index.ts reads near-0% locally but is fully covered in CI). Don't chase those lines locally — confirm via CI/Codecov.

BM25 / full-text search → describe.skipIf(!isServerDB)

Any method using the BM25 @@@ operator or sanitizeBm25 (keyword search: queryByKeyword, searchAgents, userMemory lexical search, …) throws under PGlite (often swallowed by a catch that returns [], so the test silently fails with empty results). Guard those blocks so they only run in server-db:

// BM25 search requires the pg_search extension (ParadeDB), not available in PGlite
const isServerDB = process.env.TEST_SERVER_DB === '1';
describe.skipIf(!isServerDB)('queryByKeyword', () => {
  /* ... */
});

Convention already used in session.test.ts, topic.query.test.ts, message.query.test.ts, home/index.test.ts, repositories/search/index.test.ts.

Setup boilerplate

Top-of-file pattern (see _test_template.ts for the full version). Use real DB integration via getTestDB() — not a mocked `vi.fn()` db; the integration style exercises real SQL and gives far deeper coverage.

import { eq } from 'drizzle-orm';
import { afterEach, beforeEach, describe, expect, it } from 'vitest';

import { getTestDB } from '../../core/getTestDB';
import { users } from '../../schemas';
import type { LobeChatDatabase } from '../../type';
import { MyModel } from '../myModel';

const serverDB: LobeChatDatabase = await getTestDB(); // top-level await is fine

const userId = 'my-model-test-user';
const otherUserId = 'other-user';
const myModel = new MyModel(serverDB, userId);

beforeEach(async () => {
  await serverDB.delete(users);
  await serverDB.insert(users).values([{ id: userId }, { id: otherUserId }]);
});

afterEach(async () => {
  await serverDB.delete(users); // cascades to user-scoped rows
});

Some tests need the Node environment (pgvector, server-only deps) — add // @vitest-environment node as the first line when required.

User permission check — security first 🔒

Every user-data operation must be ownership-scoped. Always add a test proving another user cannot read/update/delete the row.

// ✅ SECURE: ownership in the WHERE clause
update = async (id: string, data: Partial<MyModel>) =>
  this.db
    .update(myTable)
    .set(data)
    .where(and(eq(myTable.id, id), eq(myTable.userId, this.userId)))
    .returning();

it('should NOT update another user's record', async () => {
  const otherModel = new MyModel(serverDB, otherUserId);
  const [row] = await otherModel.create({ data: 'original' });

  await myModel.update(row.id, { data: 'hacked' });

  const unchanged = await serverDB.query.myTable.findFirst({
    where: eq(myTable.id, row.id),
  });
  expect(unchanged?.data).toBe('original');
});

What to cover

Aim each model/repository as close to 100% as practical (excluding BM25):

• Every public method
• Both branches of conditionals; empty-list / if (!x) return [] early returns
• Error fallbacks (e.g. decrypt/JSON-parse failure → null)
• Filters, pagination, ordering branches
• Ownership / user isolation, and workspace scoping if the model takes a workspaceId

Schema gotchas (real traps that fail inserts or types)

• `workspaces` requires { id, name, slug, primaryOwnerId } and has **no

userId column** — insert(workspaces).values({ id, name, slug, primaryOwnerId }).

• uuid columns: a "not found" test must pass a _valid_ UUID

('00000000-0000-0000-0000-000000000000'); a random string raises a 22P02 DB error instead of returning undefined/null.

• Enum / `$type` columns are type-checked: e.g. files.source is a

FileSource enum (image_generation | page-editor | video_generation), not free text — passing 'upload' is a type error.

• Read the table's schema in src/schemas/ for notNull columns **without

defaults**; you must supply those on insert.

Foreign key handling

// ❌ Wrong: invalid foreign key
const testData = { asyncTaskId: 'invalid-uuid', fileId: 'non-existent' };

// ✅ Use null …
const testData = { asyncTaskId: null, fileId: null };

// ✅ … or create the referenced row first
const [asyncTask] = await serverDB.insert(asyncTasks).values({ status: 'pending' }).returning();
testData.asyncTaskId = asyncTask.id;

Predictable sorting

// ✅ Use explicit timestamps — never rely on insert order
await serverDB.insert(table).values([
  { ...data1, createdAt: new Date('2024-01-01T10:00:00Z') },
  { ...data2, createdAt: new Date('2024-01-02T10:00:00Z') },
]);

Checking coverage of one file

# Per-file coverage; read the "Uncovered Line #s" column to find gaps
cd packages/database
bunx vitest run --coverage --silent='passed-only' '[test-file]' 2>&1 | grep '[sourceFile].ts'

Before finishing

1. Tests pass: bunx vitest run --silent='passed-only' '[file]' 2. Types pass: bun run type-check (vitest uses esbuild and does not type-check — a green test run can still have type errors).

Desktop Controller Unit Testing Guide

Testing Framework & Directory Structure

LobeHub Desktop uses Vitest as the test framework. Controller unit tests should be placed in the __tests__ directory adjacent to the controller file, named with the original controller filename plus .test.ts.

apps/desktop/src/main/controllers/
├── __tests__/
│   ├── index.test.ts
│   ├── MenuCtr.test.ts
│   └── ...
├── McpCtr.ts
├── MenuCtr.ts
└── ...

Basic Test File Structure

import { beforeEach, describe, expect, it, vi } from 'vitest';

import type { App } from '@/core/App';

import YourController from '../YourControllerName';

// Mock dependencies
vi.mock('dependency-module', () => ({
  dependencyFunction: vi.fn(),
}));

// Mock App instance
const mockApp = {
  // Mock necessary App properties and methods as needed
} as unknown as App;

describe('YourController', () => {
  let controller: YourController;

  beforeEach(() => {
    vi.clearAllMocks();
    controller = new YourController(mockApp);
  });

  describe('methodName', () => {
    it('test scenario description', async () => {
      // Prepare test data

      // Execute method under test
      const result = await controller.methodName(params);

      // Verify results
      expect(result).toMatchObject(expectedResult);
    });
  });
});

Mocking External Dependencies

Module Functions

const mockFunction = vi.fn();

vi.mock('module-name', () => ({
  functionName: mockFunction,
}));

Node.js Core Modules

Example: mocking child_process.exec and util.promisify:

const mockExecImpl = vi.fn();

vi.mock('child_process', () => ({
  exec: vi.fn((cmd, callback) => {
    return mockExecImpl(cmd, callback);
  }),
}));

vi.mock('util', () => ({
  promisify: vi.fn((fn) => {
    return async (cmd: string) => {
      return new Promise((resolve, reject) => {
        mockExecImpl(cmd, (error: Error | null, result: any) => {
          if (error) reject(error);
          else resolve(result);
        });
      });
    };
  }),
}));

Best Practices

1. Isolate tests: Use beforeEach to reset mocks and state 2. Comprehensive coverage: Test normal flows, edge cases, and error handling 3. Clear naming: Test names should describe content and expected results 4. Avoid implementation details: Test behavior, not implementation 5. Mock external dependencies: Use vi.mock() for all external dependencies

Example: Testing IPC Event Handler

it('should handle IPC event correctly', async () => {
  mockSomething.mockReturnValue({ result: 'success' });

  const result = await controller.ipcMethodName({
    param1: 'value1',
    param2: 'value2',
  });

  expect(result).toEqual({
    success: true,
    data: { result: 'success' },
  });

  expect(mockSomething).toHaveBeenCalledWith('value1', 'value2');
});

Electron IPC Testing Strategy

For Electron IPC tests, use Mock return values instead of real Electron environment.

Basic Mock Setup

import { vi } from 'vitest';
import { electronIpcClient } from '@/server/modules/ElectronIPCClient';

vi.mock('@/server/modules/ElectronIPCClient', () => ({
  electronIpcClient: {
    getFilePathById: vi.fn(),
    deleteFiles: vi.fn(),
  },
}));

Setting Mock Behavior

beforeEach(() => {
  vi.resetAllMocks();
  vi.mocked(electronIpcClient.getFilePathById).mockResolvedValue('/path/to/file.txt');
  vi.mocked(electronIpcClient.deleteFiles).mockResolvedValue({ success: true });
});

Testing Different Scenarios

it('should handle successful file deletion', async () => {
  vi.mocked(electronIpcClient.deleteFiles).mockResolvedValue({ success: true });

  const result = await service.deleteFiles(['desktop://file1.txt']);

  expect(electronIpcClient.deleteFiles).toHaveBeenCalledWith(['desktop://file1.txt']);
  expect(result.success).toBe(true);
});

it('should handle file deletion failure', async () => {
  vi.mocked(electronIpcClient.deleteFiles).mockRejectedValue(new Error('Delete failed'));

  const result = await service.deleteFiles(['desktop://file1.txt']);

  expect(result.success).toBe(false);
  expect(result.errors).toBeDefined();
});

Advantages

1. Environment simplification: No complex Electron setup 2. Controlled testing: Precise control over IPC return values 3. Scenario coverage: Easy to test success/failure cases 4. Speed: Mock calls are faster than real IPC

Notes

• Ensure mock behavior matches real IPC interface
• Use vi.mocked() for type safety
• Reset mocks in beforeEach to avoid test interference
• Verify both return values and that IPC methods were called correctly

Zustand Store Action Testing Guide

Basic Structure

import { act, renderHook } from '@testing-library/react';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { useChatStore } from '../../store';

vi.mock('zustand/traditional');

beforeEach(() => {
  vi.clearAllMocks();
  useChatStore.setState(
    {
      activeId: 'test-session-id',
      messagesMap: {},
      loadingIds: [],
    },
    false,
  );

  vi.spyOn(messageService, 'createMessage').mockResolvedValue('new-message-id');

  act(() => {
    useChatStore.setState({
      refreshMessages: vi.fn(),
      internal_coreProcessMessage: vi.fn(),
    });
  });
});

afterEach(() => {
  vi.restoreAllMocks();
});

Key Principles

1. Spy Direct Dependencies Only

// ✅ Good: Spy on direct dependency
const fetchAIChatSpy = vi.spyOn(result.current, 'internal_fetchAIChatMessage')
  .mockResolvedValue({ isFunctionCall: false, content: 'AI response' });

// ❌ Bad: Spy on lower-level implementation
const streamSpy = vi.spyOn(chatService, 'createAssistantMessageStream')
  .mockImplementation(...);

2. Minimize Global Spies

// ✅ Spy only when needed
it('should process message', async () => {
  const streamSpy = vi.spyOn(chatService, 'createAssistantMessageStream')
    .mockImplementation(...);
  // test logic
  streamSpy.mockRestore();
});

// ❌ Don't setup all spies globally
beforeEach(() => {
  vi.spyOn(chatService, 'createAssistantMessageStream').mockResolvedValue({});
  vi.spyOn(fileService, 'uploadFile').mockResolvedValue({});
});

3. Use act() for Async Operations

it('should send message', async () => {
  const { result } = renderHook(() => useChatStore());

  await act(async () => {
    await result.current.sendMessage({ message: 'Hello' });
  });

  expect(messageService.createMessage).toHaveBeenCalled();
});

4. Test Organization

describe('sendMessage', () => {
  describe('validation', () => {
    it('should not send when session is inactive');
    it('should not send when message is empty');
  });
  describe('message creation', () => {
    it('should create user message and trigger AI processing');
  });
  describe('error handling', () => {
    it('should handle message creation errors gracefully');
  });
});

Streaming Response Mock

it('should handle streaming chunks', async () => {
  const { result } = renderHook(() => useChatStore());

  const streamSpy = vi.spyOn(chatService, 'createAssistantMessageStream')
    .mockImplementation(async ({ onMessageHandle, onFinish }) => {
      await onMessageHandle?.({ type: 'text', text: 'Hello' } as any);
      await onMessageHandle?.({ type: 'text', text: ' World' } as any);
      await onFinish?.('Hello World', {});
    });

  await act(async () => {
    await result.current.internal_fetchAIChatMessage({...});
  });

  streamSpy.mockRestore();
});

SWR Hook Testing

it('should fetch data', async () => {
  const mockData = [{ id: '1', name: 'Item 1' }];
  vi.spyOn(discoverService, 'getPluginCategories').mockResolvedValue(mockData);

  const { result } = renderHook(() => useStore.getState().usePluginCategories(params));

  await waitFor(() => {
    expect(result.current.data).toEqual(mockData);
  });
});

Key points for SWR:

• DO NOT mock useSWR - let it use real implementation
• Only mock service methods (fetchers)
• Use waitFor for async operations

Anti-Patterns

// ❌ Don't mock entire store
vi.mock('../../store', () => ({ useChatStore: vi.fn(() => ({...})) }));

// ❌ Don't test internal state structure
expect(result.current.messagesMap).toHaveProperty('test-session');

// ✅ Test behavior instead
expect(result.current.refreshMessages).toHaveBeenCalled();