Mcp

MCP: Build & Manage Protocol Servers

Build MCP servers that integrate APIs, and execute tools from configured servers.

When to Use

Building: Create MCP servers (Python/TypeScript), integrate APIs, design agent-centric tools, implement validation/error handling, create evaluations

Managing: Discover/execute tools via Gemini CLI, filter tools for tasks, manage multi-server configs

Core Concepts

MCP = standardized protocol for AI agents to access external tools/data.

Components: Tools (executable functions), Resources (read-only data), Prompts (templates) Transports: Stdio (local), HTTP (remote), SSE (real-time)

Load: references/protocol-basics.md for full protocol details

---

Part 1: Building MCP Servers

Build high-quality MCP servers that enable LLMs to accomplish real-world tasks.

Development Workflow

Phase 1: Research & Planning 1. Study agent-centric design principles (workflows over endpoints) 2. Research target API documentation exhaustively 3. Load framework documentation (Python SDK or TypeScript SDK) 4. Plan tool selection, shared utilities, input/output design, error handling

Phase 2: Implementation 1. Set up project structure (single file for Python, full structure for TypeScript) 2. Implement core infrastructure (API clients, error handlers, formatters) 3. Register tools with proper schemas and annotations 4. Follow language-specific best practices

Phase 3: Testing & Quality 1. Code quality review (DRY, composability, consistency) 2. Run builds and syntax checks 3. Use quality checklists

Phase 4: Evaluation 1. Create 10 complex, realistic evaluation questions 2. Questions must be read-only, independent, and verifiable 3. Test LLM's ability to use your server effectively

Reference: references/building-servers.md - Load for complete development guide with:

• Agent-centric design principles
• Python (FastMCP) implementation guide with Pydantic models
• TypeScript (MCP SDK) implementation guide with Zod schemas
• Tool naming conventions, response formats, pagination patterns
• Character limits, error handling, security best practices
• Complete working examples and quality checklists
• Evaluation creation and testing methodology

Key Best Practices

Tool Design:

• Use service-prefixed names (slack_send_message, not send_message)
• Support both JSON and Markdown response formats
• Implement pagination with limit, offset, has_more
• Set CHARACTER_LIMIT constant (typically 25,000)
• Provide actionable error messages that guide agents

Code Quality:

• Extract common functionality into reusable functions
• Use async/await for all I/O operations
• Type hints (Python) or strict TypeScript throughout
• Comprehensive docstrings with explicit schemas

Reference: references/best-practices.md - Load for comprehensive guidelines

---

Part 2: Using MCP Tools

Execute and manage tools from configured MCP servers efficiently.

Configuration

MCP servers configured in .claude/.mcp.json:

{
  "mcpServers": {
    "server-name": {
      "command": "npx",
      "args": ["-y", "package-name"],
      "env": {"API_KEY": "${ENV_VAR}"}
    }
  }
}

Gemini CLI Integration: Create symlink for shared config:

mkdir -p .gemini && ln -sf .claude/.mcp.json .gemini/settings.json

Reference: references/using-tools.md - Load for complete configuration and usage guide

Execution Methods (Priority Order)

1. Gemini CLI (Primary)

Automatic tool discovery and execution via natural language.

# CRITICAL: Use stdin piping, NOT -p flag (deprecated, skips MCP init)
echo "Take a screenshot of https://example.com" | gemini -y -m gemini-2.5-flash

Benefits:

• Automatic tool discovery and selection
• Structured JSON responses (if GEMINI.md configured)
• Fastest execution
• No manual tool specification needed

GEMINI.md Response Format: Place in project root to enforce JSON-only responses:

# Gemini CLI Instructions
Always respond in this exact JSON format:
{"server":"name","tool":"name","success":true,"result":<data>,"error":null}
Maximum 500 characters. No markdown, no explanations.

2. Direct CLI Scripts (Secondary)

Manual tool specification when you know exact server/tool needed:

npx tsx scripts/cli.ts call-tool memory create_entities '{"entities":[...]}'

3. mcp-manager Subagent (Fallback)

Delegate to subagent when Gemini unavailable or for complex multi-tool workflows.

Reference: references/using-tools.md - Load for:

• Complete Gemini CLI guide with examples
• Direct script usage and options
• Subagent delegation patterns
• Tool discovery and filtering strategies
• Multi-server orchestration
• Troubleshooting and debugging

Tool Discovery

List available tools to understand capabilities:

# Saves to assets/tools.json for offline reference
npx tsx scripts/cli.ts list-tools

# List prompts and resources
npx tsx scripts/cli.ts list-prompts
npx tsx scripts/cli.ts list-resources

Intelligent Selection: LLM reads assets/tools.json directly for context-aware tool filtering (better than keyword matching).

---

Quick Start Examples

Building a Server

Python:

from mcp.server.fastmcp import FastMCP
from pydantic import BaseModel, Field

mcp = FastMCP("github_mcp")

class SearchInput(BaseModel):
    query: str = Field(..., min_length=2, max_length=200)
    limit: int = Field(default=20, ge=1, le=100)

@mcp.tool(name="github_search_repos", annotations={"readOnlyHint": True})
async def search_repos(params: SearchInput) -> str:
    # Implementation
    pass

if __name__ == "__main__":
    mcp.run()

TypeScript:

import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { z } from "zod";

const server = new McpServer({name: "github-mcp-server", version: "1.0.0"});

const SearchSchema = z.object({
  query: z.string().min(2).max(200),
  limit: z.number().int().min(1).max(100).default(20)
}).strict();

server.registerTool("github_search_repos", {
  description: "Search GitHub repositories",
  inputSchema: SearchSchema,
  annotations: {readOnlyHint: true}
}, async (params) => {
  // Implementation
});

Load references/building-servers.md for complete implementation guides.

Using Tools

Gemini CLI:

# IMPORTANT: Use stdin piping, NOT -p flag
echo "Search GitHub for MCP servers and summarize top 3" | gemini -y -m gemini-2.5-flash

Direct Script:

npx tsx scripts/cli.ts call-tool github search_repos '{"query":"mcp","limit":3}'

Load references/using-tools.md for complete usage patterns.

---

Reference Files

Load these as needed during your work:

Core References

• `references/building-servers.md` - Complete MCP server development guide
• Agent-centric design principles
• Python (FastMCP) and TypeScript (MCP SDK) implementation
• Tool patterns, response formats, pagination, error handling
• Complete examples and quality checklists
• Evaluation creation methodology

• `references/using-tools.md` - Complete MCP tool execution guide
• Gemini CLI integration and configuration
• Direct script execution patterns
• Subagent delegation strategies
• Tool discovery and filtering
• Multi-server orchestration

• `references/best-practices.md` - Universal MCP guidelines
• Server and tool naming conventions
• Response format standards (JSON vs Markdown)
• Pagination, character limits, truncation
• Security and privacy considerations
• Testing and compliance requirements

Supporting References

• `references/protocol-basics.md` - JSON-RPC protocol details
• `references/python-guide.md` - Python/FastMCP specifics (Pydantic models, async patterns)
• `references/typescript-guide.md` - TypeScript/Zod specifics (strict types, project structure)
• `references/evaluation-guide.md` - Creating effective MCP server evaluations

---

Progressive Disclosure

This SKILL.md provides high-level overview. Load reference files when:

Building Servers:

• Starting implementation → Load references/building-servers.md
• Need language-specific details → Load references/python-guide.md or references/typescript-guide.md
• Creating evaluations → Load references/evaluation-guide.md

Using Tools:

• Setting up Gemini CLI → Load references/using-tools.md
• Debugging tool execution → Load references/using-tools.md
• Multi-server configuration → Load references/using-tools.md

Best Practices:

• Reviewing standards → Load references/best-practices.md
• Security considerations → Load references/best-practices.md

---

Integration Patterns

Build + Use: Create MCP server, then test with Gemini CLI Multi-Server: Configure multiple servers, orchestrate via Gemini CLI Evaluation-Driven: Build server, create evaluations, iterate based on LLM feedback

---

Boundaries

Will:

• Guide MCP server development in Python or TypeScript
• Provide tool execution strategies via Gemini CLI or scripts
• Ensure best practices for agent-centric design
• Help create effective evaluations
• Configure multi-server setups

Will Not:

• Run long-running server processes in main thread (use tmux or evaluation harness)
• Skip input validation or error handling
• Create tools without comprehensive documentation
• Build servers without considering agent context limits

MCP Server Development Best Practices and Guidelines

Overview

This document compiles essential best practices and guidelines for building Model Context Protocol (MCP) servers. It covers naming conventions, tool design, response formats, pagination, error handling, security, and compliance requirements.

---

Quick Reference

Server Naming

• Python: {service}_mcp (e.g., slack_mcp)
• Node/TypeScript: {service}-mcp-server (e.g., slack-mcp-server)

Tool Naming

• Use snake_case with service prefix
• Format: {service}_{action}_{resource}
• Example: slack_send_message, github_create_issue

Response Formats

• Support both JSON and Markdown formats
• JSON for programmatic processing
• Markdown for human readability

Pagination

• Always respect limit parameter
• Return has_more, next_offset, total_count
• Default to 20-50 items

Character Limits

• Set CHARACTER_LIMIT constant (typically 25,000)
• Truncate gracefully with clear messages
• Provide guidance on filtering

---

Table of Contents

1. Server Naming Conventions 2. Tool Naming and Design 3. Response Format Guidelines 4. Pagination Best Practices 5. Character Limits and Truncation 6. Tool Development Best Practices 7. Transport Best Practices 8. Testing Requirements 9. OAuth and Security Best Practices 10. Resource Management Best Practices 11. Prompt Management Best Practices 12. Error Handling Standards 13. Documentation Requirements 14. Compliance and Monitoring

---

1. Server Naming Conventions

Follow these standardized naming patterns for MCP servers:

Python: Use format {service}_mcp (lowercase with underscores)

• Examples: slack_mcp, github_mcp, jira_mcp, stripe_mcp

Node/TypeScript: Use format {service}-mcp-server (lowercase with hyphens)

• Examples: slack-mcp-server, github-mcp-server, jira-mcp-server

The name should be:

• General (not tied to specific features)
• Descriptive of the service/API being integrated
• Easy to infer from the task description
• Without version numbers or dates

---

2. Tool Naming and Design

Tool Naming Best Practices

1. Use snake_case: search_users, create_project, get_channel_info 2. Include service prefix: Anticipate that your MCP server may be used alongside other MCP servers

• Use slack_send_message instead of just send_message
• Use github_create_issue instead of just create_issue
• Use asana_list_tasks instead of just list_tasks

3. Be action-oriented: Start with verbs (get, list, search, create, etc.) 4. Be specific: Avoid generic names that could conflict with other servers 5. Maintain consistency: Use consistent naming patterns within your server

Tool Design Guidelines

• Tool descriptions must narrowly and unambiguously describe functionality
• Descriptions must precisely match actual functionality
• Should not create confusion with other MCP servers
• Should provide tool annotations (readOnlyHint, destructiveHint, idempotentHint, openWorldHint)
• Keep tool operations focused and atomic

---

3. Response Format Guidelines

All tools that return data should support multiple formats for flexibility:

JSON Format (response_format="json")

• Machine-readable structured data
• Include all available fields and metadata
• Consistent field names and types
• Suitable for programmatic processing
• Use for when LLMs need to process data further

Markdown Format (response_format="markdown", typically default)

• Human-readable formatted text
• Use headers, lists, and formatting for clarity
• Convert timestamps to human-readable format (e.g., "2024-01-15 10:30:00 UTC" instead of epoch)
• Show display names with IDs in parentheses (e.g., "@john.doe (U123456)")
• Omit verbose metadata (e.g., show only one profile image URL, not all sizes)
• Group related information logically
• Use for when presenting information to users

---

4. Pagination Best Practices

For tools that list resources:

• Always respect the `limit` parameter: Never load all results when a limit is specified
• Implement pagination: Use offset or cursor-based pagination
• Return pagination metadata: Include has_more, next_offset/next_cursor, total_count
• Never load all results into memory: Especially important for large datasets
• Default to reasonable limits: 20-50 items is typical
• Include clear pagination info in responses: Make it easy for LLMs to request more data

Example pagination response structure:

{
  "total": 150,
  "count": 20,
  "offset": 0,
  "items": [...],
  "has_more": true,
  "next_offset": 20
}

---

5. Character Limits and Truncation

To prevent overwhelming responses with too much data:

• Define CHARACTER_LIMIT constant: Typically 25,000 characters at module level
• Check response size before returning: Measure the final response length
• Truncate gracefully with clear indicators: Let the LLM know data was truncated
• Provide guidance on filtering: Suggest how to use parameters to reduce results
• Include truncation metadata: Show what was truncated and how to get more

Example truncation handling:

CHARACTER_LIMIT = 25000

if len(result) > CHARACTER_LIMIT:
    truncated_data = data[:max(1, len(data) // 2)]
    response["truncated"] = True
    response["truncation_message"] = (
        f"Response truncated from {len(data)} to {len(truncated_data)} items. "
        f"Use 'offset' parameter or add filters to see more results."
    )

---

6. Transport Options

MCP servers support multiple transport mechanisms for different deployment scenarios:

Stdio Transport

Best for: Command-line tools, local integrations, subprocess execution

Characteristics:

• Standard input/output stream communication
• Simple setup, no network configuration needed
• Runs as a subprocess of the client
• Ideal for desktop applications and CLI tools

Use when:

• Building tools for local development environments
• Integrating with desktop applications (e.g., Claude Desktop)
• Creating command-line utilities
• Single-user, single-session scenarios

HTTP Transport

Best for: Web services, remote access, multi-client scenarios

Characteristics:

• Request-response pattern over HTTP
• Supports multiple simultaneous clients
• Can be deployed as a web service
• Requires network configuration and security considerations

Use when:

• Serving multiple clients simultaneously
• Deploying as a cloud service
• Integration with web applications
• Need for load balancing or scaling

Server-Sent Events (SSE) Transport

Best for: Real-time updates, push notifications, streaming data

Characteristics:

• One-way server-to-client streaming over HTTP
• Enables real-time updates without polling
• Long-lived connections for continuous data flow
• Built on standard HTTP infrastructure

Use when:

• Clients need real-time data updates
• Implementing push notifications
• Streaming logs or monitoring data
• Progressive result delivery for long operations

Transport Selection Criteria

---

7. Tool Development Best Practices

General Guidelines

1. Tool names should be descriptive and action-oriented 2. Use parameter validation with detailed JSON schemas 3. Include examples in tool descriptions 4. Implement proper error handling and validation 5. Use progress reporting for long operations 6. Keep tool operations focused and atomic 7. Document expected return value structures 8. Implement proper timeouts 9. Consider rate limiting for resource-intensive operations 10. Log tool usage for debugging and monitoring

Security Considerations for Tools

Input Validation

• Validate all parameters against schema
• Sanitize file paths and system commands
• Validate URLs and external identifiers
• Check parameter sizes and ranges
• Prevent command injection

Access Control

• Implement authentication where needed
• Use appropriate authorization checks
• Audit tool usage
• Rate limit requests
• Monitor for abuse

Error Handling

• Don't expose internal errors to clients
• Log security-relevant errors
• Handle timeouts appropriately
• Clean up resources after errors
• Validate return values

Tool Annotations

• Provide readOnlyHint and destructiveHint annotations
• Remember annotations are hints, not security guarantees
• Clients should not make security-critical decisions based solely on annotations

---

8. Transport Best Practices

General Transport Guidelines

1. Handle connection lifecycle properly 2. Implement proper error handling 3. Use appropriate timeout values 4. Implement connection state management 5. Clean up resources on disconnection

Security Best Practices for Transport

• Follow security considerations for DNS rebinding attacks
• Implement proper authentication mechanisms
• Validate message formats
• Handle malformed messages gracefully

Stdio Transport Specific

• Local MCP servers should NOT log to stdout (interferes with protocol)
• Use stderr for logging messages
• Handle standard I/O streams properly

---

9. Testing Requirements

A comprehensive testing strategy should cover:

Functional Testing

• Verify correct execution with valid/invalid inputs

Integration Testing

• Test interaction with external systems

Security Testing

• Validate auth, input sanitization, rate limiting

Performance Testing

• Check behavior under load, timeouts

Error Handling

• Ensure proper error reporting and cleanup

---

10. OAuth and Security Best Practices

Authentication and Authorization

MCP servers that connect to external services should implement proper authentication:

OAuth 2.1 Implementation:

• Use secure OAuth 2.1 with certificates from recognized authorities
• Validate access tokens before processing requests
• Only accept tokens specifically intended for your server
• Reject tokens without proper audience claims
• Never pass through tokens received from MCP clients

API Key Management:

• Store API keys in environment variables, never in code
• Validate keys on server startup
• Provide clear error messages when authentication fails
• Use secure transmission for sensitive credentials

Input Validation and Security

Always validate inputs:

• Sanitize file paths to prevent directory traversal
• Validate URLs and external identifiers
• Check parameter sizes and ranges
• Prevent command injection in system calls
• Use schema validation (Pydantic/Zod) for all inputs

Error handling security:

• Don't expose internal errors to clients
• Log security-relevant errors server-side
• Provide helpful but not revealing error messages
• Clean up resources after errors

Privacy and Data Protection

Data collection principles:

• Only collect data strictly necessary for functionality
• Don't collect extraneous conversation data
• Don't collect PII unless explicitly required for the tool's purpose
• Provide clear information about what data is accessed

Data transmission:

• Don't send data to servers outside your organization without disclosure
• Use secure transmission (HTTPS) for all network communication
• Validate certificates for external services

---

11. Resource Management Best Practices

1. Only suggest necessary resources 2. Use clear, descriptive names for roots 3. Handle resource boundaries properly 4. Respect client control over resources 5. Use model-controlled primitives (tools) for automatic data exposure

---

12. Prompt Management Best Practices

• Clients should show users proposed prompts
• Users should be able to modify or reject prompts
• Clients should show users completions
• Users should be able to modify or reject completions
• Consider costs when using sampling

---

13. Error Handling Standards

• Use standard JSON-RPC error codes
• Report tool errors within result objects (not protocol-level)
• Provide helpful, specific error messages
• Don't expose internal implementation details
• Clean up resources properly on errors

---

14. Documentation Requirements

• Provide clear documentation of all tools and capabilities
• Include working examples (at least 3 per major feature)
• Document security considerations
• Specify required permissions and access levels
• Document rate limits and performance characteristics

---

15. Compliance and Monitoring

• Implement logging for debugging and monitoring
• Track tool usage patterns
• Monitor for potential abuse
• Maintain audit trails for security-relevant operations
• Be prepared for ongoing compliance reviews

---

Summary

These best practices represent the comprehensive guidelines for building secure, efficient, and compliant MCP servers that work well within the ecosystem. Developers should follow these guidelines to ensure their MCP servers meet the standards for inclusion in the MCP directory and provide a safe, reliable experience for users.

----------

Tools

Enable LLMs to perform actions through your server

Tools are a powerful primitive in the Model Context Protocol (MCP) that enable servers to expose executable functionality to clients. Through tools, LLMs can interact with external systems, perform computations, and take actions in the real world.

<Note> Tools are designed to be model-controlled, meaning that tools are exposed from servers to clients with the intention of the AI model being able to automatically invoke them (with a human in the loop to grant approval). </Note>

Overview

Tools in MCP allow servers to expose executable functions that can be invoked by clients and used by LLMs to perform actions. Key aspects of tools include:

• Discovery: Clients can obtain a list of available tools by sending a tools/list request
• Invocation: Tools are called using the tools/call request, where servers perform the requested operation and return results
• Flexibility: Tools can range from simple calculations to complex API interactions

Like resources, tools are identified by unique names and can include descriptions to guide their usage. However, unlike resources, tools represent dynamic operations that can modify state or interact with external systems.

Tool definition structure

Each tool is defined with the following structure:

{
  name: string;          // Unique identifier for the tool
  description?: string;  // Human-readable description
  inputSchema: {         // JSON Schema for the tool's parameters
    type: "object",
    properties: { ... }  // Tool-specific parameters
  },
  annotations?: {        // Optional hints about tool behavior
    title?: string;      // Human-readable title for the tool
    readOnlyHint?: boolean;    // If true, the tool does not modify its environment
    destructiveHint?: boolean; // If true, the tool may perform destructive updates
    idempotentHint?: boolean;  // If true, repeated calls with same args have no additional effect
    openWorldHint?: boolean;   // If true, tool interacts with external entities
  }
}

Implementing tools

Here's an example of implementing a basic tool in an MCP server:

<Tabs> <Tab title="TypeScript">

    const server = new Server({
      name: "example-server",
      version: "1.0.0"
    }, {
      capabilities: {
        tools: {}
      }
    });

    // Define available tools
    server.setRequestHandler(ListToolsRequestSchema, async () => {
      return {
        tools: [{
          name: "calculate_sum",
          description: "Add two numbers together",
          inputSchema: {
            type: "object",
            properties: {
              a: { type: "number" },
              b: { type: "number" }
            },
            required: ["a", "b"]
          }
        }]
      };
    });

    // Handle tool execution
    server.setRequestHandler(CallToolRequestSchema, async (request) => {
      if (request.params.name === "calculate_sum") {
        const { a, b } = request.params.arguments;
        return {
          content: [
            {
              type: "text",
              text: String(a + b)
            }
          ]
        };
      }
      throw new Error("Tool not found");
    });

</Tab>

<Tab title="Python">

    app = Server("example-server")

    @app.list_tools()
    async def list_tools() -> list[types.Tool]:
        return [
            types.Tool(
                name="calculate_sum",
                description="Add two numbers together",
                inputSchema={
                    "type": "object",
                    "properties": {
                        "a": {"type": "number"},
                        "b": {"type": "number"}
                    },
                    "required": ["a", "b"]
                }
            )
        ]

    @app.call_tool()
    async def call_tool(
        name: str,
        arguments: dict
    ) -> list[types.TextContent | types.ImageContent | types.EmbeddedResource]:
        if name == "calculate_sum":
            a = arguments["a"]
            b = arguments["b"]
            result = a + b
            return [types.TextContent(type="text", text=str(result))]
        raise ValueError(f"Tool not found: {name}")

</Tab> </Tabs>

Example tool patterns

Here are some examples of types of tools that a server could provide:

System operations

Tools that interact with the local system:

{
  name: "execute_command",
  description: "Run a shell command",
  inputSchema: {
    type: "object",
    properties: {
      command: { type: "string" },
      args: { type: "array", items: { type: "string" } }
    }
  }
}

API integrations

Tools that wrap external APIs:

{
  name: "github_create_issue",
  description: "Create a GitHub issue",
  inputSchema: {
    type: "object",
    properties: {
      title: { type: "string" },
      body: { type: "string" },
      labels: { type: "array", items: { type: "string" } }
    }
  }
}

Data processing

Tools that transform or analyze data:

{
  name: "analyze_csv",
  description: "Analyze a CSV file",
  inputSchema: {
    type: "object",
    properties: {
      filepath: { type: "string" },
      operations: {
        type: "array",
        items: {
          enum: ["sum", "average", "count"]
        }
      }
    }
  }
}

Best practices

When implementing tools:

1. Provide clear, descriptive names and descriptions 2. Use detailed JSON Schema definitions for parameters 3. Include examples in tool descriptions to demonstrate how the model should use them 4. Implement proper error handling and validation 5. Use progress reporting for long operations 6. Keep tool operations focused and atomic 7. Document expected return value structures 8. Implement proper timeouts 9. Consider rate limiting for resource-intensive operations 10. Log tool usage for debugging and monitoring

Tool name conflicts

MCP client applications and MCP server proxies may encounter tool name conflicts when building their own tool lists. For example, two connected MCP servers web1 and web2 may both expose a tool named search_web.

Applications may disambiguiate tools with one of the following strategies (among others; not an exhaustive list):

• Concatenating a unique, user-defined server name with the tool name, e.g. web1___search_web and web2___search_web. This strategy may be preferable when unique server names are already provided by the user in a configuration file.
• Generating a random prefix for the tool name, e.g. jrwxs___search_web and 6cq52___search_web. This strategy may be preferable in server proxies where user-defined unique names are not available.
• Using the server URI as a prefix for the tool name, e.g. web1.example.com:search_web and web2.example.com:search_web. This strategy may be suitable when working with remote MCP servers.

Note that the server-provided name from the initialization flow is not guaranteed to be unique and is not generally suitable for disambiguation purposes.

Security considerations

When exposing tools:

Input validation

• Validate all parameters against the schema
• Sanitize file paths and system commands
• Validate URLs and external identifiers
• Check parameter sizes and ranges
• Prevent command injection

Access control

• Implement authentication where needed
• Use appropriate authorization checks
• Audit tool usage
• Rate limit requests
• Monitor for abuse

Error handling

• Don't expose internal errors to clients
• Log security-relevant errors
• Handle timeouts appropriately
• Clean up resources after errors
• Validate return values

Tool discovery and updates

MCP supports dynamic tool discovery:

1. Clients can list available tools at any time 2. Servers can notify clients when tools change using notifications/tools/list_changed 3. Tools can be added or removed during runtime 4. Tool definitions can be updated (though this should be done carefully)

Error handling

Tool errors should be reported within the result object, not as MCP protocol-level errors. This allows the LLM to see and potentially handle the error. When a tool encounters an error:

1. Set isError to true in the result 2. Include error details in the content array

Here's an example of proper error handling for tools:

<Tabs> <Tab title="TypeScript">

    try {
      // Tool operation
      const result = performOperation();
      return {
        content: [
          {
            type: "text",
            text: `Operation successful: ${result}`
          }
        ]
      };
    } catch (error) {
      return {
        isError: true,
        content: [
          {
            type: "text",
            text: `Error: ${error.message}`
          }
        ]
      };
    }

</Tab>

<Tab title="Python">

    try:
        # Tool operation
        result = perform_operation()
        return types.CallToolResult(
            content=[
                types.TextContent(
                    type="text",
                    text=f"Operation successful: {result}"
                )
            ]
        )
    except Exception as error:
        return types.CallToolResult(
            isError=True,
            content=[
                types.TextContent(
                    type="text",
                    text=f"Error: {str(error)}"
                )
            ]
        )

</Tab> </Tabs>

This approach allows the LLM to see that an error occurred and potentially take corrective action or request human intervention.

Tool annotations

Tool annotations provide additional metadata about a tool's behavior, helping clients understand how to present and manage tools. These annotations are hints that describe the nature and impact of a tool, but should not be relied upon for security decisions.

Purpose of tool annotations

Tool annotations serve several key purposes:

1. Provide UX-specific information without affecting model context 2. Help clients categorize and present tools appropriately 3. Convey information about a tool's potential side effects 4. Assist in developing intuitive interfaces for tool approval

Available tool annotations

The MCP specification defines the following annotations for tools:

Example usage

Here's how to define tools with annotations for different scenarios:

// A read-only search tool
{
  name: "web_search",
  description: "Search the web for information",
  inputSchema: {
    type: "object",
    properties: {
      query: { type: "string" }
    },
    required: ["query"]
  },
  annotations: {
    title: "Web Search",
    readOnlyHint: true,
    openWorldHint: true
  }
}

// A destructive file deletion tool
{
  name: "delete_file",
  description: "Delete a file from the filesystem",
  inputSchema: {
    type: "object",
    properties: {
      path: { type: "string" }
    },
    required: ["path"]
  },
  annotations: {
    title: "Delete File",
    readOnlyHint: false,
    destructiveHint: true,
    idempotentHint: true,
    openWorldHint: false
  }
}

// A non-destructive database record creation tool
{
  name: "create_record",
  description: "Create a new record in the database",
  inputSchema: {
    type: "object",
    properties: {
      table: { type: "string" },
      data: { type: "object" }
    },
    required: ["table", "data"]
  },
  annotations: {
    title: "Create Database Record",
    readOnlyHint: false,
    destructiveHint: false,
    idempotentHint: false,
    openWorldHint: false
  }
}

Integrating annotations in server implementation

<Tabs> <Tab title="TypeScript">

    server.setRequestHandler(ListToolsRequestSchema, async () => {
      return {
        tools: [{
          name: "calculate_sum",
          description: "Add two numbers together",
          inputSchema: {
            type: "object",
            properties: {
              a: { type: "number" },
              b: { type: "number" }
            },
            required: ["a", "b"]
          },
          annotations: {
            title: "Calculate Sum",
            readOnlyHint: true,
            openWorldHint: false
          }
        }]
      };
    });

</Tab>

<Tab title="Python">

    from mcp.server.fastmcp import FastMCP

    mcp = FastMCP("example-server")

    @mcp.tool(
        annotations={
            "title": "Calculate Sum",
            "readOnlyHint": True,
            "openWorldHint": False
        }
    )
    async def calculate_sum(a: float, b: float) -> str:
        """Add two numbers together.

        Args:
            a: First number to add
            b: Second number to add
        """
        result = a + b
        return str(result)

</Tab> </Tabs>

Best practices for tool annotations

1. Be accurate about side effects: Clearly indicate whether a tool modifies its environment and whether those modifications are destructive.

2. Use descriptive titles: Provide human-friendly titles that clearly describe the tool's purpose.

3. Indicate idempotency properly: Mark tools as idempotent only if repeated calls with the same arguments truly have no additional effect.

4. Set appropriate open/closed world hints: Indicate whether a tool interacts with a closed system (like a database) or an open system (like the web).

5. Remember annotations are hints: All properties in ToolAnnotations are hints and not guaranteed to provide a faithful description of tool behavior. Clients should never make security-critical decisions based solely on annotations.

Testing tools

A comprehensive testing strategy for MCP tools should cover:

• Functional testing: Verify tools execute correctly with valid inputs and handle invalid inputs appropriately
• Integration testing: Test tool interaction with external systems using both real and mocked dependencies
• Security testing: Validate authentication, authorization, input sanitization, and rate limiting
• Performance testing: Check behavior under load, timeout handling, and resource cleanup
• Error handling: Ensure tools properly report errors through the MCP protocol and clean up resources

Building MCP Servers: Complete Guide

Comprehensive guide for creating high-quality MCP servers that enable LLMs to accomplish real-world tasks.

---

Table of Contents

1. Agent-Centric Design Principles 2. Development Workflow 3. Python Implementation (FastMCP) 4. TypeScript Implementation (MCP SDK) 5. Tool Design Patterns 6. Response Formats 7. Pagination & Character Limits 8. Error Handling 9. Creating Evaluations

---

Agent-Centric Design Principles

Build tools for AI agents, not just API wrappers.

Build for Workflows, Not Endpoints

• Consolidate related operations (e.g., schedule_event checks availability AND creates event)
• Focus on complete tasks, not individual API calls
• Consider what workflows agents actually need to accomplish

Optimize for Limited Context

• Agents have constrained context windows - make every token count
• Return high-signal information, not exhaustive data dumps
• Provide "concise" vs "detailed" response format options
• Default to human-readable identifiers over technical codes

Design Actionable Error Messages

• Error messages should guide agents toward correct usage
• Suggest specific next steps: "Try using filter='active_only'"
• Make errors educational, not just diagnostic

Follow Natural Task Subdivisions

• Tool names should reflect how humans think about tasks
• Group related tools with consistent prefixes
• Design around natural workflows, not just API structure

Use Evaluation-Driven Development

• Create realistic evaluation scenarios early
• Let agent feedback drive tool improvements
• Prototype quickly and iterate based on actual agent performance

---

Development Workflow

Phase 1: Research & Planning

1.1 Study MCP Protocol

• Fetch: https://modelcontextprotocol.io/llms-full.txt
• Understand tools, resources, prompts, transports

1.2 Study Framework Documentation

Python:

• Fetch: https://raw.githubusercontent.com/modelcontextprotocol/python-sdk/main/README.md
• Review: references/python-guide.md

TypeScript:

• Fetch: https://raw.githubusercontent.com/modelcontextprotocol/typescript-sdk/main/README.md
• Review: references/typescript-guide.md

1.3 Study API Documentation Exhaustively

• Official API reference
• Authentication requirements
• Rate limiting, pagination
• Error responses
• Available endpoints and parameters
• Data models and schemas

1.4 Create Implementation Plan

• List most valuable endpoints to implement
• Identify shared utilities needed
• Design input/output formats
• Plan error handling strategies
• Consider pagination and character limits

Phase 2: Implementation

2.1 Set Up Project Structure

Python: Single .py file or module structure TypeScript: Full project with src/, package.json, tsconfig.json

2.2 Implement Core Infrastructure First

• API request helper functions
• Error handling utilities
• Response formatting (JSON and Markdown)
• Pagination helpers
• Authentication/token management

2.3 Implement Tools Systematically

For each tool: 1. Define input schema (Pydantic or Zod) 2. Write comprehensive descriptions 3. Implement tool logic using shared utilities 4. Add proper annotations (readOnlyHint, etc.) 5. Handle errors gracefully

2.4 Follow Language-Specific Best Practices

Load appropriate guide:

• Python → references/python-guide.md
• TypeScript → references/typescript-guide.md

Phase 3: Review & Testing

3.1 Code Quality Review

• DRY: No duplicated code
• Composability: Shared logic extracted
• Consistency: Similar operations return similar formats
• Error Handling: All external calls covered
• Type Safety: Full type coverage
• Documentation: Comprehensive docstrings

3.2 Test & Build

Python:

python -m py_compile your_server.py
# Or use evaluation harness

TypeScript:

npm run build  # Must complete without errors

IMPORTANT: Servers are long-running processes. Don't run directly in main thread. Use:

• Evaluation harness (recommended)
• tmux for manual testing
• timeout 5s for quick checks

3.3 Quality Checklist

See language-specific guide for complete checklist.

Phase 4: Evaluation

Create 10 realistic, complex evaluation questions.

Load references/evaluation-guide.md for complete methodology.

Key Requirements:

• Questions must be READ-ONLY and NON-DESTRUCTIVE
• Each requires multiple tool calls (potentially dozens)
• Answers must be single, verifiable values
• Answers must be STABLE (won't change over time)

---

Python Implementation (FastMCP)

Complete guide: references/python-guide.md

Quick Start

from mcp.server.fastmcp import FastMCP
from pydantic import BaseModel, Field, field_validator, ConfigDict

mcp = FastMCP("service_mcp")

class SearchInput(BaseModel):
    model_config = ConfigDict(str_strip_whitespace=True, validate_assignment=True)
    
    query: str = Field(..., min_length=2, max_length=200, 
                       description="Search query")
    limit: int = Field(default=20, ge=1, le=100)

@mcp.tool(
    name="service_search",
    annotations={
        "readOnlyHint": True,
        "destructiveHint": False,
        "idempotentHint": True,
        "openWorldHint": True
    }
)
async def search(params: SearchInput) -> str:
    """Search for items in the service.
    
    Args:
        params (SearchInput): Validated search parameters
        
    Returns:
        str: JSON-formatted search results
    """
    # Implementation
    pass

if __name__ == "__main__":
    mcp.run()

Key Features

• Automatic schema generation from Pydantic models
• Decorator-based tool registration (@mcp.tool)
• Type hints throughout
• Async/await for all I/O
• Context injection for progress reporting

Full details: references/python-guide.md

---

TypeScript Implementation (MCP SDK)

Complete guide: references/typescript-guide.md

Quick Start

import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import { z } from "zod";

const server = new McpServer({
  name: "service-mcp-server",
  version: "1.0.0"
});

const SearchInputSchema = z.object({
  query: z.string().min(2).max(200).describe("Search query"),
  limit: z.number().int().min(1).max(100).default(20)
}).strict();

server.registerTool(
  "service_search",
  {
    title: "Search Service",
    description: "Search for items in the service",
    inputSchema: SearchInputSchema,
    annotations: {
      readOnlyHint: true,
      destructiveHint: false,
      idempotentHint: true,
      openWorldHint: true
    }
  },
  async (params: z.infer<typeof SearchInputSchema>) => {
    // Implementation
    return {
      content: [{type: "text", text: JSON.stringify(result)}]
    };
  }
);

async function main() {
  const transport = new StdioServerTransport();
  await server.connect(transport);
}

main();

Key Features

• Zod schemas for runtime validation
• Explicit registerTool configuration
• TypeScript strict mode
• No any types
• Build process required (npm run build)

Full details: references/typescript-guide.md

---

Tool Design Patterns

Naming Conventions

Server Names:

• Python: {service}_mcp (e.g., slack_mcp)
• TypeScript: {service}-mcp-server (e.g., slack-mcp-server)

Tool Names:

• Format: {service}_{action}_{resource}
• Use snake_case
• Include service prefix to avoid conflicts
• Examples:
• slack_send_message (not send_message)
• github_create_issue (not create_issue)
• asana_list_tasks (not list_tasks)

Tool Annotations

annotations={
    "readOnlyHint": True,      # Tool doesn't modify environment
    "destructiveHint": False,  # Tool doesn't perform destructive updates
    "idempotentHint": True,    # Repeated calls have no additional effect
    "openWorldHint": True      # Tool interacts with external entities
}

Note: Annotations are hints, not security guarantees.

Tool Documentation

Every tool needs:

• One-line summary
• Detailed explanation of purpose
• Explicit parameter types with examples
• Complete return type schema
• Usage examples (when to use, when not to use)
• Error handling documentation

---

Response Formats

Support both JSON and Markdown for flexibility.

JSON Format (response_format="json")

• Machine-readable structured data
• Include all available fields
• Consistent field names and types
• Use for programmatic processing

Markdown Format (response_format="markdown")

• Human-readable formatted text
• Use headers, lists, formatting
• Convert timestamps to readable format
• Show names with IDs in parentheses
• Omit verbose metadata
• Use for presenting to users

Implementation

Python:

from enum import Enum

class ResponseFormat(str, Enum):
    MARKDOWN = "markdown"
    JSON = "json"

class SearchInput(BaseModel):
    response_format: ResponseFormat = Field(default=ResponseFormat.MARKDOWN)

TypeScript:

enum ResponseFormat {
  MARKDOWN = "markdown",
  JSON = "json"
}

const schema = z.object({
  response_format: z.nativeEnum(ResponseFormat).default(ResponseFormat.MARKDOWN)
});

---

Pagination & Character Limits

Pagination

Always implement for list operations:

class ListInput(BaseModel):
    limit: Optional[int] = Field(default=20, ge=1, le=100)
    offset: Optional[int] = Field(default=0, ge=0)

# Response structure
{
    "total": 150,
    "count": 20,
    "offset": 0,
    "items": [...],
    "has_more": True,
    "next_offset": 20
}

Character Limits

Prevent overwhelming responses:

CHARACTER_LIMIT = 25000  # Module-level constant

if len(result) > CHARACTER_LIMIT:
    truncated_data = data[:max(1, len(data) // 2)]
    response["truncated"] = True
    response["truncation_message"] = (
        f"Response truncated from {len(data)} to {len(truncated_data)} items. "
        f"Use 'offset' parameter or add filters to see more results."
    )

---

Error Handling

Principles

• Clear, actionable error messages
• Guide agents toward correct usage
• Don't expose internal errors
• Log security-relevant errors
• Clean up resources

Implementation

Python:

def _handle_api_error(e: Exception) -> str:
    if isinstance(e, httpx.HTTPStatusError):
        if e.response.status_code == 404:
            return "Error: Resource not found. Please check the ID is correct."
        elif e.response.status_code == 403:
            return "Error: Permission denied. You don't have access."
        elif e.response.status_code == 429:
            return "Error: Rate limit exceeded. Please wait."
    return f"Error: Unexpected error occurred: {type(e).__name__}"

TypeScript:

function handleApiError(error: unknown): string {
  if (error instanceof AxiosError) {
    switch (error.response?.status) {
      case 404: return "Error: Resource not found. Check the ID.";
      case 403: return "Error: Permission denied.";
      case 429: return "Error: Rate limit exceeded. Wait before retrying.";
    }
  }
  return `Error: ${error instanceof Error ? error.message : String(error)}`;
}

---

Creating Evaluations

Complete guide: references/evaluation-guide.md

Overview

Create 10 questions that test whether LLMs can use your server effectively.

Requirements

Questions must be:

• Independent (no dependencies on other questions)
• Read-only and non-destructive
• Complex (requiring multiple tool calls)
• Realistic (real human use cases)
• Unambiguous with clear single answer
• Based on stable data (won't change over time)

Answers must be:

• Single verifiable values (username, ID, count, date, etc.)
• Human-readable where possible
• Verifiable via direct string comparison
• Stable (based on historical/closed data)

Process

1. Study Documentation: Understand API and MCP server tools 2. Explore Content: Use READ-ONLY operations to find specific content 3. Generate Questions: Create 10 complex, realistic questions 4. Verify Answers: Solve each yourself to ensure correctness

Output Format

<evaluation>
  <qa_pair>
    <question>Find the project created in Q2 2024 with the highest number of completed tasks. What is the project name?</question>
    <answer>Website Redesign</answer>
  </qa_pair>
  <qa_pair>
    <question>Search for issues labeled "bug" closed in March 2024. Which user closed the most? Provide username.</question>
    <answer>sarah_dev</answer>
  </qa_pair>
</evaluation>

Running Evaluations

pip install anthropic mcp
export ANTHROPIC_API_KEY=your_key

python scripts/evaluation.py \
  -t stdio \
  -c python \
  -a your_server.py \
  -e API_KEY=xxx \
  evaluation.xml

Load references/evaluation-guide.md for complete details.

---

Resources

MCP Protocol

• Spec: https://modelcontextprotocol.io/llms-full.txt
• Protocol basics: references/protocol-basics.md

SDK Documentation

• Python: https://raw.githubusercontent.com/modelcontextprotocol/python-sdk/main/README.md
• TypeScript: https://raw.githubusercontent.com/modelcontextprotocol/typescript-sdk/main/README.md

Implementation Guides

• Python: references/python-guide.md
• TypeScript: references/typescript-guide.md

Standards & Best Practices

• Best practices: references/best-practices.md
• Evaluation guide: references/evaluation-guide.md

---

Quality Checklist

Strategic Design

• [ ] Tools enable complete workflows, not just API wrappers
• [ ] Tool names reflect natural task subdivisions
• [ ] Response formats optimize for agent context efficiency
• [ ] Human-readable identifiers used where appropriate
• [ ] Error messages guide agents toward correct usage

Implementation

• [ ] Most important tools implemented
• [ ] All tools have descriptive names and documentation
• [ ] Proper annotations (readOnlyHint, etc.)
• [ ] Input validation with Pydantic/Zod
• [ ] Comprehensive docstrings with schemas
• [ ] Error handling for all external calls

Code Quality

• [ ] Common functionality extracted (DRY)
• [ ] Async/await for all I/O
• [ ] Type hints/strict TypeScript throughout
• [ ] Pagination implemented where applicable
• [ ] CHARACTER_LIMIT respected with clear truncation
• [ ] Consistent response formats

Testing

• [ ] Build completes successfully
• [ ] Sample tool calls work
• [ ] Evaluations created and passing

Load language-specific guide for complete checklist.

MCP Server Evaluation Guide

Overview

This document provides guidance on creating comprehensive evaluations for MCP servers. Evaluations test whether LLMs can effectively use your MCP server to answer realistic, complex questions using only the tools provided.

---

Quick Reference

Evaluation Requirements

• Create 10 human-readable questions
• Questions must be READ-ONLY, INDEPENDENT, NON-DESTRUCTIVE
• Each question requires multiple tool calls (potentially dozens)
• Answers must be single, verifiable values
• Answers must be STABLE (won't change over time)

Output Format

<evaluation>
   <qa_pair>
      <question>Your question here</question>
      <answer>Single verifiable answer</answer>
   </qa_pair>
</evaluation>

---

Purpose of Evaluations

The measure of quality of an MCP server is NOT how well or comprehensively the server implements tools, but how well these implementations (input/output schemas, docstrings/descriptions, functionality) enable LLMs with no other context and access ONLY to the MCP servers to answer realistic and difficult questions.

Evaluation Overview

Create 10 human-readable questions requiring ONLY READ-ONLY, INDEPENDENT, NON-DESTRUCTIVE, and IDEMPOTENT operations to answer. Each question should be:

• Realistic
• Clear and concise
• Unambiguous
• Complex, requiring potentially dozens of tool calls or steps
• Answerable with a single, verifiable value that you identify in advance

Question Guidelines

Core Requirements

1. Questions MUST be independent

• Each question should NOT depend on the answer to any other question
• Should not assume prior write operations from processing another question

2. Questions MUST require ONLY NON-DESTRUCTIVE AND IDEMPOTENT tool use

• Should not instruct or require modifying state to arrive at the correct answer

3. Questions must be REALISTIC, CLEAR, CONCISE, and COMPLEX

• Must require another LLM to use multiple (potentially dozens of) tools or steps to answer

Complexity and Depth

4. Questions must require deep exploration

• Consider multi-hop questions requiring multiple sub-questions and sequential tool calls
• Each step should benefit from information found in previous questions

5. Questions may require extensive paging

• May need paging through multiple pages of results
• May require querying old data (1-2 years out-of-date) to find niche information
• The questions must be DIFFICULT

6. Questions must require deep understanding

• Rather than surface-level knowledge
• May pose complex ideas as True/False questions requiring evidence
• May use multiple-choice format where LLM must search different hypotheses

7. Questions must not be solvable with straightforward keyword search

• Do not include specific keywords from the target content
• Use synonyms, related concepts, or paraphrases
• Require multiple searches, analyzing multiple related items, extracting context, then deriving the answer

Tool Testing

8. Questions should stress-test tool return values

• May elicit tools returning large JSON objects or lists, overwhelming the LLM
• Should require understanding multiple modalities of data:
• IDs and names
• Timestamps and datetimes (months, days, years, seconds)
• File IDs, names, extensions, and mimetypes
• URLs, GIDs, etc.
• Should probe the tool's ability to return all useful forms of data

9. Questions should MOSTLY reflect real human use cases

• The kinds of information retrieval tasks that HUMANS assisted by an LLM would care about

10. Questions may require dozens of tool calls

• This challenges LLMs with limited context
• Encourages MCP server tools to reduce information returned

11. Include ambiguous questions

• May be ambiguous OR require difficult decisions on which tools to call
• Force the LLM to potentially make mistakes or misinterpret
• Ensure that despite AMBIGUITY, there is STILL A SINGLE VERIFIABLE ANSWER

Stability

12. Questions must be designed so the answer DOES NOT CHANGE

• Do not ask questions that rely on "current state" which is dynamic
• For example, do not count:
• Number of reactions to a post
• Number of replies to a thread
• Number of members in a channel

13. DO NOT let the MCP server RESTRICT the kinds of questions you create

• Create challenging and complex questions
• Some may not be solvable with the available MCP server tools
• Questions may require specific output formats (datetime vs. epoch time, JSON vs. MARKDOWN)
• Questions may require dozens of tool calls to complete

Answer Guidelines

Verification

1. Answers must be VERIFIABLE via direct string comparison

• If the answer can be re-written in many formats, clearly specify the output format in the QUESTION
• Examples: "Use YYYY/MM/DD.", "Respond True or False.", "Answer A, B, C, or D and nothing else."
• Answer should be a single VERIFIABLE value such as:
• User ID, user name, display name, first name, last name
• Channel ID, channel name
• Message ID, string
• URL, title
• Numerical quantity
• Timestamp, datetime
• Boolean (for True/False questions)
• Email address, phone number
• File ID, file name, file extension
• Multiple choice answer
• Answers must not require special formatting or complex, structured output
• Answer will be verified using DIRECT STRING COMPARISON

Readability

2. Answers should generally prefer HUMAN-READABLE formats

• Examples: names, first name, last name, datetime, file name, message string, URL, yes/no, true/false, a/b/c/d
• Rather than opaque IDs (though IDs are acceptable)
• The VAST MAJORITY of answers should be human-readable

Stability

3. Answers must be STABLE/STATIONARY

• Look at old content (e.g., conversations that have ended, projects that have launched, questions answered)
• Create QUESTIONS based on "closed" concepts that will always return the same answer
• Questions may ask to consider a fixed time window to insulate from non-stationary answers
• Rely on context UNLIKELY to change
• Example: if finding a paper name, be SPECIFIC enough so answer is not confused with papers published later

4. Answers must be CLEAR and UNAMBIGUOUS

• Questions must be designed so there is a single, clear answer
• Answer can be derived from using the MCP server tools

Diversity

5. Answers must be DIVERSE

• Answer should be a single VERIFIABLE value in diverse modalities and formats
• User concept: user ID, user name, display name, first name, last name, email address, phone number
• Channel concept: channel ID, channel name, channel topic
• Message concept: message ID, message string, timestamp, month, day, year

6. Answers must NOT be complex structures

• Not a list of values
• Not a complex object
• Not a list of IDs or strings
• Not natural language text
• UNLESS the answer can be straightforwardly verified using DIRECT STRING COMPARISON
• And can be realistically reproduced
• It should be unlikely that an LLM would return the same list in any other order or format

Evaluation Process

Step 1: Documentation Inspection

Read the documentation of the target API to understand:

• Available endpoints and functionality
• If ambiguity exists, fetch additional information from the web
• Parallelize this step AS MUCH AS POSSIBLE
• Ensure each subagent is ONLY examining documentation from the file system or on the web

Step 2: Tool Inspection

List the tools available in the MCP server:

• Inspect the MCP server directly
• Understand input/output schemas, docstrings, and descriptions
• WITHOUT calling the tools themselves at this stage

Step 3: Developing Understanding

Repeat steps 1 & 2 until you have a good understanding:

• Iterate multiple times
• Think about the kinds of tasks you want to create
• Refine your understanding
• At NO stage should you READ the code of the MCP server implementation itself
• Use your intuition and understanding to create reasonable, realistic, but VERY challenging tasks

Step 4: Read-Only Content Inspection

After understanding the API and tools, USE the MCP server tools:

• Inspect content using READ-ONLY and NON-DESTRUCTIVE operations ONLY
• Goal: identify specific content (e.g., users, channels, messages, projects, tasks) for creating realistic questions
• Should NOT call any tools that modify state
• Will NOT read the code of the MCP server implementation itself
• Parallelize this step with individual sub-agents pursuing independent explorations
• Ensure each subagent is only performing READ-ONLY, NON-DESTRUCTIVE, and IDEMPOTENT operations
• BE CAREFUL: SOME TOOLS may return LOTS OF DATA which would cause you to run out of CONTEXT
• Make INCREMENTAL, SMALL, AND TARGETED tool calls for exploration
• In all tool call requests, use the limit parameter to limit results (<10)
• Use pagination

Step 5: Task Generation

After inspecting the content, create 10 human-readable questions:

• An LLM should be able to answer these with the MCP server
• Follow all question and answer guidelines above

Output Format

Each QA pair consists of a question and an answer. The output should be an XML file with this structure:

<evaluation>
   <qa_pair>
      <question>Find the project created in Q2 2024 with the highest number of completed tasks. What is the project name?</question>
      <answer>Website Redesign</answer>
   </qa_pair>
   <qa_pair>
      <question>Search for issues labeled as "bug" that were closed in March 2024. Which user closed the most issues? Provide their username.</question>
      <answer>sarah_dev</answer>
   </qa_pair>
   <qa_pair>
      <question>Look for pull requests that modified files in the /api directory and were merged between January 1 and January 31, 2024. How many different contributors worked on these PRs?</question>
      <answer>7</answer>
   </qa_pair>
   <qa_pair>
      <question>Find the repository with the most stars that was created before 2023. What is the repository name?</question>
      <answer>data-pipeline</answer>
   </qa_pair>
</evaluation>

Evaluation Examples

Good Questions

Example 1: Multi-hop question requiring deep exploration (GitHub MCP)

<qa_pair>
   <question>Find the repository that was archived in Q3 2023 and had previously been the most forked project in the organization. What was the primary programming language used in that repository?</question>
   <answer>Python</answer>
</qa_pair>

This question is good because:

• Requires multiple searches to find archived repositories
• Needs to identify which had the most forks before archival
• Requires examining repository details for the language
• Answer is a simple, verifiable value
• Based on historical (closed) data that won't change

Example 2: Requires understanding context without keyword matching (Project Management MCP)

<qa_pair>
   <question>Locate the initiative focused on improving customer onboarding that was completed in late 2023. The project lead created a retrospective document after completion. What was the lead's role title at that time?</question>
   <answer>Product Manager</answer>
</qa_pair>

This question is good because:

• Doesn't use specific project name ("initiative focused on improving customer onboarding")
• Requires finding completed projects from specific timeframe
• Needs to identify the project lead and their role
• Requires understanding context from retrospective documents
• Answer is human-readable and stable
• Based on completed work (won't change)

Example 3: Complex aggregation requiring multiple steps (Issue Tracker MCP)

<qa_pair>
   <question>Among all bugs reported in January 2024 that were marked as critical priority, which assignee resolved the highest percentage of their assigned bugs within 48 hours? Provide the assignee's username.</question>
   <answer>alex_eng</answer>
</qa_pair>

This question is good because:

• Requires filtering bugs by date, priority, and status
• Needs to group by assignee and calculate resolution rates
• Requires understanding timestamps to determine 48-hour windows
• Tests pagination (potentially many bugs to process)
• Answer is a single username
• Based on historical data from specific time period

Example 4: Requires synthesis across multiple data types (CRM MCP)

<qa_pair>
   <question>Find the account that upgraded from the Starter to Enterprise plan in Q4 2023 and had the highest annual contract value. What industry does this account operate in?</question>
   <answer>Healthcare</answer>
</qa_pair>

This question is good because:

• Requires understanding subscription tier changes
• Needs to identify upgrade events in specific timeframe
• Requires comparing contract values
• Must access account industry information
• Answer is simple and verifiable
• Based on completed historical transactions

Poor Questions

Example 1: Answer changes over time

<qa_pair>
   <question>How many open issues are currently assigned to the engineering team?</question>
   <answer>47</answer>
</qa_pair>

This question is poor because:

• The answer will change as issues are created, closed, or reassigned
• Not based on stable/stationary data
• Relies on "current state" which is dynamic

Example 2: Too easy with keyword search

<qa_pair>
   <question>Find the pull request with title "Add authentication feature" and tell me who created it.</question>
   <answer>developer123</answer>
</qa_pair>

This question is poor because:

• Can be solved with a straightforward keyword search for exact title
• Doesn't require deep exploration or understanding
• No synthesis or analysis needed

Example 3: Ambiguous answer format

<qa_pair>
   <question>List all the repositories that have Python as their primary language.</question>
   <answer>repo1, repo2, repo3, data-pipeline, ml-tools</answer>
</qa_pair>

This question is poor because:

• Answer is a list that could be returned in any order
• Difficult to verify with direct string comparison
• LLM might format differently (JSON array, comma-separated, newline-separated)
• Better to ask for a specific aggregate (count) or superlative (most stars)

Verification Process

After creating evaluations:

1. Examine the XML file to understand the schema 2. Load each task instruction and in parallel using the MCP server and tools, identify the correct answer by attempting to solve the task YOURSELF 3. Flag any operations that require WRITE or DESTRUCTIVE operations 4. Accumulate all CORRECT answers and replace any incorrect answers in the document 5. Remove any `<qa_pair>` that require WRITE or DESTRUCTIVE operations

Remember to parallelize solving tasks to avoid running out of context, then accumulate all answers and make changes to the file at the end.

Tips for Creating Quality Evaluations

1. Think Hard and Plan Ahead before generating tasks 2. Parallelize Where Opportunity Arises to speed up the process and manage context 3. Focus on Realistic Use Cases that humans would actually want to accomplish 4. Create Challenging Questions that test the limits of the MCP server's capabilities 5. Ensure Stability by using historical data and closed concepts 6. Verify Answers by solving the questions yourself using the MCP server tools 7. Iterate and Refine based on what you learn during the process

---

Running Evaluations

After creating your evaluation file, you can use the provided evaluation harness to test your MCP server.

Setup

1. Install Dependencies

   pip install -r scripts/requirements.txt

Or install manually:

   pip install anthropic mcp

2. Set API Key

   export ANTHROPIC_API_KEY=your_api_key_here

Evaluation File Format

Evaluation files use XML format with <qa_pair> elements:

<evaluation>
   <qa_pair>
      <question>Find the project created in Q2 2024 with the highest number of completed tasks. What is the project name?</question>
      <answer>Website Redesign</answer>
   </qa_pair>
   <qa_pair>
      <question>Search for issues labeled as "bug" that were closed in March 2024. Which user closed the most issues? Provide their username.</question>
      <answer>sarah_dev</answer>
   </qa_pair>
</evaluation>

Running Evaluations

The evaluation script (scripts/evaluation.py) supports three transport types:

Important:

• stdio transport: The evaluation script automatically launches and manages the MCP server process for you. Do not run the server manually.
• sse/http transports: You must start the MCP server separately before running the evaluation. The script connects to the already-running server at the specified URL.

1. Local STDIO Server

For locally-run MCP servers (script launches the server automatically):

python scripts/evaluation.py \
  -t stdio \
  -c python \
  -a my_mcp_server.py \
  evaluation.xml

With environment variables:

python scripts/evaluation.py \
  -t stdio \
  -c python \
  -a my_mcp_server.py \
  -e API_KEY=abc123 \
  -e DEBUG=true \
  evaluation.xml

2. Server-Sent Events (SSE)

For SSE-based MCP servers (you must start the server first):

python scripts/evaluation.py \
  -t sse \
  -u https://example.com/mcp \
  -H "Authorization: Bearer token123" \
  -H "X-Custom-Header: value" \
  evaluation.xml

3. HTTP (Streamable HTTP)

For HTTP-based MCP servers (you must start the server first):

python scripts/evaluation.py \
  -t http \
  -u https://example.com/mcp \
  -H "Authorization: Bearer token123" \
  evaluation.xml

Command-Line Options

usage: evaluation.py [-h] [-t {stdio,sse,http}] [-m MODEL] [-c COMMAND]
                     [-a ARGS [ARGS ...]] [-e ENV [ENV ...]] [-u URL]
                     [-H HEADERS [HEADERS ...]] [-o OUTPUT]
                     eval_file

positional arguments:
  eval_file             Path to evaluation XML file

optional arguments:
  -h, --help            Show help message
  -t, --transport       Transport type: stdio, sse, or http (default: stdio)
  -m, --model           Claude model to use (default: claude-3-7-sonnet-20250219)
  -o, --output          Output file for report (default: print to stdout)

stdio options:
  -c, --command         Command to run MCP server (e.g., python, node)
  -a, --args            Arguments for the command (e.g., server.py)
  -e, --env             Environment variables in KEY=VALUE format

sse/http options:
  -u, --url             MCP server URL
  -H, --header          HTTP headers in 'Key: Value' format

Output

The evaluation script generates a detailed report including:

• Summary Statistics:
• Accuracy (correct/total)
• Average task duration
• Average tool calls per task
• Total tool calls

• Per-Task Results:
• Prompt and expected response
• Actual response from the agent
• Whether the answer was correct (✅/❌)
• Duration and tool call details
• Agent's summary of its approach
• Agent's feedback on the tools

Save Report to File

python scripts/evaluation.py \
  -t stdio \
  -c python \
  -a my_server.py \
  -o evaluation_report.md \
  evaluation.xml

Complete Example Workflow

Here's a complete example of creating and running an evaluation:

1. Create your evaluation file (my_evaluation.xml):

<evaluation>
   <qa_pair>
      <question>Find the user who created the most issues in January 2024. What is their username?</question>
      <answer>alice_developer</answer>
   </qa_pair>
   <qa_pair>
      <question>Among all pull requests merged in Q1 2024, which repository had the highest number? Provide the repository name.</question>
      <answer>backend-api</answer>
   </qa_pair>
   <qa_pair>
      <question>Find the project that was completed in December 2023 and had the longest duration from start to finish. How many days did it take?</question>
      <answer>127</answer>
   </qa_pair>
</evaluation>

2. Install dependencies:

pip install -r scripts/requirements.txt
export ANTHROPIC_API_KEY=your_api_key

3. Run evaluation:

python scripts/evaluation.py \
  -t stdio \
  -c python \
  -a github_mcp_server.py \
  -e GITHUB_TOKEN=ghp_xxx \
  -o github_eval_report.md \
  my_evaluation.xml

4. Review the report in github_eval_report.md to:

• See which questions passed/failed
• Read the agent's feedback on your tools
• Identify areas for improvement
• Iterate on your MCP server design

Troubleshooting

Connection Errors

If you get connection errors:

• STDIO: Verify the command and arguments are correct
• SSE/HTTP: Check the URL is accessible and headers are correct
• Ensure any required API keys are set in environment variables or headers

Low Accuracy

If many evaluations fail:

• Review the agent's feedback for each task
• Check if tool descriptions are clear and comprehensive
• Verify input parameters are well-documented
• Consider whether tools return too much or too little data
• Ensure error messages are actionable

Timeout Issues

If tasks are timing out:

• Use a more capable model (e.g., claude-3-7-sonnet-20250219)
• Check if tools are returning too much data
• Verify pagination is working correctly
• Consider simplifying complex questions

Model Context Protocol (MCP) Reference

Protocol Overview

MCP is JSON-RPC 2.0 based protocol for AI-tool integration.

Version: 2025-03-26 Foundation: JSON-RPC 2.0 Architecture: Client-Host-Server

Connection Lifecycle

1. Initialize: Client sends initialize request with capabilities 2. Response: Server responds with its capabilities 3. Handshake: Client sends notifications/initialized 4. Active: Bidirectional messaging 5. Shutdown: Close connections, cleanup

Core Capabilities

Tools (Executable Functions)

Tools are functions that servers expose for execution.

List Tools:

{"method": "tools/list"}

Call Tool:

{
  "method": "tools/call",
  "params": {
    "name": "tool_name",
    "arguments": {}
  }
}

Prompts (Interaction Templates)

Prompts are reusable templates for LLM interactions.

List Prompts:

{"method": "prompts/list"}

Get Prompt:

{
  "method": "prompts/get",
  "params": {
    "name": "prompt_name",
    "arguments": {}
  }
}

Resources (Data Sources)

Resources expose read-only data to clients.

List Resources:

{"method": "resources/list"}

Read Resource:

{
  "method": "resources/read",
  "params": {"uri": "resource://path"}
}

Transport Types

stdio (Local)

Server runs as subprocess. Messages via stdin/stdout.

const transport = new StdioClientTransport({
  command: 'node',
  args: ['server.js']
});

HTTP+SSE (Remote)

POST for requests, GET for server events.

const transport = new StreamableHTTPClientTransport({
  url: 'http://localhost:3000/mcp'
});

Error Codes

• -32700: Parse error
• -32600: Invalid request
• -32601: Method not found
• -32602: Invalid params
• -32603: Internal error
• -32002: Resource not found (MCP-specific)

Best Practices

1. Progressive Disclosure: Load tool definitions on-demand 2. Context Efficiency: Filter data before returning 3. Security: Validate inputs, sanitize outputs 4. Resource Management: Cleanup connections properly 5. Error Handling: Handle all error cases gracefully

Python MCP Server Implementation Guide

Overview

This document provides Python-specific best practices and examples for implementing MCP servers using the MCP Python SDK. It covers server setup, tool registration patterns, input validation with Pydantic, error handling, and complete working examples.

---

Quick Reference

Key Imports

from mcp.server.fastmcp import FastMCP
from pydantic import BaseModel, Field, field_validator, ConfigDict
from typing import Optional, List, Dict, Any
from enum import Enum
import httpx

Server Initialization

mcp = FastMCP("service_mcp")

Tool Registration Pattern

@mcp.tool(name="tool_name", annotations={...})
async def tool_function(params: InputModel) -> str:
    # Implementation
    pass

---

MCP Python SDK and FastMCP

The official MCP Python SDK provides FastMCP, a high-level framework for building MCP servers. It provides:

• Automatic description and inputSchema generation from function signatures and docstrings
• Pydantic model integration for input validation
• Decorator-based tool registration with @mcp.tool

For complete SDK documentation, use WebFetch to load: https://raw.githubusercontent.com/modelcontextprotocol/python-sdk/main/README.md

Server Naming Convention

Python MCP servers must follow this naming pattern:

• Format: {service}_mcp (lowercase with underscores)
• Examples: github_mcp, jira_mcp, stripe_mcp

The name should be:

• General (not tied to specific features)
• Descriptive of the service/API being integrated
• Easy to infer from the task description
• Without version numbers or dates

Tool Implementation

Tool Naming

Use snake_case for tool names (e.g., "search_users", "create_project", "get_channel_info") with clear, action-oriented names.

Avoid Naming Conflicts: Include the service context to prevent overlaps:

• Use "slack_send_message" instead of just "send_message"
• Use "github_create_issue" instead of just "create_issue"
• Use "asana_list_tasks" instead of just "list_tasks"

Tool Structure with FastMCP

Tools are defined using the @mcp.tool decorator with Pydantic models for input validation:

from pydantic import BaseModel, Field, ConfigDict
from mcp.server.fastmcp import FastMCP

# Initialize the MCP server
mcp = FastMCP("example_mcp")

# Define Pydantic model for input validation
class ServiceToolInput(BaseModel):
    '''Input model for service tool operation.'''
    model_config = ConfigDict(
        str_strip_whitespace=True,  # Auto-strip whitespace from strings
        validate_assignment=True,    # Validate on assignment
        extra='forbid'              # Forbid extra fields
    )

    param1: str = Field(..., description="First parameter description (e.g., 'user123', 'project-abc')", min_length=1, max_length=100)
    param2: Optional[int] = Field(default=None, description="Optional integer parameter with constraints", ge=0, le=1000)
    tags: Optional[List[str]] = Field(default_factory=list, description="List of tags to apply", max_items=10)

@mcp.tool(
    name="service_tool_name",
    annotations={
        "title": "Human-Readable Tool Title",
        "readOnlyHint": True,     # Tool does not modify environment
        "destructiveHint": False,  # Tool does not perform destructive operations
        "idempotentHint": True,    # Repeated calls have no additional effect
        "openWorldHint": False     # Tool does not interact with external entities
    }
)
async def service_tool_name(params: ServiceToolInput) -> str:
    '''Tool description automatically becomes the 'description' field.

    This tool performs a specific operation on the service. It validates all inputs
    using the ServiceToolInput Pydantic model before processing.

    Args:
        params (ServiceToolInput): Validated input parameters containing:
            - param1 (str): First parameter description
            - param2 (Optional[int]): Optional parameter with default
            - tags (Optional[List[str]]): List of tags

    Returns:
        str: JSON-formatted response containing operation results
    '''
    # Implementation here
    pass

Pydantic v2 Key Features

• Use model_config instead of nested Config class
• Use field_validator instead of deprecated validator
• Use model_dump() instead of deprecated dict()
• Validators require @classmethod decorator
• Type hints are required for validator methods

from pydantic import BaseModel, Field, field_validator, ConfigDict

class CreateUserInput(BaseModel):
    model_config = ConfigDict(
        str_strip_whitespace=True,
        validate_assignment=True
    )

    name: str = Field(..., description="User's full name", min_length=1, max_length=100)
    email: str = Field(..., description="User's email address", pattern=r'^[\w\.-]+@[\w\.-]+\.\w+$')
    age: int = Field(..., description="User's age", ge=0, le=150)

    @field_validator('email')
    @classmethod
    def validate_email(cls, v: str) -> str:
        if not v.strip():
            raise ValueError("Email cannot be empty")
        return v.lower()

Response Format Options

Support multiple output formats for flexibility:

from enum import Enum

class ResponseFormat(str, Enum):
    '''Output format for tool responses.'''
    MARKDOWN = "markdown"
    JSON = "json"

class UserSearchInput(BaseModel):
    query: str = Field(..., description="Search query")
    response_format: ResponseFormat = Field(
        default=ResponseFormat.MARKDOWN,
        description="Output format: 'markdown' for human-readable or 'json' for machine-readable"
    )

Markdown format:

• Use headers, lists, and formatting for clarity
• Convert timestamps to human-readable format (e.g., "2024-01-15 10:30:00 UTC" instead of epoch)
• Show display names with IDs in parentheses (e.g., "@john.doe (U123456)")
• Omit verbose metadata (e.g., show only one profile image URL, not all sizes)
• Group related information logically

JSON format:

• Return complete, structured data suitable for programmatic processing
• Include all available fields and metadata
• Use consistent field names and types

Pagination Implementation

For tools that list resources:

class ListInput(BaseModel):
    limit: Optional[int] = Field(default=20, description="Maximum results to return", ge=1, le=100)
    offset: Optional[int] = Field(default=0, description="Number of results to skip for pagination", ge=0)

async def list_items(params: ListInput) -> str:
    # Make API request with pagination
    data = await api_request(limit=params.limit, offset=params.offset)

    # Return pagination info
    response = {
        "total": data["total"],
        "count": len(data["items"]),
        "offset": params.offset,
        "items": data["items"],
        "has_more": data["total"] > params.offset + len(data["items"]),
        "next_offset": params.offset + len(data["items"]) if data["total"] > params.offset + len(data["items"]) else None
    }
    return json.dumps(response, indent=2)

Character Limits and Truncation

Add a CHARACTER_LIMIT constant to prevent overwhelming responses:

# At module level
CHARACTER_LIMIT = 25000  # Maximum response size in characters

async def search_tool(params: SearchInput) -> str:
    result = generate_response(data)

    # Check character limit and truncate if needed
    if len(result) > CHARACTER_LIMIT:
        # Truncate data and add notice
        truncated_data = data[:max(1, len(data) // 2)]
        response["data"] = truncated_data
        response["truncated"] = True
        response["truncation_message"] = (
            f"Response truncated from {len(data)} to {len(truncated_data)} items. "
            f"Use 'offset' parameter or add filters to see more results."
        )
        result = json.dumps(response, indent=2)

    return result

Error Handling

Provide clear, actionable error messages:

def _handle_api_error(e: Exception) -> str:
    '''Consistent error formatting across all tools.'''
    if isinstance(e, httpx.HTTPStatusError):
        if e.response.status_code == 404:
            return "Error: Resource not found. Please check the ID is correct."
        elif e.response.status_code == 403:
            return "Error: Permission denied. You don't have access to this resource."
        elif e.response.status_code == 429:
            return "Error: Rate limit exceeded. Please wait before making more requests."
        return f"Error: API request failed with status {e.response.status_code}"
    elif isinstance(e, httpx.TimeoutException):
        return "Error: Request timed out. Please try again."
    return f"Error: Unexpected error occurred: {type(e).__name__}"

Shared Utilities

Extract common functionality into reusable functions:

# Shared API request function
async def _make_api_request(endpoint: str, method: str = "GET", **kwargs) -> dict:
    '''Reusable function for all API calls.'''
    async with httpx.AsyncClient() as client:
        response = await client.request(
            method,
            f"{API_BASE_URL}/{endpoint}",
            timeout=30.0,
            **kwargs
        )
        response.raise_for_status()
        return response.json()

Async/Await Best Practices

Always use async/await for network requests and I/O operations:

# Good: Async network request
async def fetch_data(resource_id: str) -> dict:
    async with httpx.AsyncClient() as client:
        response = await client.get(f"{API_URL}/resource/{resource_id}")
        response.raise_for_status()
        return response.json()

# Bad: Synchronous request
def fetch_data(resource_id: str) -> dict:
    response = requests.get(f"{API_URL}/resource/{resource_id}")  # Blocks
    return response.json()

Type Hints

Use type hints throughout:

from typing import Optional, List, Dict, Any

async def get_user(user_id: str) -> Dict[str, Any]:
    data = await fetch_user(user_id)
    return {"id": data["id"], "name": data["name"]}

Tool Docstrings

Every tool must have comprehensive docstrings with explicit type information:

async def search_users(params: UserSearchInput) -> str:
    '''
    Search for users in the Example system by name, email, or team.

    This tool searches across all user profiles in the Example platform,
    supporting partial matches and various search filters. It does NOT
    create or modify users, only searches existing ones.

    Args:
        params (UserSearchInput): Validated input parameters containing:
            - query (str): Search string to match against names/emails (e.g., "john", "@example.com", "team:marketing")
            - limit (Optional[int]): Maximum results to return, between 1-100 (default: 20)
            - offset (Optional[int]): Number of results to skip for pagination (default: 0)

    Returns:
        str: JSON-formatted string containing search results with the following schema:

        Success response:
        {
            "total": int,           # Total number of matches found
            "count": int,           # Number of results in this response
            "offset": int,          # Current pagination offset
            "users": [
                {
                    "id": str,      # User ID (e.g., "U123456789")
                    "name": str,    # Full name (e.g., "John Doe")
                    "email": str,   # Email address (e.g., "john@example.com")
                    "team": str     # Team name (e.g., "Marketing") - optional
                }
            ]
        }

        Error response:
        "Error: <error message>" or "No users found matching '<query>'"

    Examples:
        - Use when: "Find all marketing team members" -> params with query="team:marketing"
        - Use when: "Search for John's account" -> params with query="john"
        - Don't use when: You need to create a user (use example_create_user instead)
        - Don't use when: You have a user ID and need full details (use example_get_user instead)

    Error Handling:
        - Input validation errors are handled by Pydantic model
        - Returns "Error: Rate limit exceeded" if too many requests (429 status)
        - Returns "Error: Invalid API authentication" if API key is invalid (401 status)
        - Returns formatted list of results or "No users found matching 'query'"
    '''

Complete Example

See below for a complete Python MCP server example:

#!/usr/bin/env python3
'''
MCP Server for Example Service.

This server provides tools to interact with Example API, including user search,
project management, and data export capabilities.
'''

from typing import Optional, List, Dict, Any
from enum import Enum
import httpx
from pydantic import BaseModel, Field, field_validator, ConfigDict
from mcp.server.fastmcp import FastMCP

# Initialize the MCP server
mcp = FastMCP("example_mcp")

# Constants
API_BASE_URL = "https://api.example.com/v1"
CHARACTER_LIMIT = 25000  # Maximum response size in characters

# Enums
class ResponseFormat(str, Enum):
    '''Output format for tool responses.'''
    MARKDOWN = "markdown"
    JSON = "json"

# Pydantic Models for Input Validation
class UserSearchInput(BaseModel):
    '''Input model for user search operations.'''
    model_config = ConfigDict(
        str_strip_whitespace=True,
        validate_assignment=True
    )

    query: str = Field(..., description="Search string to match against names/emails", min_length=2, max_length=200)
    limit: Optional[int] = Field(default=20, description="Maximum results to return", ge=1, le=100)
    offset: Optional[int] = Field(default=0, description="Number of results to skip for pagination", ge=0)
    response_format: ResponseFormat = Field(default=ResponseFormat.MARKDOWN, description="Output format")

    @field_validator('query')
    @classmethod
    def validate_query(cls, v: str) -> str:
        if not v.strip():
            raise ValueError("Query cannot be empty or whitespace only")
        return v.strip()

# Shared utility functions
async def _make_api_request(endpoint: str, method: str = "GET", **kwargs) -> dict:
    '''Reusable function for all API calls.'''
    async with httpx.AsyncClient() as client:
        response = await client.request(
            method,
            f"{API_BASE_URL}/{endpoint}",
            timeout=30.0,
            **kwargs
        )
        response.raise_for_status()
        return response.json()

def _handle_api_error(e: Exception) -> str:
    '''Consistent error formatting across all tools.'''
    if isinstance(e, httpx.HTTPStatusError):
        if e.response.status_code == 404:
            return "Error: Resource not found. Please check the ID is correct."
        elif e.response.status_code == 403:
            return "Error: Permission denied. You don't have access to this resource."
        elif e.response.status_code == 429:
            return "Error: Rate limit exceeded. Please wait before making more requests."
        return f"Error: API request failed with status {e.response.status_code}"
    elif isinstance(e, httpx.TimeoutException):
        return "Error: Request timed out. Please try again."
    return f"Error: Unexpected error occurred: {type(e).__name__}"

# Tool definitions
@mcp.tool(
    name="example_search_users",
    annotations={
        "title": "Search Example Users",
        "readOnlyHint": True,
        "destructiveHint": False,
        "idempotentHint": True,
        "openWorldHint": True
    }
)
async def example_search_users(params: UserSearchInput) -> str:
    '''Search for users in the Example system by name, email, or team.

    [Full docstring as shown above]
    '''
    try:
        # Make API request using validated parameters
        data = await _make_api_request(
            "users/search",
            params={
                "q": params.query,
                "limit": params.limit,
                "offset": params.offset
            }
        )

        users = data.get("users", [])
        total = data.get("total", 0)

        if not users:
            return f"No users found matching '{params.query}'"

        # Format response based on requested format
        if params.response_format == ResponseFormat.MARKDOWN:
            lines = [f"# User Search Results: '{params.query}'", ""]
            lines.append(f"Found {total} users (showing {len(users)})")
            lines.append("")

            for user in users:
                lines.append(f"## {user['name']} ({user['id']})")
                lines.append(f"- **Email**: {user['email']}")
                if user.get('team'):
                    lines.append(f"- **Team**: {user['team']}")
                lines.append("")

            return "\n".join(lines)

        else:
            # Machine-readable JSON format
            import json
            response = {
                "total": total,
                "count": len(users),
                "offset": params.offset,
                "users": users
            }
            return json.dumps(response, indent=2)

    except Exception as e:
        return _handle_api_error(e)

if __name__ == "__main__":
    mcp.run()

---

Advanced FastMCP Features

Context Parameter Injection

FastMCP can automatically inject a Context parameter into tools for advanced capabilities like logging, progress reporting, resource reading, and user interaction:

from mcp.server.fastmcp import FastMCP, Context

mcp = FastMCP("example_mcp")

@mcp.tool()
async def advanced_search(query: str, ctx: Context) -> str:
    '''Advanced tool with context access for logging and progress.'''

    # Report progress for long operations
    await ctx.report_progress(0.25, "Starting search...")

    # Log information for debugging
    await ctx.log_info("Processing query", {"query": query, "timestamp": datetime.now()})

    # Perform search
    results = await search_api(query)
    await ctx.report_progress(0.75, "Formatting results...")

    # Access server configuration
    server_name = ctx.fastmcp.name

    return format_results(results)

@mcp.tool()
async def interactive_tool(resource_id: str, ctx: Context) -> str:
    '''Tool that can request additional input from users.'''

    # Request sensitive information when needed
    api_key = await ctx.elicit(
        prompt="Please provide your API key:",
        input_type="password"
    )

    # Use the provided key
    return await api_call(resource_id, api_key)

Context capabilities:

• ctx.report_progress(progress, message) - Report progress for long operations
• ctx.log_info(message, data) / ctx.log_error() / ctx.log_debug() - Logging
• ctx.elicit(prompt, input_type) - Request input from users
• ctx.fastmcp.name - Access server configuration
• ctx.read_resource(uri) - Read MCP resources

Resource Registration

Expose data as resources for efficient, template-based access:

@mcp.resource("file://documents/{name}")
async def get_document(name: str) -> str:
    '''Expose documents as MCP resources.

    Resources are useful for static or semi-static data that doesn't
    require complex parameters. They use URI templates for flexible access.
    '''
    document_path = f"./docs/{name}"
    with open(document_path, "r") as f:
        return f.read()

@mcp.resource("config://settings/{key}")
async def get_setting(key: str, ctx: Context) -> str:
    '''Expose configuration as resources with context.'''
    settings = await load_settings()
    return json.dumps(settings.get(key, {}))

When to use Resources vs Tools:

• Resources: For data access with simple parameters (URI templates)
• Tools: For complex operations with validation and business logic

Structured Output Types

FastMCP supports multiple return types beyond strings:

from typing import TypedDict
from dataclasses import dataclass
from pydantic import BaseModel

# TypedDict for structured returns
class UserData(TypedDict):
    id: str
    name: str
    email: str

@mcp.tool()
async def get_user_typed(user_id: str) -> UserData:
    '''Returns structured data - FastMCP handles serialization.'''
    return {"id": user_id, "name": "John Doe", "email": "john@example.com"}

# Pydantic models for complex validation
class DetailedUser(BaseModel):
    id: str
    name: str
    email: str
    created_at: datetime
    metadata: Dict[str, Any]

@mcp.tool()
async def get_user_detailed(user_id: str) -> DetailedUser:
    '''Returns Pydantic model - automatically generates schema.'''
    user = await fetch_user(user_id)
    return DetailedUser(**user)

Lifespan Management

Initialize resources that persist across requests:

from contextlib import asynccontextmanager

@asynccontextmanager
async def app_lifespan():
    '''Manage resources that live for the server's lifetime.'''
    # Initialize connections, load config, etc.
    db = await connect_to_database()
    config = load_configuration()

    # Make available to all tools
    yield {"db": db, "config": config}

    # Cleanup on shutdown
    await db.close()

mcp = FastMCP("example_mcp", lifespan=app_lifespan)

@mcp.tool()
async def query_data(query: str, ctx: Context) -> str:
    '''Access lifespan resources through context.'''
    db = ctx.request_context.lifespan_state["db"]
    results = await db.query(query)
    return format_results(results)

Multiple Transport Options

FastMCP supports different transport mechanisms:

# Default: Stdio transport (for CLI tools)
if __name__ == "__main__":
    mcp.run()

# HTTP transport (for web services)
if __name__ == "__main__":
    mcp.run(transport="streamable_http", port=8000)

# SSE transport (for real-time updates)
if __name__ == "__main__":
    mcp.run(transport="sse", port=8000)

Transport selection:

• Stdio: Command-line tools, subprocess integration
• HTTP: Web services, remote access, multiple clients
• SSE: Real-time updates, push notifications

---

Code Best Practices

Code Composability and Reusability

Your implementation MUST prioritize composability and code reuse:

1. Extract Common Functionality:

• Create reusable helper functions for operations used across multiple tools
• Build shared API clients for HTTP requests instead of duplicating code
• Centralize error handling logic in utility functions
• Extract business logic into dedicated functions that can be composed
• Extract shared markdown or JSON field selection & formatting functionality

2. Avoid Duplication:

• NEVER copy-paste similar code between tools
• If you find yourself writing similar logic twice, extract it into a function
• Common operations like pagination, filtering, field selection, and formatting should be shared
• Authentication/authorization logic should be centralized

Python-Specific Best Practices

1. Use Type Hints: Always include type annotations for function parameters and return values 2. Pydantic Models: Define clear Pydantic models for all input validation 3. Avoid Manual Validation: Let Pydantic handle input validation with constraints 4. Proper Imports: Group imports (standard library, third-party, local) 5. Error Handling: Use specific exception types (httpx.HTTPStatusError, not generic Exception) 6. Async Context Managers: Use async with for resources that need cleanup 7. Constants: Define module-level constants in UPPER_CASE

Quality Checklist

Before finalizing your Python MCP server implementation, ensure:

Strategic Design

• [ ] Tools enable complete workflows, not just API endpoint wrappers
• [ ] Tool names reflect natural task subdivisions
• [ ] Response formats optimize for agent context efficiency
• [ ] Human-readable identifiers used where appropriate
• [ ] Error messages guide agents toward correct usage

Implementation Quality

• [ ] FOCUSED IMPLEMENTATION: Most important and valuable tools implemented
• [ ] All tools have descriptive names and documentation
• [ ] Return types are consistent across similar operations
• [ ] Error handling is implemented for all external calls
• [ ] Server name follows format: {service}_mcp
• [ ] All network operations use async/await
• [ ] Common functionality is extracted into reusable functions
• [ ] Error messages are clear, actionable, and educational
• [ ] Outputs are properly validated and formatted

Tool Configuration

• [ ] All tools implement 'name' and 'annotations' in the decorator
• [ ] Annotations correctly set (readOnlyHint, destructiveHint, idempotentHint, openWorldHint)
• [ ] All tools use Pydantic BaseModel for input validation with Field() definitions
• [ ] All Pydantic Fields have explicit types and descriptions with constraints
• [ ] All tools have comprehensive docstrings with explicit input/output types
• [ ] Docstrings include complete schema structure for dict/JSON returns
• [ ] Pydantic models handle input validation (no manual validation needed)

Advanced Features (where applicable)

• [ ] Context injection used for logging, progress, or elicitation
• [ ] Resources registered for appropriate data endpoints
• [ ] Lifespan management implemented for persistent connections
• [ ] Structured output types used (TypedDict, Pydantic models)
• [ ] Appropriate transport configured (stdio, HTTP, SSE)

Code Quality

• [ ] File includes proper imports including Pydantic imports
• [ ] Pagination is properly implemented where applicable
• [ ] Large responses check CHARACTER_LIMIT and truncate with clear messages
• [ ] Filtering options are provided for potentially large result sets
• [ ] All async functions are properly defined with async def
• [ ] HTTP client usage follows async patterns with proper context managers
• [ ] Type hints are used throughout the code
• [ ] Constants are defined at module level in UPPER_CASE

Testing

• [ ] Server runs successfully: python your_server.py --help
• [ ] All imports resolve correctly
• [ ] Sample tool calls work as expected
• [ ] Error scenarios handled gracefully

Using MCP Tools: Complete Guide

Comprehensive guide for discovering, executing, and managing MCP tools from configured servers.

---

Table of Contents

1. Configuration 2. Gemini CLI Integration (Primary) 3. Direct Script Execution (Secondary) 4. Subagent Delegation (Fallback) 5. Tool Discovery 6. Multi-Server Management 7. Troubleshooting

---

Configuration

MCP Server Configuration File

MCP servers configured in .claude/.mcp.json:

{
  "mcpServers": {
    "memory": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-memory"]
    },
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "/allowed/path"]
    },
    "brave-search": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-brave-search"],
      "env": {
        "BRAVE_API_KEY": "${BRAVE_API_KEY}"
      }
    },
    "puppeteer": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-puppeteer"]
    }
  }
}

Environment Variables

Reference env vars with ${VAR_NAME} syntax:

{
  "api-server": {
    "command": "node",
    "args": ["server.js"],
    "env": {
      "API_KEY": "${MY_API_KEY}",
      "BASE_URL": "${API_BASE_URL}"
    }
  }
}

Configuration Loading Order

Scripts check for config in this order: 1. process.env (runtime environment) 2. .claude/skills/mcp/.env 3. .claude/.env

---

Gemini CLI Integration (Primary)

Recommended primary method for MCP tool execution.

Installation

npm install -g gemini-cli
gemini --version

Configuration

Create symlink to share MCP config with Gemini CLI:

# Unix/Linux/macOS
mkdir -p .gemini
ln -sf .claude/.mcp.json .gemini/settings.json

# Windows (requires admin or developer mode)
mkdir .gemini
mklink .gemini\settings.json .claude\.mcp.json

Add to .gitignore:

.gemini/settings.json

Critical Usage Pattern

IMPORTANT: Use stdin piping, NOT -p flag.

# ❌ WRONG - Skips MCP initialization!
gemini -y -m gemini-2.5-flash -p "Take a screenshot"

# ✅ CORRECT - Initializes MCP servers
echo "Take a screenshot" | gemini -y -m gemini-2.5-flash

Why: The -p flag runs in "quick mode" and bypasses MCP server connection initialization. Always use stdin piping (echo + pipe).

Essential Flags

• -y: Skip confirmation prompts (auto-approve tool execution)
• -m <model>: Model selection
• gemini-2.5-flash (fast, recommended for MCP)
• gemini-2.5-flash (balanced)
• gemini-pro (high quality)

Usage Examples

Screenshot Capture:

echo "Take a screenshot of https://www.google.com" | gemini -y -m gemini-2.5-flash

Memory Operations:

echo "Remember that Alice is a React developer working on e-commerce" | gemini -y -m gemini-2.5-flash

Web Research:

echo "Search for latest Next.js 15 features and summarize top 3" | gemini -y -m gemini-2.5-flash

Multi-Tool Orchestration:

echo "Search for Claude AI docs, screenshot homepage, save to memory" | gemini -y -m gemini-2.5-flash

Browser Automation:

echo "Navigate to https://example.com, click signup, take screenshot" | gemini -y -m gemini-2.5-flash

How It Works

1. Configuration Loading: Reads .gemini/settings.json (symlinked to .claude/.mcp.json) 2. Server Connection: Connects to all configured MCP servers 3. Tool Discovery: Lists all available tools from servers 4. Prompt Analysis: Gemini model analyzes the prompt 5. Tool Selection: Automatically selects relevant tools 6. Execution: Calls tools with appropriate parameters 7. Result Synthesis: Combines tool outputs into coherent response

Structured JSON Responses

Create GEMINI.md in project root to enforce JSON-only responses:

# Gemini CLI Response Format

You are executing MCP tools. Always respond in this exact JSON format:

{
  "server": "server_name",
  "tool": "tool_name",
  "success": true,
  "result": <data>,
  "error": null
}

Rules:
- Maximum 500 characters
- JSON only, no markdown formatting
- No explanations outside JSON
- If error: set success=false, populate error field

Benefits:

• Programmatically parseable output
• Consistent error reporting
• Auto-loaded by Gemini CLI
• No natural language ambiguity

Advanced Configuration

Trusted Servers (skip confirmations):

{
  "mcpServers": {
    "memory": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-memory"],
      "trust": true
    }
  }
}

Tool Filtering:

{
  "chrome-devtools": {
    "command": "npx",
    "args": ["-y", "chrome-devtools-mcp@latest"],
    "includeTools": ["navigate_page", "screenshot"],
    "excludeTools": ["evaluate_js"]
  }
}

Debugging

Check MCP Status:

gemini
> /mcp

Shows connected servers, available tools, configuration errors.

Verify Symlink:

# Unix/Linux/macOS
ls -la .gemini/settings.json

# Windows
dir .gemini\settings.json

Debug Mode:

echo "Take a screenshot" | gemini --debug

---

Direct Script Execution (Secondary)

Manual tool specification when you know exact server/tool needed.

Installation

cd .claude/skills/mcp/scripts
npm install

Available Commands

List Tools (saves to assets/tools.json):

npx tsx cli.ts list-tools

List Prompts:

npx tsx cli.ts list-prompts

List Resources:

npx tsx cli.ts list-resources

Call Tool:

npx tsx cli.ts call-tool <server> <tool> '<json-args>'

Examples

Memory Operations:

npx tsx cli.ts call-tool memory create_entities '{
  "entities": [
    {"name": "Alice", "type": "person", "observations": ["React developer"]}
  ]
}'

Filesystem Operations:

npx tsx cli.ts call-tool filesystem read_file '{"path": "/path/to/file.txt"}'

Puppeteer Screenshot:

npx tsx cli.ts call-tool puppeteer screenshot '{
  "url": "https://example.com",
  "name": "example-screenshot"
}'

When to Use

• Need specific server/tool control
• Gemini CLI unavailable
• Scripting/automation scenarios
• Debugging specific tool behavior

---

Subagent Delegation (Fallback)

Delegate to mcp-manager agent when Gemini unavailable or for complex workflows.

Pattern

Main Agent → mcp-manager Subagent → Tool Discovery/Execution → Report Back

Benefits

• Main context stays clean
• Only relevant tool definitions loaded when needed
• Handles multi-tool orchestration
• Provides structured feedback

When to Use

• Gemini CLI unavailable
• Complex multi-step workflows
• Need context-efficient tool discovery
• Building MCP client implementations

---

Tool Discovery

List All Tools

npx tsx cli.ts list-tools

Output saved to assets/tools.json with complete schemas:

{
  "memory": [
    {
      "name": "create_entities",
      "description": "Create new entities in the knowledge graph",
      "inputSchema": {
        "type": "object",
        "properties": {
          "entities": {
            "type": "array",
            "items": {"type": "object"}
          }
        }
      }
    }
  ]
}

Intelligent Tool Selection

LLM reads assets/tools.json directly for context-aware filtering:

• Better than keyword matching algorithms
• Understands synonyms and intent
• Considers task context
• Selects relevant tools across servers

Progressive Disclosure

Only load tool definitions when needed:

1. Check assets/tools.json for overview 2. Filter relevant tools by task 3. Load only necessary tool details 4. Execute with proper parameters

---

Multi-Server Management

Configuration

Multiple servers in .claude/.mcp.json:

{
  "mcpServers": {
    "memory": {...},
    "filesystem": {...},
    "brave-search": {...},
    "puppeteer": {...}
  }
}

Orchestration Patterns

Sequential:

echo "Search web, then save results to memory, then take screenshot" | gemini -y

Parallel (Gemini decides):

echo "Search docs AND capture screenshot simultaneously" | gemini -y

Conditional:

echo "If search finds results, save to memory, else notify user" | gemini -y

Server Identification

Each tool knows its source server:

{
  "server": "puppeteer",
  "tool": "screenshot",
  "result": "screenshot.png"
}

Enables proper routing across multiple servers.

---

Troubleshooting

Connection Errors

Stdio Transport:

• Verify command and arguments correct in .claude/.mcp.json
• Check executable is in PATH
• Ensure required dependencies installed

HTTP/SSE Transport:

• Check URL is accessible
• Verify headers/authentication
• Ensure server is running

Tool Not Found

Check Configuration:

gemini
> /mcp

Verify Tool List:

npx tsx cli.ts list-tools

Check Symlink:

ls -la .gemini/settings.json

Execution Failures

Use Debug Mode:

echo "Your task" | gemini --debug

Check Environment Variables:

echo $API_KEY
printenv | grep -i api

Verify Direct Script:

npx tsx cli.ts call-tool server tool '{}'

Performance Issues

Use Faster Model:

echo "Task" | gemini -y -m gemini-2.5-flash

Filter Tools:

{
  "server": {
    "includeTools": ["tool1", "tool2"]
  }
}

Check Response Size:

• Verify pagination working
• Check CHARACTER_LIMIT respected
• Review tool output verbosity

---

Execution Priority

Follow this order for maximum efficiency:

1. Gemini CLI (Primary)

When: All tasks when available

Benefits:

• Automatic tool discovery
• Intelligent selection
• Fastest execution
• Natural language interface

Command:

echo "<task>" | gemini -y -m gemini-2.5-flash

2. Direct Scripts (Secondary)

When: Need specific server/tool control

Benefits:

• Explicit tool specification
• No LLM overhead
• Scripting/automation friendly

Command:

npx tsx cli.ts call-tool server tool '{"args": "value"}'

3. Subagent (Fallback)

When: Gemini unavailable or complex workflows

Benefits:

• Context-efficient
• Handles discovery + execution
• Structured feedback

Pattern: Delegate to mcp-manager agent

---

Scripts Reference

mcp-client.ts

Core MCP client manager:

• Config loading from .claude/.mcp.json
• Multi-server connection management
• Tool/prompt/resource listing
• Tool execution with error handling
• Connection lifecycle management

cli.ts

Command-line interface:

# List all tools (saves to assets/tools.json)
npx tsx cli.ts list-tools

# List all prompts
npx tsx cli.ts list-prompts

# List all resources
npx tsx cli.ts list-resources

# Execute tool
npx tsx cli.ts call-tool <server> <tool> '<json>'

---

Integration Strategy

With Agent Workflows

Pattern 1: Direct Gemini

Task → Gemini CLI → Tool Execution → Result

Pattern 2: Script Automation

Script → Direct CLI → Multiple Tools → Aggregated Results

Pattern 3: Subagent Delegation

Main Agent → mcp-manager → Tool Discovery → Execution → Report

With Development Workflow

1. Build MCP server (see references/building-servers.md) 2. Configure in .claude/.mcp.json 3. Test with Gemini CLI 4. Iterate based on results 5. Evaluate with comprehensive questions

---

Comparison Matrix

Recommendation: Gemini CLI primary, direct scripts for specific needs, subagent for complex workflows.

---

Resources

Documentation

• Gemini CLI Docs
• MCP Server Config
• Tool Reference

Related References

• Configuration: references/protocol-basics.md
• Building servers: references/building-servers.md
• Best practices: references/best-practices.md