Mongodb Search And Ai

MongoDB Search and AI Recommendations Skill

You are helping MongoDB users implement, optimize, and troubleshoot Atlas Search (lexical), Vector Search (semantic), and Hybrid Search (combined) solutions. Your goal is to understand their use case, recommend the appropriate search approach, and help them build effective indexes and queries.

Core Principles

1. Understand before building - Validate the use case to ensure you recommend the right solution 2. Always inspect first - Check existing indexes and schema before making recommendations 3. Explain before executing - Describe what indexes will be created and require explicit approval 4. Optimize for the use case - Different use cases require different index configurations and query patterns 5. Handle read-only scenarios - If you do not have access to create, update, or delete operation tools, you are in read-only mode. Provide the complete index configuration JSON so the user can create it themselves, including via the Atlas UI.

Workflow

1. Discovery Phase

Check the environment:

• Use list-databases and list-collections to understand available data
• If the user mentions a collection, use collection-schema to inspect field structure
• Use collection-indexes to see existing indexes
• Use atlas-inspect-cluster to determine the cluster's MongoDB version

Understand the use case: If the user's request is vague:

• Ask clarifying questions about their needs
• Infer likely collection and fields from schema
• Confirm understanding before proceeding

Common questions to ask:

• What are users searching for? (products, movies, documents, etc.)
• What fields contain the searchable content?
• Do they need exact matching, fuzzy matching, or semantic similarity?
• Do they need filters (price ranges, categories, dates)?
• Do they need autocomplete/typeahead functionality?

2. Determine Search Type

Atlas Search (Lexical/Full-Text): Use when users need:

• Keyword matching with relevance scoring
• Fuzzy matching for typo tolerance
• Autocomplete/typeahead
• Faceted search with filters
• Language-specific text analysis
• Token-based search
• Lexical search with views

Vector Search (Semantic): Use when users need:

• Semantic similarity ("find movies about coming of age stories")
• Natural language understanding
• RAG (Retrieval Augmented Generation) applications
• Finding conceptually similar items
• Cross-modal search
• Vector search with views

Hybrid Search: Use when users need:

• Combining multiple search approaches (e.g., vector + lexical, multiple text searches)
• Queries like "find action movies similar to 'epic space battles'" (combining keyword filtering with semantic similarity)
• Results that factor in multiple relevance criteria
• Uses $rankFusion (rank-based) or $scoreFusion (score-based) to merge pipelines

3. Version Check (Hybrid Search only)

If the search type is Hybrid using `$rankFusion` or `$scoreFusion`, verify the cluster version before proceeding:

• $rankFusion requires MongoDB 8.0+
• $scoreFusion requires MongoDB 8.2+

If the version requirement is not met, do not proceed — inform the user the feature is unavailable and suggest upgrading. Do not consult references/hybrid-search.md.

If the search type is Lexical, Vector, or the lexical prefilter pattern (vectorSearch operator inside $search), proceed to the next step.

4. Consult Reference Files

Always consult the appropriate reference file(s) before recommending indexes or queries:

• Lexical: consult both references/lexical-search-indexing.md (index) and references/lexical-search-querying.md (query)
• Vector: consult references/vector-search.md
• Hybrid: consult references/hybrid-search.md (and the lexical/vector files for the individual pipeline stages within it)

5. Execution and Validation

Creating indexes: 1. Explain the index configuration in plain language 2. Show the JSON structure 3. Ask what the user wants to name the index 4. Get explicit approval: "Should I create this index?" 5. Use MCP's create-index tool after approval 6. In read-only mode, provide the complete index JSON for creation via the Atlas UI

Running queries: 1. Show the aggregation pipeline 2. Execute using MCP's aggregate tool 3. Present results clearly

Refining existing queries: 1. Ask the user to share their current query 2. Compare against the query patterns and best practices in the relevant reference file(s) 3. Propose specific improvements with before/after examples 4. Run the revised query with aggregate to validate the results

Anti-Patterns to Avoid

NEVER recommend $regex or $text for search use cases:

• $regex: Not designed for full-text search. Lacks relevance scoring, fuzzy matching, and language-aware tokenization.
• $text: Legacy operator that doesn't scale well for search workloads.

If a user asks for regex/text for a search use case, explain why Atlas Search is more appropriate and show the equivalent pattern.

Handling Edge Cases

User mentions fields you can't find:

• Use collection-schema to inspect available fields
• Suggest alternatives or ask for clarification

Required field doesn't exist:

• Explain what needs to be added and how (e.g., embedding field for vector search)

Query fails or index missing:

• Use collection-indexes to verify index exists
• If missing, explain index needs to be created first

Multiple collections are relevant:

• List options and ask which one they mean
• If context makes it obvious, confirm your assumption

Remember

• Always check existing indexes before recommending new ones
• Explain technical concepts in accessible language
• Require approval before creating indexes
• Map user's business requirements to technical implementations
• Use the appropriate search type for the use case

Hybrid Search

This guide covers hybrid search patterns in MongoDB Atlas: combining vector and lexical search using $rankFusion and $scoreFusion, and using lexical prefilters with the vectorSearch operator inside $search.

Scope: This guide covers hybrid pipelines. For pure vector search indexes and $vectorSearch query construction, see vector-search.md. For lexical index definitions and query patterns, see lexical-search-indexing.md and lexical-search-querying.md.

Table of Contents

• Overview
• Choosing the Right Approach
• Indexing for Hybrid Search
• $rankFusion
• $scoreFusion
• Lexical Prefilters (vectorSearch Operator)
• Best Practices and Limitations

---

Overview

Hybrid search combines multiple search methods on the same collection and merges the results into a single ranked or scored list.

Three patterns covered in this guide:

$rankFusion vs $scoreFusion:

• $rankFusion ranks by position in each input pipeline using the Reciprocal Rank Fusion (RRF) algorithm. A document ranked #1 in multiple pipelines scores much higher than one ranked #1 in only one. Weights influence how much each pipeline's rank contributes.
• $scoreFusion ranks by the actual score values from each pipeline. Supports normalization (sigmoid, minMaxScaler) and custom combination expressions. Use when score magnitude, not just ordering, matters.

---

Choosing the Right Approach

Version requirements: $rankFusion requires MongoDB 8.0+. $scoreFusion requires MongoDB 8.2+. Only proceed with this guide if the use case is lexical prefilters, or if the cluster meets the version requirement for the fusion stage of interest. Otherwise do not proceed.

---

Indexing for Hybrid Search

For $rankFusion and $scoreFusion

You need two separate indexes on the collection:

1. A vectorSearch-type index for the $vectorSearch input pipeline:

db.collection.createSearchIndex(
  "<vector-index-name>",
  "vectorSearch",
  {
    "fields": [
      {
        "type": "vector",
        "path": "<embedding-field>",
        "numDimensions": <number>,
        "similarity": "dotProduct"
      }
    ]
  }
)

2. A search-type index for the $search input pipeline:

db.collection.createSearchIndex(
  "<search-index-name>",
  {
    "mappings": { "dynamic": true }
  }
)

---

For Lexical Prefilters (vectorSearch Operator)

The vectorSearch operator runs inside $search, so you need a single search-type index that includes a vector field type. This is different from a vectorSearch-type index — you cannot use the $vectorSearch stage to query fields indexed this way.

db.collection.createSearchIndex(
  "<search-index-name>",
  {
    "mappings": {
      "dynamic": true,
      "fields": {
        "<embedding-field>": {
          "type": "vector",
          "numDimensions": <number>,
          "similarity": "dotProduct",
          "quantization": "scalar"  // Optional
        }
      }
    }
  }
)

Note: storedSource: true is not supported on indexes that contain a vector field type. Use include or exclude to specify stored fields explicitly.

---

Common Rules for Fusion Stages

The following rules apply to both $rankFusion and $scoreFusion.

Pipeline naming restrictions: Pipeline names must not be empty, start with $, contain the null character \0, or contain .

Not allowed inside input pipelines: $project or storedSource fields. Apply modifications ($project, $addFields, $set) in stages after the fusion stage.

---

$rankFusion

$rankFusion executes all input pipelines independently, de-duplicates results, and ranks them using the Reciprocal Rank Fusion (RRF) algorithm. Documents appearing highly ranked in multiple pipelines score highest.

Syntax

{
  $rankFusion: {
    input: {
      pipelines: {
        <pipelineName1>: [ <stages> ],
        <pipelineName2>: [ <stages> ],
        ...
      }
    },
    combination: {
      weights: {
        <pipelineName1>: <number>,
        <pipelineName2>: <number>
      }
    },
    scoreDetails: <boolean>  // Default: false
  }
}

Fields

RRF Formula

For each document, the RRF score is:

RRFscore(d) = sum over all pipelines of: weight * (1 / (60 + rank_of_d_in_pipeline))

The constant 60 is a sensitivity parameter set by MongoDB and cannot be changed. Documents not present in a pipeline do not contribute a term for that pipeline.

Input Pipeline Restrictions

See Common Rules for naming and modification restrictions. Allowed stages: $search, $vectorSearch, $match, $geoNear, $sample, $sort, $skip, $limit.

The ordering requirement is satisfied if the pipeline begins with $search, $vectorSearch, or $geoNear, or contains an explicit $sort.

---

Example 1: Basic Hybrid (Vector + Lexical, Equal Weights)

db.embedded_movies.aggregate([
  {
    $rankFusion: {
      input: {
        pipelines: {
          vectorPipeline: [
            {
              $vectorSearch: {
                index: "<vector-index-name>",
                path: "plot_embedding",
                queryVector: [<query-vector-2048-dimensions>],
                numCandidates: 100,
                limit: 20
              }
            }
          ],
          textPipeline: [
            {
              $search: {
                index: "<search-index-name>",
                text: {
                  query: "<query-term>",
                  path: "title"
                }
              }
            },
            { $limit: 20 }
          ]
        }
      }
    }
  },
  { $limit: 10 }
])

Note: $search does not auto-limit results — always add $limit inside the $search input pipeline.

---

Example 2: Weighted Hybrid (Boosting One Pipeline)

Assign higher weight to the pipeline whose ranking should contribute more to the final score:

db.embedded_movies.aggregate([
  {
    $rankFusion: {
      input: {
        pipelines: {
          vectorPipeline: [
            {
              $vectorSearch: {
                index: "<vector-index-name>",
                path: "plot_embedding",
                queryVector: [<query-vector-2048-dimensions>],
                numCandidates: 100,
                limit: 20
              }
            }
          ],
          textPipeline: [
            {
              $search: {
                index: "<search-index-name>",
                phrase: {
                  query: "<query-term>",
                  path: "title"
                }
              }
            },
            { $limit: 20 }
          ]
        }
      },
      combination: {
        weights: {
          vectorPipeline: 0.7,
          textPipeline: 0.3
        }
      }
    }
  },
  { $limit: 10 }
])

Recommendation: Set weights per-query based on which method is more appropriate for that query, rather than using static weights for all queries.

---

Example 3: Multiple $vectorSearch Pipelines

Use multiple vector pipelines to search different fields, different query vectors, or different embedding models:

db.embedded_movies.aggregate([
  {
    $rankFusion: {
      input: {
        pipelines: {
          plotPipeline: [
            {
              $vectorSearch: {
                index: "<vector-index-name>",
                path: "plot_embedding_voyage",
                queryVector: [<query-vector-2048-dimensions>],
                numCandidates: 200,
                limit: 50
              }
            }
          ],
          titlePipeline: [
            {
              $vectorSearch: {
                index: "<vector-index-name>",
                path: "title_embedding_voyage",
                queryVector: [<query-vector-2048-dimensions>],
                numCandidates: 200,
                limit: 50
              }
            }
          ]
        }
      },
      combination: {
        weights: {
          plotPipeline: 0.5,
          titlePipeline: 0.5
        }
      }
    }
  },
  { $limit: 20 }
])

---

Surfacing scoreDetails

Set scoreDetails: true on the stage, then project via $meta: "scoreDetails". The output includes a value (final RRF score), description, and a details array — one entry per input pipeline — containing inputPipelineName, rank, weight, and optionally value (raw pipeline score). See the $scoreFusion scoreDetails section below for a concrete structure example; $rankFusion follows the same pattern with rank instead of inputPipelineRawScore.

---

$scoreFusion

$scoreFusion executes all input pipelines independently, de-duplicates results, and combines them using the actual score values from each pipeline. Supports normalization and custom combination expressions for fine-grained control over how scores are merged.

Syntax

{
  $scoreFusion: {
    input: {
      pipelines: {
        <pipelineName1>: [ <stages> ],
        <pipelineName2>: [ <stages> ],
        ...
      },
      normalization: "none | sigmoid | minMaxScaler"
    },
    combination: {
      weights: {
        <pipelineName1>: <number>,
        <pipelineName2>: <number>
      },
      method: "avg | expression",
      expression: <arithmetic-expression>
    },
    scoreDetails: <boolean>
  }
}

Fields

Normalization Options

Input Pipeline Restrictions

See Common Rules for naming and modification restrictions. Allowed stages: $search, $vectorSearch, $match, $geoNear, $sort, $skip, $limit. Note: unlike $rankFusion, $sample is not permitted.

The scoring requirement is satisfied if the pipeline begins with $search, $vectorSearch, $match with legacy text search, or $geoNear. Otherwise, include an explicit $score stage.

---

Example 1: avg Method with Weights

db.embedded_movies.aggregate([
  {
    $scoreFusion: {
      input: {
        pipelines: {
          vectorPipeline: [
            {
              $vectorSearch: {
                index: "<vector-index-name>",
                path: "plot_embedding",
                queryVector: [<query-vector-2048-dimensions>],
                numCandidates: 100,
                limit: 20
              }
            }
          ],
          textPipeline: [
            {
              $search: {
                index: "<search-index-name>",
                text: {
                  query: "<query-term>",
                  path: "title"
                }
              }
            },
            { $limit: 20 }
          ]
        },
        normalization: "sigmoid"
      },
      combination: {
        method: "avg",
        weights: {
          vectorPipeline: 2,
          textPipeline: 1
        }
      }
    }
  },
  { $limit: 10 }
])

---

Example 2: expression Method with Custom Score Math

Use expression when you need full control over how pipeline scores are combined. Reference pipeline names as variables in the expression:

db.embedded_movies.aggregate([
  {
    $scoreFusion: {
      input: {
        pipelines: {
          searchOne: [
            {
              $vectorSearch: {
                index: "<vector-index-name>",
                path: "plot_embedding",
                queryVector: [<query-vector-2048-dimensions>],
                numCandidates: 100,
                limit: 20
              }
            }
          ],
          searchTwo: [
            {
              $search: {
                index: "<search-index-name>",
                text: {
                  query: "<query-term>",
                  path: "title"
                }
              }
            },
            { $limit: 20 }
          ]
        },
        normalization: "sigmoid"
      },
      combination: {
        method: "expression",
        expression: {
          $sum: [
            { $multiply: ["$searchOne", 10] },
            "$searchTwo"
          ]
        }
      },
      scoreDetails: true
    }
  },
  {
    $project: {
      _id: 1,
      title: 1,
      plot: 1,
      scoreDetails: { $meta: "scoreDetails" }
    }
  },
  { $limit: 10 }
])

Note: combination.expression and combination.weights are mutually exclusive. When using expression, embed weights directly via $multiply as shown above.

---

Surfacing scoreDetails

Set scoreDetails: true, then use $meta: "scoreDetails" in $project, $addFields, or $set:

{
  $project: {
    title: 1,
    scoreDetails: { $meta: "scoreDetails" }
  }
}

scoreDetails structure:

{
  value: 7.847,
  description: "the value calculated by combining the scores...",
  normalization: "sigmoid",
  combination: {
    method: "custom expression",
    expression: "{ $sum: [{ $multiply: ['$searchOne', 10] }, '$searchTwo'] }"
  },
  details: [
    {
      inputPipelineName: "searchOne",
      inputPipelineRawScore: 0.798,
      weight: 1,
      value: 0.689,
      details: []
    },
    {
      inputPipelineName: "searchTwo",
      inputPipelineRawScore: 2.962,
      weight: 1,
      value: 0.950,
      details: []
    }
  ]
}

---

Lexical Prefilters (vectorSearch Operator)

The vectorSearch operator runs inside a $search stage and performs ANN or ENN vector search with the ability to pre-filter using any Atlas Search operator — including text with fuzzy matching, phrase, wildcard, queryString, and compound. This is more expressive than the MQL-only filter option in the $vectorSearch stage.

Requires: A search-type index (not vectorSearch-type) with the embedding field configured as vector type. See Indexing for Hybrid Search.

Cannot be used: Inside embeddedDocument, compound, or facet operators.

Syntax

{
  $search: {
    index: "<search-index-name>",
    vectorSearch: {
      path: "<vector-field>",
      queryVector: [<array-of-numbers>],
      limit: <number>,
      numCandidates: <number>,       // Required for ANN (exact: false)
      exact: true | false,           // Optional, default false
      filter: { <search-operator> }, // Optional
      score: { <score-options> }     // Optional
    },
    concurrent: true  // Optional, dedicated search nodes only
  }
}

Key Fields

---

Example 1: compound Prefilter (queryString + range)

Filter by text match OR date range before running vector search:

db.embedded_movies.aggregate([
  {
    $search: {
      index: "<search-index-name>",
      vectorSearch: {
        path: "plot_embedding",
        queryVector: [<query-vector-2048-dimensions>],
        limit: 10,
        exact: true,
        filter: {
          compound: {
            should: [
              {
                queryString: {
                  defaultPath: "fullplot",
                  query: "plot:courtroom OR lawyer"
                }
              },
              {
                range: {
                  path: "year",
                  gte: 2000,
                  lte: 2015
                }
              }
            ]
          }
        }
      },
      concurrent: true
    }
  },
  {
    $project: {
      _id: 0,
      title: 1,
      plot: 1,
      score: { $meta: "searchScore" }
    }
  }
])

---

Example 2: text Prefilter with Fuzzy Matching

Filter by fuzzy text match before running ANN vector search:

db.embedded_movies.aggregate([
  {
    $search: {
      index: "<search-index-name>",
      vectorSearch: {
        path: "plot_embedding",
        queryVector: [<query-vector-2048-dimensions>],
        limit: 10,
        numCandidates: 200,
        filter: {
          text: {
            path: "fullplot",
            query: "charming animal",
            fuzzy: {}
          }
        }
      },
      concurrent: true
    }
  },
  {
    $project: {
      _id: 0,
      title: 1,
      plot: 1,
      score: { $meta: "searchScore" }
    }
  }
])

---

Best Practices and Limitations

Best Practices

Set limits inside $search sub-pipelines: $search does not limit results by default. Always add $limit inside the input pipeline, or $rankFusion/$scoreFusion evaluates all search results.

textPipeline: [
  { $search: { ... } },
  { $limit: 20 }  // Required
]

Set weights per-query: Tune weights based on which search method is most appropriate for a given query rather than using fixed weights for all queries. This improves relevance and resource utilization.

Handle disjoint results: If most results come from one pipeline and not the other, the two methods are returning largely different documents. Increase per-pipeline limits to improve overlap.

Use `$match` for non-search filtering: To filter on specific fields without a search pipeline (e.g., boost on a flag field), add a $match pipeline inside input.pipelines. It must contain an explicit $sort to qualify as a ranked pipeline.

Limitations

Single collection only: $rankFusion and $scoreFusion cannot span multiple collections. For cross-collection hybrid search, use $unionWith with $vectorSearch.

Pipelines run serially: Input pipelines do not execute in parallel.

No pagination inside sub-pipelines: $rankFusion and $scoreFusion do not support pagination within input pipelines.

vectorSearch operator restrictions: Cannot be used inside embeddedDocument, compound, or facet operators. Cannot use highlight, sort, or searchSequenceToken with the vectorSearch operator — use $skip and $limit after $search instead.

Lexical Search - Indexing

This guide covers how to configure MongoDB Atlas Search indexes. Use this reference to build index definitions with proper field types, analyzers, mappings, and optimization settings.

Table of Contents

• Atlas Search Index Definition
• Analyzer Selection
• Field Types
• Dynamic vs Explicit Mappings
• Stored Source
• Synonyms

---

Atlas Search Index Definition

Syntax

{
  "analyzer": "<analyzer-for-index>",
  "searchAnalyzer": "<analyzer-for-query>",
  "mappings": {
    "dynamic": <boolean> | {
      "typeSet": "<typeSet-name>"
    },
    "fields": {
      <field-definition>
    }
  },
  "numPartitions": <integer>,
  "analyzers": [ <custom-analyzer> ],
  "storedSource": <boolean> | {
    <stored-source-definition>
  },
  "synonyms": [
    {
      <synonym-mapping-definition>
    }
  ],
  "typeSets": [
    {
      "types": [
        {<field-types-definition>}
      ]
    }
  ]
}

Options

Basic Definition

Most indexes only need the mappings configuration:

{
  "mappings": {
    "dynamic": <boolean> | { <typeSet-definition> },
    "fields": { <field-definition> }
  }
}

---

Analyzer Selection

The analyzer determines how text is processed for indexing and searching.

Default behavior: Most queries don't specify an analyzer and use MongoDB's default standard analyzer, which:

• Divides text into terms based on word boundaries (language-neutral)
• Converts terms to lowercase and removes punctuation
• Recognizes email addresses, acronyms, CJK characters, alphanumerics, and more

Common built-in analyzers:

---

Index Analyzer (Applied at Index Time)

Specify per-field or top-level for all fields:

// Field-level (add to mappings.fields):
{
  "title": {
    "type": "string",
    "analyzer": "lucene.standard"  // Or omit to use default
  },
  "category": {
    "type": "token"   // Exact matching — use token, not string with lucene.keyword
  }
}

// Top-level (applies to all fields unless overridden):
{
  "analyzer": "lucene.standard",
  "mappings": {
    "fields": {
      "title": { "type": "string" }  // Uses top-level analyzer
    }
  }
}

---

Search Analyzer (Applied at Query Time)

Apply different analysis to queries than to indexed content:

// Top-level searchAnalyzer:
{
  "searchAnalyzer": "lucene.simple",  // Query-time analyzer
  "mappings": {
    "fields": {
      "description": {
        "type": "string",
        "analyzer": "lucene.standard"  // Index-time analyzer
      }
    }
  }
}

Use case: Index with standard analysis, but search with simpler/synonym-aware analyzer.

If omitted, uses the index analyzer. If both omitted, defaults to lucene.standard.

---

Multi Analyzer (Alternate Analyzers for Same Field)

Index the same field with multiple analyzers:

// Field configuration (add to mappings.fields):
{
  "title": {
    "type": "string",
    "analyzer": "lucene.standard",  // Default analyzer
    "multi": {
      "keywordAnalyzer": {
        "type": "string",
        "analyzer": "lucene.keyword"  // Alternate analyzer
      }
    }
  }
}

Use case: Support both fuzzy matching and exact matching on the same field.

To query using the alternate analyzer, specify the path as fieldName.alternateAnalyzerName (e.g., title.keywordAnalyzer).

---

Custom Analyzers

Define custom tokenization and filtering:

// Index definition with custom analyzer:
{
  "analyzers": [
    {
      "name": "customAnalyzer",
      "tokenizer": {
        "type": "standard"
      },
      "charFilters": [],
      "tokenFilters": [
        { "type": "lowercase" },
        { "type": "stop", "tokens": ["the", "a", "an"] }
      ]
    }
  ],
  "mappings": {
    "fields": {
      "content": {
        "type": "string",
        "analyzer": "customAnalyzer"  // Reference custom analyzer
      }
    }
  }
}

Use case: Need specific tokenization or filtering not provided by built-in analyzers.

---

Normalizers (Token Type Only)

Normalizers produce a single token (used with token field type):

// Field configuration (add to mappings.fields):
{
  "username": {
    "type": "token",
    "normalizer": "lowercase"  // Options: "lowercase", "none"
  }
}

Normalizers:

• lowercase: Transforms to lowercase, creates single token
• none: No transformation, creates single token

Use case: Exact matching with case normalization for token fields.

---

Decision Guide

• lucene.standard (or omit): Default for most text fields
• lucene.keyword: Categories, tags
• Language-specific: When you know the content language
• searchAnalyzer: Different analysis for queries vs indexed content (e.g., synonyms)
• multi: Support multiple search patterns on same field
• Custom: Need specific tokenization/filtering logic
• Normalizers: Token fields requiring case normalization

---

Field Types

Dynamic mapping includes: boolean, date, number, objectId, string, uuid Must configure explicitly: autocomplete, token, geo, embeddedDocuments, vector

Quick Reference

Field definition structure:

{
  "mappings": {
    "fields": {
      "<field-name>": {
        "type": "<field-type>",
        // type-specific options here
      }
    }
  }
}

Multiple types on same field:

"<field-name>": [
  { "type": "string" },
  { "type": "token", "normalizer": "lowercase" }
]

Arrays: MongoDB Search automatically flattens arrays during indexing. Specify only the element type, not that it's an array.

---

Dynamic vs Explicit Mappings

Choose based on user's needs:

Use dynamic (true) when:

• User is prototyping or exploring data with unknown schema
• Need to get started quickly without defining all fields
• All or most fields need to be searchable
• Accept larger index size and slower performance for convenience

Use explicit (dynamic: false) (recommended for production) when:

• User has completed early stages of prototyping and knows exactly which fields to search
• Performance and index size are priorities
• Schema is stable and well-defined
• Only a subset of fields need to be searchable

Use typeSets (recommended for production) when:

• User wants automatic indexing but with control over which types
• Document schema is dynamic and new fields need to be indexed automatically without an index rebuild
• Want different indexing strategies for different nested documents
• Balance between convenience and performance is important

---

Dynamic mappings automatically index all fields:

{
  "mappings": {
    "dynamic": true  // Index everything
  }
}

• Pros: Quick setup, works immediately
• Cons: Larger index, slower queries, wastes resources on unused fields

Explicit mappings define exactly what to index:

{
  "mappings": {
    "dynamic": false,  // Only index specified fields
    "fields": {
      "title": { "type": "string" },
      "genre": { "type": "token" }
    }
  }
}

• Pros: Smaller index, faster queries, precise control
• Cons: Requires knowing your schema

Configurable dynamic with typeSets (recommended middle ground):

{
  "mappings": {
    "dynamic": {
      "typeSet": "customTypes"
    },
    "fields": {
      "metadata": {
        "type": "document",
        "dynamic": {
          "typeSet": "metadataTypes"
        }
      }
    }
  },
  "typeSets": [
    {
      "name": "customTypes",
      "types": [
        { "type": "string" },
        { "type": "number" }
      ]
    },
    {
      "name": "metadataTypes",
      "types": [
        {
          "type": "string",
          "analyzer": "lucene.standard"
        }
      ]
    }
  ]
}

• Pros: Automatically indexes specified field types, more control than full dynamic, can configure different typeSets for sub-documents
• Cons: Still indexes all fields of specified types

Recommendation: Use static mappings or a dynamic typeSet (within a specific path, not at the root document level) in production for optimized index size and performance.

---

Stored Source

Store frequently accessed fields directly in the search index (mongot) to avoid full document lookups from the database. This dramatically improves query performance, especially when filtering or sorting.

Requirements:

• Available on clusters running MongoDB 7.0+
• Stored fields must still be indexed separately to query them
• Retrieve stored fields at query time using returnStoredSource: true (see lexical-search-querying.md)

Syntax:

{
  "storedSource": true | false | {
    "include" | "exclude": ["<field-name>", ...]
  }
}

Options:

true - Store all fields in documents. Not supported if index contains vector type field. Can significantly impact performance.

false - Don't store any fields (default behavior).

{ "include": [...] } - Store only specified fields. MongoDB Search also stores _id by default. List field names or dot-separated paths.

{ "exclude": [...] } - Store all fields except specified ones.

Examples:

Store specific fields:

{
  "mappings": {
    "dynamic": false,
    "fields": {
      "title": { "type": "string" },
      "genre": { "type": "token" },
      "year": { "type": "number" },
      "rating": { "type": "number" }
    }
  },
  "storedSource": {
    "include": ["title", "genre", "year", "rating"]
  }
}

Exclude specific fields:

{
  "storedSource": {
    "exclude": ["largeTextField", "unusedField"]
  }
}

Store all fields:

{
  "storedSource": true
}

When to use:

• Fields used for filtering, sorting, or projection after $search
• Frequently accessed fields in search results
• When avoiding database lookups is critical for performance

When NOT to use:

• Very large text fields (increases index size significantly)
• Fields rarely used in queries
• When index size is a concern

Note: For using stored source at query time, see lexical-search-querying.md. For vector field storage considerations, see vector-search.md.

---

Synonyms

Use when user wants query expansion with equivalent terms (e.g., "car" also finds "automobile", "vehicle").

Agent Workflow:

1. Ask user to create synonym collection in the same database as their indexed collection.

2. Provide synonym document format based on user's needs:

For bidirectional synonyms (all terms interchangeable):

db.synonyms.insertMany([
  {
    "mappingType": "equivalent",
    "synonyms": ["car", "vehicle", "automobile"]
  },
  {
    "mappingType": "equivalent",
    "synonyms": ["happy", "joyful", "glad"]
  }
])

For one-way synonyms (input maps to synonyms only):

db.synonyms.insertMany([
  {
    "mappingType": "explicit",
    "input": ["pants"],
    "synonyms": ["trousers", "slacks"]
  }
])

3. Add synonym mapping to index definition:

{
  "mappings": {
    "fields": {
      "<field-name>": {
        "type": "string",
        "analyzer": "lucene.standard"  // Note the analyzer
      }
    }
  },
  "synonyms": [
    {
      "name": "<synonym-mapping-name>",
      "analyzer": "lucene.standard",  // Must match field analyzer
      "source": {
        "collection": "<synonym-collection-name>"
      }
    }
  ]
}

Critical Rules:

• Synonym mapping analyzer MUST match the field analyzer being queried
• Only one synonym mapping allowed per index
• Changes to synonym collection auto-update (no reindex needed)
• Works only with text and phrase operators

Example: If user wants "car" to also find "vehicle" and "automobile": 1. Tell user to create collection: db.synonyms.insertOne({ "mappingType": "equivalent", "synonyms": ["car", "vehicle", "automobile"] }) 2. Add to index with analyzer matching the field being searched 3. Queries automatically expand (user searches "car", MongoDB Search searches: car OR vehicle OR automobile)

---

Searching on Views

Requires MongoDB 8.0+. Create Atlas Search indexes on Views to partially index a collection, transform documents, or support incompatible data types.

Note: Programmatic index creation via mongosh/driver methods requires 8.1+. On 8.0, also note that queries must run against the source collection referencing the view's index name. On 8.1+, you can query the view directly.

Supported view stages: $addFields, $set, $match with $expr only.

Key limitations:

• Index names must be unique across source collection and all its views
• No operators producing dynamic results (e.g., $USER_ROLES, $rand)
• Queries return original source documents. Use storedSource to retrieve transformed fields

Example: partial index (filter documents)

db.createView("movies_After2000", "movies", [
  { $match: { $expr: { $gt: ["$released", ISODate("2000-01-01")] } } }
])

db.movies_After2000.createSearchIndex(
  "after2000Index",
  { "mappings": { "dynamic": true } }
)

// 8.1+: query view directly; 8.0: query source collection using index name
db.movies_After2000.aggregate([
  { $search: { index: "after2000Index", text: { path: "title", query: "<query>" } } }
])

Editing a view: Use collMod. MongoDB Search auto-reindexes on view definition changes with no downtime.

Performance: Complex transformations slow performance. For heavy transformations consider a materialized view, or query the source collection directly.

Troubleshooting:

• Index goes FAILED: view is incompatible with Search, or source collection was removed/changed
• Index goes STALE: view's pipeline fails on a document. Index remains queryable while STALE; returns to READY after fixing the document or view definition
• `$search is only valid as first stage` error: you're on MongoDB 8.0 querying the view directly. Query the source collection instead, or upgrade to 8.1+

Lexical Search - Querying

This guide covers query patterns and optimization techniques for MongoDB Atlas Search.

Table of Contents

• $search vs $searchMeta
• Query Patterns
• Query Optimization
• Query Performance Analysis

---

$search vs $searchMeta

Both stages must be the first stage in an aggregation pipeline.

$searchMeta shares the following fields with $search: index, all operator names (e.g. text, range, compound), concurrent (parallelizes search across segments on dedicated search nodes only — ignored otherwise), and returnStoredSource.

---

Query Patterns

Operator Reference

---

Count Results

Use the count option in $searchMeta to count matching documents without fetching them. Also works in $search via the $SEARCH_META aggregation variable when you need both results and count.

// Count only (recommended)
db.movies.aggregate([
  {
    $searchMeta: {
      range: { path: "year", gte: 2010, lte: 2015 },
      count: { type: "lowerBound" }  // or "total" for exact count
    }
  }
])
// Returns: { count: { lowerBound: NumberLong(1001) } }

// Count alongside results using $SEARCH_META
db.movies.aggregate([
  {
    $search: {
      text: { path: "title", query: "<query>" },
      count: { type: "total" }
    }
  },
  { $project: { title: 1, meta: "$SEARCH_META" } },
  { $limit: 10 }
])

Note: Count affects performance — use only when needed (e.g., first page of paginated results).

---

Pagination with searchSequenceToken

Cursor-based pagination using tokens. More efficient than $skip alone for deep pagination.

Step 1 — Get tokens from the initial query:

db.movies.aggregate([
  {
    $search: {
      index: "<index-name>",
      text: { path: "title", query: "summer" },
      sort: { released: 1, _id: 1 }  // Sort on a unique field to prevent tie-ordering issues
    }
  },
  { $limit: 10 },
  {
    $project: {
      title: 1, released: 1,
      paginationToken: { $meta: "searchSequenceToken" }
    }
  }
])

Step 2 — Next page using searchAfter:

db.movies.aggregate([
  {
    $search: {
      index: "<index-name>",
      text: { path: "title", query: "summer" },
      searchAfter: "<token-from-last-document-on-previous-page>",
      sort: { released: 1, _id: 1 }  // maintain the same sort order
    }
  },
  { $limit: 10 },
  { $project: { title: 1, paginationToken: { $meta: "searchSequenceToken" } } }
])

Use searchBefore with the first document's token on the current page to go to the previous page — results are returned in reverse order. Combine searchAfter with $skip to jump pages.

Key constraint: Query semantics (operator, path, query value, sort) must be identical between the initial query and any searchAfter/searchBefore query.

---

Retrieve Arrays of Objects with returnScope

Return each element of an embedded document array as an individually scored document. Works in both $search and $searchMeta.

Requirements:

• Array field indexed as embeddedDocuments type with storedSource defined on the fields to return
• returnStoredSource: true in the query
• All operator paths must be nested under returnScope.path (use hasAncestor or hasRoot to query outside it)

Index:

{
  "mappings": {
    "dynamic": false,
    "fields": {
      "funding_rounds": {
        "type": "embeddedDocuments",
        "dynamic": true,
        "storedSource": {
          "include": ["round_code", "raised_currency_code", "raised_amount"]
        }
      }
    }
  }
}

Query:

db.companies.aggregate([
  {
    $search: {
      range: { path: "funding_rounds.raised_amount", gte: 5000000, lte: 10000000 },
      returnStoredSource: true,
      returnScope: { path: "funding_rounds" }
    }
  },
  { $limit: 5 }
])

Only fields defined in storedSource within the embedded document are returned — root-level fields are excluded. When returnScope is specified, all query paths must start with returnScope.path.

---

Advanced Query Syntax (queryString)

Use case: Complex search with boolean operators, wildcards, and field-specific queries.

Fields configuration:

// Add to mappings.fields in your index:
{
  "title": { "type": "string" },
  "director": { "type": "string" },
  "year": { "type": "number" }
}

Query patterns:

// Boolean operators
db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      queryString: {
        defaultPath: "title",
        query: "detective AND (noir OR thriller) NOT comedy"
      }
    }
  }
])

// Field-specific searches
db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      queryString: {
        defaultPath: "title",
        query: "title:inception AND director:nolan"
      }
    }
  }
])

// Wildcards and ranges
db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      queryString: {
        defaultPath: "title",
        query: "star* AND year:[2010 TO 2020]"
      }
    }
  }
])

Supported syntax:

• Boolean: AND, OR, NOT
• Grouping: (term1 OR term2)
• Wildcards: * (0+ chars), ? (single char)
• Ranges: [min TO max] for numbers/dates
• Field-specific: fieldName:value

Key considerations:

• Great for building search UIs with advanced options
• Users can construct complex queries without API changes
• Validate/sanitize user input to prevent injection

---

Searching Nested Arrays (embeddedDocument)

Use case: Search within arrays of objects where element-wise comparisons are required (similar to $elemMatch), or each element must be scored independently.

Fields configuration:

// Add to mappings.fields in your index:
{
  "title": { "type": "string" },
  "reviews": {
    "type": "embeddedDocuments",  // Required for array search
    "fields": {
      "author": { "type": "string" },
      "text": { "type": "string" },
      "rating": { "type": "number" }
    }
  }
}

Query pattern:

db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      embeddedDocument: {
        path: "reviews",
        operator: {
          compound: {
            must: [
              { text: { query: "excellent", path: "reviews.text" } }
            ],
            filter: [
              { range: { path: "reviews.rating", gte: 4 } }
            ]
          }
        },
        score: { embedded: { aggregate: "maximum" } }  // or sum, minimum, mean
      }
    }
  }
])

Score aggregation options:

• sum: Add scores from all matching array elements
• maximum: Use highest score from array elements
• minimum: Use lowest score from array elements
• mean: Average scores from array elements

Key considerations:

• Each array element is indexed as a separate document
• Use embeddedDocuments field type, not regular document
• Score aggregation controls how array matches affect overall document score
• Performance can be degraded due to complexity of parent-child joins

---

Search Highlighting

Use case: Show users which parts of documents matched their query.

Fields configuration:

// Add to mappings.fields in your index:
{
  "title": { "type": "string" },
  "plot": { "type": "string" }
}

Query pattern:

db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      text: {
        query: "detective noir",
        path: "plot"
      },
      highlight: {
        path: "plot",
        maxCharsToExamine: 500000,  // Default
        maxNumPassages: 5            // Number of snippets
      }
    }
  },
  {
    $project: {
      title: 1,
      plot: 1,
      highlights: { $meta: "searchHighlights" },
      score: { $meta: "searchScore" }
    }
  }
])

Highlight result structure:

{
  "highlights": [
    {
      "path": "plot",
      "texts": [
        { "value": "A ", "type": "text" },
        { "value": "detective", "type": "hit" },
        { "value": " investigates a murder in ", "type": "text" },
        { "value": "noir", "type": "hit" },
        { "value": " Los Angeles", "type": "text" }
      ],
      "score": 1.23
    }
  ]
}

Key considerations:

• type: "hit" indicates matched terms
• type: "text" is surrounding context
• Multiple passages returned for long documents
• Use in search results UI to show match context

---

Compound Queries

Compound queries combine multiple operators efficiently:

db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      compound: {
        must: [
          { text: { query: "detective", path: "plot" } }  // Required, affects score
        ],
        should: [
          { text: { query: "mystery", path: "genre" } }   // Optional, boosts score
        ],
        filter: [
          { range: { path: "year", gte: 2000 } }          // Required, no score impact
        ],
        mustNot: [
          { text: { query: "comedy", path: "genre" } }    // Excludes results
        ]
      }
    }
  }
])

Clause types:

• must: Required matches that affect scoring
• should: Optional matches that boost scores
• filter: Required matches that don't affect scoring (faster)
• mustNot: Exclusions

Performance tips:

• Use filter instead of must for criteria that shouldn't affect scoring (faster)
• Put most selective criteria in must or filter first
• Limit should clauses to 3-5 for best performance

---

Query with Synonyms

When your index is configured with synonyms, specify the synonym mapping name in your query:

db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      text: {
        query: "car chase",
        path: "description",
        synonyms: "synonym-mapping-name"  // Reference the mapping from your index
      }
    }
  }
])

Note: When you specify a synonym mapping name, MongoDB Search automatically searches for the query terms AND all their synonyms (e.g., "car" also matches "automobile", "vehicle").

---

Using Multi Analyzers

Query specific analyzer variants of a field:

// Standard fuzzy search
db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      text: {
        query: "Action",
        path: "title"  // Uses default analyzer
      }
    }
  }
])

// Exact match using keyword analyzer
db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      text: {
        query: "Action",
        path: "title.keywordAnalyzer"  // Uses alternate analyzer
      }
    }
  }
])

Use case: Support both fuzzy and exact matching on the same field without duplicating data.

---

Autocomplete

Search-as-you-type on fields indexed as autocomplete type (see lexical-search-indexing.md).

To score exact matches higher, index the field as both autocomplete and string types and query using compound.

---

Facet

Groups results into buckets by field values or ranges. Use with $searchMeta for metadata only, or with $search + $SEARCH_META variable for results and metadata.

{ "$searchMeta": { "facet": {
    "operator": { <operator> },
    "facets": {
      "<facet-name>": { "type": "string|number|date", "path": "<field>", ...options }
    }
} } }

---

geoShape

Query shapes by spatial relation. Field must be indexed as geo type with indexShapes: true. Required fields: geometry (GeoJSON Polygon, MultiPolygon, or LineString), path, and relation:

---

geoWithin

Query geographic points within a region. Field must be indexed as geo type. Specify one of:

• box: { bottomLeft: <GeoJSON Point>, topRight: <GeoJSON Point> }
• circle: { center: <GeoJSON Point>, radius: <meters> }
• geometry: GeoJSON Polygon or MultiPolygon

For both geo operators: longitude must be specified before latitude; longitude range [-180, 180], latitude range [-90, 90].

---

Query Optimization

Sorting Search Results

Use the sort option inside $search to sort at the mongot level (more efficient than a $sort stage after). Supports: boolean, date, number, objectId, uuid, and string (must be indexed as token type). Cannot sort on embeddedDocuments type fields.

db.collection.aggregate([
  {
    $search: {
      text: { ... },
      sort: { "fieldName": -1, "title": 1, score: { $meta: "searchScore" } }
    }
  },
  { $limit: 10 }
])

Sort by score:

sort: { score: { $meta: "searchScore", order: 1 } }  // ascending (lowest score first)
sort: { score: { $meta: "searchScore" } }             // descending (default)

Null/missing values: Appear first in ascending sort by default. Use noData: "highest" to push them last:

sort: { "field": { order: 1, noData: "highest" } }

Key rules:

• sort inside $search only works on indexed fields — use $sort after for non-indexed or computed fields
• For searchSequenceToken pagination, sort must include a unique field (e.g., _id) to avoid tie-ordering
• Arrays: ascending uses smallest element, descending uses largest

Using Stored Source

Retrieve frequently accessed fields directly from the search index instead of the database:

db.collection.aggregate([
  {
    $search: {
      index: "search_index",
      text: { query: "detective", path: "plot" },
      returnStoredSource: true  // Retrieve from mongot, not DB
    }
  },
  { $limit: 20 },
  { $match: { rating: { $gte: 7 } } }  // Filter on stored fields
])

Requirements:

• Fields must be configured in storedSource in your index definition
• Dramatically improves performance by avoiding database lookups
• Especially beneficial when filtering or sorting after $search

---

$match After $search

Minimize blocking stages after $search — prefer encapsulating filter logic inside the $search stage itself using compound.filter. This avoids additional mongod operations and makes full use of the Atlas Search index.

Prefer `compound.filter` over `$match` for fields indexed in the search index (string, token, number, date, boolean, objectId, uuid, geo):

// Prefer this
{ $search: { compound: { must: [{ text: { ... } }], filter: [{ range: { path: "year", gte: 2000 } }] } } }

// Avoid this where possible
{ $search: { text: { ... } } },
{ $match: { year: { $gte: 2000 } } }

If you must use `$match` (e.g., for non-indexed or computed fields), use storedSource + returnStoredSource to avoid a full document lookup in mongod:

{ $search: { text: { ... }, returnStoredSource: true } },
{ $match: { storedField: { $exists: true } } }

---

Query Performance Analysis

Use explain to analyze query performance:

db.collection.explain("executionStats").aggregate([
  { $search: { /* ... */ } }
])

Important: Atlas Search explain output differs from standard MongoDB explain. It shows execution on the search engine (mongot) side with Lucene-specific statistics, not standard MongoDB execution plans.

Vector Search - Indexing and Querying

This guide covers how to configure MongoDB Vector Search indexes and construct queries for semantic similarity search.

Scope: This guide covers pure vector search indexes. For hybrid search (combining lexical and vector search), see hybrid-search.md.

Table of Contents

• Vector Search Index Definition
• Index Configuration Parameters
• Filter Fields (Pre-filtering)
• Query Construction
• Query Optimization

---

Vector Search Index Definition

Syntax

MongoDB Vector Search index definitions have the following structure:

{
  "fields": [
    {
      "type": "vector",
      "path": "<field-to-index>",
      "numDimensions": <number-of-dimensions>,
      "similarity": "euclidean | cosine | dotProduct",
      "quantization": "none | scalar | binary",  // Optional
      "hnswOptions": {  // Optional (Preview feature)
        "maxEdges": <number-of-connected-neighbors>,
        "numEdgeCandidates": <number-of-nearest-neighbors>
      }
    },
    {
      "type": "filter",  // Optional: for pre-filtering
      "path": "<field-to-index>"
    }
  ]
}

Note: The exact syntax for creating indexes varies by driver/interface. The above shows the core index definition structure that applies across all methods.

Basic Definition

Most vector search indexes only need the vector field:

{
  "fields": [
    {
      "type": "vector",
      "path": "<embedding-field>",
      "numDimensions": <number>,
      "similarity": "<similarity-function>"
    }
  ]
}

---

Index Configuration Parameters

Required: numDimensions

Definition: Number of dimensions in your vector embeddings. MongoDB enforces this at both index-time and query-time.

Constraints:

• Must be less than or equal to 8192
• For int1 (binary) vectors: MUST be a multiple of 8
• For int8 vectors: 1 to 8192
• For float32 vectors: 1 to 8192

How to Determine:

• The embedding model determines this value
• It MUST match the actual dimension count of your vectors
• Cannot be changed after index creation (requires dropping and recreating index)

Example - Voyage AI Models:

• voyage-3-large: 2048 dimensions
• voyage-4: Configurable output dimensions (256, 512, 1024, 2048, 4096)

---

Required: similarity

Definition: The similarity function used to compare vectors and rank results.

Available Options:

Decision Process: 1. Check your embedding model documentation for recommended similarity function 2. If model produces normalized vectors -> use dotProduct (fastest) 3. If model does NOT normalize vectors -> use cosine 4. If using binary quantization (int1) -> MUST use euclidean 5. When uncertain -> start with dotProduct and normalize your vectors

Notes:

• All functions return scores in range [0, 1] where 1 = most similar
• dotProduct is most efficient but requires normalized vectors
• Check embedding model documentation for recommendations

---

Optional: quantization

Definition: Automatic vector compression to reduce storage and improve query speed at the cost of some accuracy.

Syntax:

{
  "type": "vector",
  "path": "<field>",
  "numDimensions": <number>,
  "similarity": "<function>",
  "quantization": "none | scalar | binary"
}

Options:

Important Rules:

• none: Default if omitted. Use for pre-quantized vectors (int1, int8)
• scalar: Transforms float32/double values to 1-byte integers
• binary: Transforms values to single bit. numDimensions MUST be multiple of 8
• Binary quantization REQUIRES euclidean similarity
• Only use with float32 or double vectors (NOT with pre-quantized int1/int8)

Example:

{
  "type": "vector",
  "path": "plot_embedding",
  "numDimensions": 1536,
  "similarity": "cosine",
  "quantization": {
    "type": "scalar"
  }
}

---

Optional: hnswOptions (Preview Feature)

Definition: Parameters for the Hierarchical Navigable Small Worlds graph construction algorithm.

Warning: Modifying default values might negatively impact your index and queries. Use with caution.

Syntax:

{
  "type": "vector",
  "path": "<field>",
  "numDimensions": <number>,
  "similarity": "<function>",
  "hnswOptions": {
    "maxEdges": <16-64>,  // Default: 16
    "numEdgeCandidates": <100-3200>  // Default: 100
  }
}

Parameters:

maxEdges (16-64, default: 16):

• Maximum number of connections per node in the graph
• Higher values:
• Better recall (finds more relevant results)
• Slower queries (more neighbors to evaluate)
• More memory usage (more connections stored)
• Slower indexing (more neighbors to adjust)

numEdgeCandidates (100-3200, default: 100):

• Maximum nodes evaluated to find best connections for new nodes
• Higher values:
• Better graph quality (improves search accuracy)
• Can negatively affect query latency

Recommendation: Leave at defaults unless you have specific performance requirements and understand the trade-offs.

---

Filter Fields (Pre-filtering)

About Filter Fields

Definition: Additional fields indexed to enable pre-filtering before vector similarity computation. This narrows the search scope and improves performance.

Use Case: Filter by specific criteria (e.g., category, date range, user ID) BEFORE computing vector similarity.

Performance: Filtering before similarity computation is much faster than post-filtering with $match.

Supported Field Types: boolean, date, objectId, numeric (int32, int64, double), string, UUID, and arrays of these types.

---

Syntax

{
  "fields": [
    {
      "type": "vector",
      "path": "embedding",
      "numDimensions": 1024,
      "similarity": "cosine"
    },
    {
      "type": "filter",
      "path": "category"  // String field for filtering
    },
    {
      "type": "filter",
      "path": "year"  // Numeric field for filtering
    }
  ]
}

---

When to Use Filter Fields

Use filter fields when:

• You need to filter by exact values (category = "Action")
• You need range filtering (year >= 2020)
• Filter criteria are known at query time
• You want maximum query performance (filters before computing similarity)
• You have multi-tenant data that needs isolation

Use post-filtering ($match) when:

• Filters are ad-hoc and change frequently
• Complex aggregation logic is needed
• Fields are not worth indexing (rarely used)
• Combining with other aggregation stages

---

Supported Filter Operators

MongoDB Vector Search supports the following MQL operators in the filter option:

Note: Other query operators, aggregation pipeline operators, and MongoDB Search operators are NOT supported in the filter option.

---

Filter Examples

Index with filter fields:

{
  "fields": [
    {
      "type": "vector",
      "path": "plot_embedding",
      "numDimensions": 2048,
      "similarity": "dotProduct"
    },
    {
      "type": "filter",
      "path": "genres"  // String or array of strings
    },
    {
      "type": "filter",
      "path": "year"  // Numeric field
    }
  ]
}

Query with single filter:

{
  $vectorSearch: {
    queryVector: [<array-of-numbers>],
    path: "plot_embedding",
    filter: {
      genres: { $eq: "Action" }
    },
    numCandidates: 150,
    limit: 10
  }
}

Query with multiple filters using $and:

{
  $vectorSearch: {
    queryVector: [<array-of-numbers>],
    path: "plot_embedding",
    filter: {
      $and: [
        { genres: "Action" },
        { year: { $gte: 2020 } }
      ]
    },
    numCandidates: 150,
    limit: 10
  }
}

Short form of $eq (recommended):

{
  $vectorSearch: {
    queryVector: [<array-of-numbers>],
    path: "plot_embedding",
    filter: {
      genres: "Action",  // Equivalent to { genres: { $eq: "Action" } }
      year: { $gte: 2020 }
    },
    numCandidates: 150,
    limit: 10
  }
}

---

Important Notes

Pre-filtering does NOT affect scores: The vectorSearchScore returned for documents is based only on vector similarity, not on how well they matched the filter criteria.

Filter fields must be indexed: You must add fields as type "filter" in your index definition to use them in the filter option. Fields not indexed cannot be used for pre-filtering.

Arrays are supported: You can filter on fields that contain arrays. MongoDB automatically handles array matching.

---

Query Construction

$vectorSearch Stage

Definition: The $vectorSearch stage performs semantic search for a query vector on indexed vector fields. It must be the first stage in an aggregation pipeline.

Requirements:

• Atlas cluster running MongoDB v6.0.11, v7.0.2, or later
• A vector search index on the collection with vector-type fields
• $vectorSearch MUST be the first stage in the pipeline

---

Basic Query Syntax

{
  "$vectorSearch": {
    "index": "<index-name>",
    "path": "<field-to-search>",
    "queryVector": [<array-of-numbers>],
    "numCandidates": <number-of-candidates>,
    "limit": <number-of-results>,
    "filter": {<filter-specification>},  // Optional
    "exact": true | false  // Optional
  }
}

---

Required Fields

index (String, Required):

• Name of the MongoDB Vector Search index to use
• MongoDB returns no results if the index name is misspelled or doesn't exist
• Must match the name specified when creating the index

path (String, Required):

• Name of the indexed vector field to search
• Must be a field indexed as type "vector" in your index definition
• Use dot notation for nested fields (e.g., "metadata.embedding")

queryVector (Array of Numbers, Required):

• Array of numbers representing your query vector
• Can be float32, BSON BinData float32, or BSON BinData int1/int8
• Array size MUST match numDimensions specified in the index
• You must use the same embedding model that generated the indexed vectors

limit (Integer, Required):

• Number of documents to return in results
• Must be an integer value
• Cannot exceed numCandidates if numCandidates is specified

---

Conditional Fields

numCandidates (Integer, Conditional):

• Number of nearest neighbors to use during ANN search
• Required if exact is false or omitted
• Must be less than or equal to 10000
• Cannot be less than limit
• Recommended: Set to at least 20x the limit value for good recall

Example:

{
  $vectorSearch: {
    queryVector: [<array>],
    path: "embedding",
    numCandidates: 150,  // 15x the limit
    limit: 10
  }
}

---

Optional Fields

filter (Object, Optional):

• MQL expression to pre-filter documents before vector search
• Only works with fields indexed as type "filter"
• Supported operators: $eq, $ne, $gt, $lt, $gte, $lte, $in, $nin, $exists, $and, $or, $not, $nor
• See Filter Fields section for details and examples

exact (Boolean, Optional):

• Set to true for ENN (Exact Nearest Neighbor) search
• Set to false or omit for ANN (Approximate Nearest Neighbor) search
• Default: false

ENN vs ANN:

• ANN (default): Faster, uses HNSW algorithm, good for large datasets, 90-95% recall
• ENN: Exhaustive search, guaranteed exact matches, slower, use for small datasets (less than 10K docs) or measuring accuracy baseline

---

Complete Query Examples

Basic ANN query:

db.collection.aggregate([
  {
    $vectorSearch: {
      index: "vector_index",
      path: "plot_embedding",
      queryVector: [<1536-dimension-array>],
      numCandidates: 150,
      limit: 10
    }
  },
  {
    $project: {
      _id: 0,
      title: 1,
      plot: 1,
      score: { $meta: "vectorSearchScore" }
    }
  }
])

ANN query with pre-filtering:

db.collection.aggregate([
  {
    $vectorSearch: {
      index: "vector_index",
      path: "plot_embedding",
      queryVector: [<2048-dimension-array>],
      filter: {
        $and: [
          { year: { $gte: 1955 } },
          { year: { $lt: 1975 } }
        ]
      },
      numCandidates: 150,
      limit: 10
    }
  },
  {
    $project: {
      _id: 0,
      title: 1,
      year: 1,
      score: { $meta: "vectorSearchScore" }
    }
  }
])

ENN query (exact search):

db.collection.aggregate([
  {
    $vectorSearch: {
      index: "vector_index",
      path: "plot_embedding",
      queryVector: [<2048-dimension-array>],
      exact: true,
      limit: 10
    }
  },
  {
    $project: {
      _id: 0,
      title: 1,
      score: { $meta: "vectorSearchScore" }
    }
  }
])

---

Retrieving Vector Search Scores

Use $meta: "vectorSearchScore" in a $project stage to include similarity scores:

{
  $project: {
    title: 1,
    score: { $meta: "vectorSearchScore" }
  }
}

Important:

• Scores are in range [0, 1] where 1 = most similar
• You can ONLY use vectorSearchScore after a $vectorSearch stage
• Pre-filtering does NOT affect the score (only vector similarity affects score)

---

Post-filtering with $match

For ad-hoc filters or complex logic not indexed as filter fields, use $match after $vectorSearch:

db.collection.aggregate([
  {
    $vectorSearch: {
      index: "vector_index",
      path: "plot_embedding",
      queryVector: [<array>],
      numCandidates: 150,
      limit: 50  // Get more candidates for post-filtering
    }
  },
  {
    $match: {
      category: "Electronics",
      "reviews.rating": { $gte: 4.5 }  // Complex nested field
    }
  },
  { $limit: 10 }
])

Performance Note: Post-filtering is slower than pre-filtering because it computes similarity for all candidates first.

---

Query Optimization

numCandidates Tuning

Definition: The numCandidates parameter controls the trade-off between recall (finding relevant results) and query performance in ANN searches.

Rule of Thumb: A good starting point is 20x your limit value. You can adjust between 10-20x (or higher) based on your recall and performance requirements.

Example:

{
  $vectorSearch: {
    queryVector: [<array>],
    path: "embedding",
    numCandidates: 200,  // 20x the limit — good starting point; tune between 10-50x based on recall and latency requirements
    limit: 10
  }
}

---

When to Adjust numCandidates

Increase when:

• Search results miss relevant documents
• Large dataset (millions of vectors)
• Using quantized vectors (int8 or int1)
• Heavy pre-filtering is applied

Decrease when:

• Queries are too slow and results are already good
• Small dataset (thousands of vectors)
• Speed is more important than perfect recall

Note on low limit values: A very low limit (e.g., 5) may need proportionally higher numCandidates (e.g., 40x) to maintain recall.

Test and Measure

• Start with 20x limit and run sample queries
• Check result quality and query latency
• Adjust up or down based on your accuracy vs performance requirements

---

ANN vs ENN Search

ANN (Approximate Nearest Neighbor):

• Default search method
• Uses HNSW algorithm for fast approximate search
• Typically 90-95% recall (finds 90-95% of exact matches)
• Much faster than ENN for large datasets
• Requires numCandidates parameter

Use ANN when:

• You have production queries
• Dataset is large (more than 10K documents)
• 90-95% recall is acceptable
• Query speed is important

ENN (Exact Nearest Neighbor):

• Exhaustive search of all indexed vectors
• Guaranteed to find exact best matches
• Much slower than ANN
• Set exact: true in query
• Does NOT require numCandidates parameter
• Uses full-fidelity vectors even when quantization is enabled

Use ENN when:

• Measuring accuracy baseline (ground truth for testing)
• Collection has less than 10K documents
• Very selective filters (less than 5% of data matches)
• You need guaranteed best matches

---

Pre-filtering vs Post-filtering Performance

Pre-filtering (filter option):

• Fastest: Filters BEFORE computing similarity
• Use for exact matches, range queries, known criteria
• Requires fields indexed as type "filter"
• Limited to supported MQL operators ($eq, $ne, $gt, $lt, $gte, $lte, $in, $nin, $exists, $and, $or, $not, $nor)

Post-filtering ($match stage):

• Slower: Computes similarity for all candidates first
• Use for ad-hoc filters, complex logic, unindexed fields
• Full MQL operator support
• Can combine with other aggregation stages

Recommendation: Use pre-filtering whenever possible for best performance. Reserve post-filtering for complex or ad-hoc queries.

---

Parallel Query Execution

MongoDB Vector Search parallelizes query execution across segments when running on dedicated search nodes, which can improve response time for queries on large datasets.

Notes:

• Works automatically on dedicated search nodes
• High-CPU systems provide more performance improvement
• Not guaranteed for every query (e.g., when too many concurrent queries are queued)
• May cause slight inconsistencies in results for successive identical queries

If you see inconsistent results: Increase numCandidates to improve consistency.

---

Best Practices Summary

1. Start with numCandidates = 20x limit: Provides good balance of recall and performance 2. Use pre-filtering when possible: Index filter fields for known filtering criteria 3. Choose appropriate similarity function: Match your embedding model's recommendations 4. Consider quantization for large datasets: Use scalar or binary quantization for 10M+ vectors 5. Use ANN for production: Reserve ENN for testing/small datasets 6. Test with your data: Run sample queries and measure recall vs latency 7. Monitor and adjust: Use query performance metrics to tune numCandidates 8. Match query vectors to index: Use the same embedding model and dimensions

---

Vector Search on Views

Version requirements, supported stages, limitations, and troubleshooting are identical to Atlas Search on Views — see lexical-search-indexing.md. The difference is using a vectorSearch-type index and querying with $vectorSearch.

Example: partial index (exclude documents without embeddings)

db.createView("moviesWithEmbeddings", "embedded_movies", [
  {
    $match: {
      $expr: { $ne: [{ $type: "$plot_embedding_voyage_3_large" }, "missing"] }
    }
  }
])

db.moviesWithEmbeddings.createSearchIndex(
  "embeddingsIndex",
  "vectorSearch",
  {
    "fields": [
      {
        "type": "vector",
        "numDimensions": 2048,
        "path": "plot_embedding_voyage_3_large",
        "similarity": "cosine"
      }
    ]
  }
)

// 8.1+: query view directly; 8.0: query source collection using index name
db.moviesWithEmbeddings.aggregate([
  {
    $vectorSearch: {
      index: "embeddingsIndex",
      path: "plot_embedding_voyage_3_large",
      queryVector: [<query-vector-2048-dimensions>],
      numCandidates: 100,
      limit: 10
    }
  }
])

---