Now liveThe Skillselion MCP - thousands of ranked skills, loaded into your agent mid-task. No install.Get it →
volcengine avatar

Byted Bytehouse Hybrid Search

  • 2 installs
  • 408 repo stars
  • Updated August 3, 2026
  • volcengine/agentkit-samples

Run hybrid search in ByteHouse combining BM25 full-text and HNSW vector retrieval, reranked with RRF, using Doubao embeddings for text vectorization.

About

Implements ByteHouse hybrid retrieval combining BM25 full-text and HNSW vector search reranked via RRF, with automatic Doubao-based text vectorization on insert. A developer uses it to build more accurate keyword-plus-semantic search over ByteHouse data.

  • Dual index: BM25 full-text inverted index and HNSW vector index
  • hybrid_search() reranks both recall paths with the RRF algorithm

Byted Bytehouse Hybrid Search by the numbers

  • 2 all-time installs (skills.sh)
  • Ranked #741 of 911 Databases skills by installs in the Skillselion catalog
  • Data as of Aug 4, 2026 (Skillselion catalog sync)
npx skills add https://github.com/volcengine/agentkit-samples --skill byted-bytehouse-hybrid-search

Add your badge

Show developers this skill is listed on Skillselion. Paste this into your README.

Listed on Skillselion
Installs2
repo stars408
Last updatedAugust 3, 2026
Repositoryvolcengine/agentkit-samples

What it does

Run hybrid search in ByteHouse combining BM25 full-text and HNSW vector retrieval, reranked with RRF, using Doubao embeddings for text vectorization.

Files

SKILL.mdMarkdownGitHub ↗

ByteHouse 混合检索 Skill

🚀 快速开始

环境准备

pip install clickhouse-connect volcengine-python-sdk[ark] numpy scipy
环境变量配置

优先从环境变量读取配置,禁止硬编码明文敏感信息

# ByteHouse 配置
export BYTEHOUSE_HOST="<你的ByteHouse连接地址>"
export BYTEHOUSE_PORT="<ByteHouse端口>"
export BYTEHOUSE_USER="<ByteHouse用户名>"
export BYTEHOUSE_PASSWORD="<ByteHouse密码>"
export BYTEHOUSE_DATABASE="<默认数据库,可选,默认default>"
export BYTEHOUSE_SECURE="<是否启用加密,可选,默认true>"

# 火山引擎方舟 API 配置
export ARK_API_KEY="<火山引擎方舟API密钥>"
export ARK_BASE_URL="https://ark.cn-beijing.volces.com/api/v3"
export EMBEDDING_MODEL="doubao-embedding-vision-251215"  # 文本向量化模型
export EMBEDDING_DIMENSIONS="1536"  # 向量维度,可选,默认1536

如果环境变量未配置,会自动提示用户输入。

---

📚 核心能力

1. 文本向量化

基于豆包文本向量化模型生成文本向量,支持任意长度中文文本。

2. 双索引构建

索引类型说明适用场景
全文倒排索引基于BM25算法的全文检索,支持关键词匹配精准关键词召回
向量索引基于HNSW的向量相似度检索,支持语义匹配语义相似召回

3. 核心功能

功能方法说明
全文检索fulltext_search()基于BM25的全文检索,返回BM25分数
向量检索vector_search()基于余弦相似度的向量检索,返回相似度分数
混合检索+RRF重排hybrid_search()双路召回后使用RRF算法重排,返回最终结果
自动生成向量insert_document()/batch_insert_documents()插入文档时自动生成向量并存储,无需手动处理
单个文档向量更新update_document_embedding()为单个文档重新生成并更新向量
批量补全缺失向量batch_update_missing_embeddings()自动扫描表中所有缺少向量的文档,批量生成并补全向量

4. RRF重排算法

Reciprocal Rank Fusion 算法,综合全文检索和向量检索的排名结果,公式:

score = Σ 1 / (k + rank)

默认k=60,可自定义调整。

---

📖 代码实现

完整示例代码实现位于 scripts/ 目录:

  • `scripts/embedding.py` - 文本向量化模块
  • `scripts/hybrid_search_client.py` - ByteHouse 混合检索客户端
  • `scripts/examples.py` - 使用示例

快速使用

from scripts import ByteHouseHybridSearch

# 初始化客户端
search = ByteHouseHybridSearch(connection_type="http")

# 创建混合检索表(自动构建全文索引和向量索引)
search.create_hybrid_table("my_hybrid_index")

# 插入文档(自动生成向量 + 存储原始文本)
search.insert_document("my_hybrid_index", doc_id=1, 
                      title="ByteHouse 混合检索", 
                      content="ByteHouse 支持全文检索和向量检索,可实现混合检索能力")

# 混合检索(自动执行全文+向量检索,RRF重排返回结果)
results = search.hybrid_search("my_hybrid_index", query="ByteHouse检索能力", top_k=10)

---

⚙️ 最佳实践

建表配置

CREATE TABLE {table_name} (
    `doc_id` UInt64,
    `title` String,
    `content` String,
    `embedding` Array(Float32),
    -- 全文倒排索引(version=2支持BM25分数)
    INDEX content_idx content TYPE inverted('standard', '{"version":"v2"}') GRANULARITY 1,
    -- 向量索引(HNSW算法,余弦相似度)
    INDEX embedding_idx embedding TYPE HNSW_SQ('DIM={vec_dimensions}', 'metric=COSINE', 'M=32', 'EF_CONSTRUCTION=256') GRANULARITY 1
)
ENGINE = MergeTree()
ORDER BY doc_id
SETTINGS 
    index_granularity = 1024,
    enable_vector_index_preload = 1

RRF参数调整

  • 当全文检索结果更重要时,可降低rrf_k值(推荐30-60)
  • 当向量检索结果更重要时,可提高rrf_k值(推荐60-100)

🔗 参考文档

Related skills

Databasesdatabasesanalytics

This week in AI coding

Five minutes, every Monday - the tools, releases and tactics for developers.

unsubscribe anytime.