
Byted Data Search
- 2 installs
- 408 repo stars
- Updated August 3, 2026
- volcengine/agentkit-samples
Queries compliant public data sources for company registration, A-share, and industry-chain data with exact, fuzzy, aggregate, and grouped queries.
About
Queries compliant public industry data sources for company registration, A-share, and industry-chain information with exact, fuzzy, aggregation, and grouping. A developer uses it to look up businesses, listed companies, and industry-chain metrics by region, sector, or company tags.
- Covers business registration, A-share fundamentals, and industry-chain node/region metrics
- Requires describe_datasource discovery before building any query
Byted Data Search by the numbers
- 2 all-time installs (skills.sh)
- Ranked #1,757 of 2,065 Data Science & ML skills by installs in the Skillselion catalog
- Data as of Aug 4, 2026 (Skillselion catalog sync)
npx skills add https://github.com/volcengine/agentkit-samples --skill byted-data-searchAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 2 |
|---|---|
| repo stars | ★ 408 |
| Last updated | August 3, 2026 |
| Repository | volcengine/agentkit-samples ↗ |
What it does
Queries compliant public data sources for company registration, A-share, and industry-chain data with exact, fuzzy, aggregate, and grouped queries.
Files
数据查询工具
前置要求
需要环境变量(脚本会自动读取,若读取不到需提醒用户设置):
VOLCENGINE_ACCESS_KEY(或VOLC_ACCESS_KEY)VOLCENGINE_SECRET_KEY(或VOLC_SECRET_KEY)
工作流程(严格按顺序执行)
第一步:查询可用数据源(必须先执行)
在构造任何查询之前,必须先调用此步骤了解有哪些数据源及其字段定义。这一步的作用是:确认用户需要的数据存在于哪个数据源中,以及该数据源有哪些字段和过滤规则。跳过这一步直接去猜字段名几乎一定会出错。
# 列出所有可用数据源摘要(含 datasource_id、名称、描述、维度/过滤字段数量)
python3 scripts/describe_datasource.py --datasource-id all
# 获取某个数据源的完整字段定义(维度 dimensions、字段类型、可用过滤操作符)
python3 scripts/describe_datasource.py --datasource-id <数据源ID>返回内容包含:
datasource_id:数据源唯一标识datasource_name:数据源中文名称description:数据源说明dimensions:所有字段列表,每个字段包含 field(字段名)、label(显示名)、type(类型)、description(描述)、filterable(是否可作为过滤条件字段)notes:使用备注
关键:根据返回的字段信息(尤其是 field 名称和 type 类型),确定需要用到的字段和过滤操作符,再进入第二步。
字段类型与操作符对照表
每种字段类型只支持特定操作符。用错操作符会直接报错,所以在构造 filters 之前请务必对照此表。
| 字段类型 | 支持的操作符 | 说明 |
|---|---|---|
keyword | eq, in, not_in | 精确匹配类字段(如编码、状态、类型) |
text | like, keyword | 文本类字段(如名称、地址、描述),支持模糊搜索 |
date / datetime | between, eq | 日期类字段,范围查询用 between |
long / integer / float / double | range, eq | 数值类字段,范围查询用 range |
注意:long类型字段如企业标签(is_longtou_flag等)虽然是数值类型,但用于布尔判断时用eq即可,如is_longtou_flag:eq:1。
字段取值不确定时:先探查再过滤
构造过滤条件时,经常会遇到"知道要按某个字段过滤,但不确定该字段的实际取值是什么"的情况。比如用户想按企业状态筛选,但不知道取值是"存续"、"在业"还是"正常";或者想按产业分类过滤,但不确定分类名称的准确写法。
正确做法:先做一次不带该过滤条件(或只带其他确定条件)的查询,从返回数据中观察目标字段的实际取值,再用准确的值构造过滤条件。
具体步骤: 1. 先用宽松条件查询几条数据,观察目标字段返回了哪些值 2. 如果需要看该字段有哪些不同取值,可以用 --group-by + --aggregation 做分组统计 3. 确认取值后,再加上精确的过滤条件做正式查询
示例——想按"企业状态"过滤但不确定取值:
# 第 1 步:先查几条数据,观察 reg_status 字段的实际值
python3 scripts/query_datasource.py \
--datasource-id enterprise_basic_wide \
--filters 'company_name:like:科技'
# 第 2 步:或者直接做分组统计,看 reg_status 有哪些取值及各有多少条
python3 scripts/query_datasource.py \
--datasource-id enterprise_basic_wide \
--filters 'company_name:like:科技' \
--group-by 'reg_status' \
--aggregation 'company_id:count'
# 第 3 步:确认取值后,加上精确过滤条件
python3 scripts/query_datasource.py \
--datasource-id enterprise_basic_wide \
--filters 'company_name:like:科技;reg_status:eq:存续'这个策略适用于所有 keyword 类型的枚举字段(如 reg_status、category、region_level、taxpayer_type、company_org_type 等),因为这些字段使用 eq 精确匹配,写错一个字都会导致零结果。
查询字段枚举值
当你不确定某个字段有哪些可选值时(尤其是 keyword 类型的枚举字段),可以用专门的枚举值查询脚本一步获取,而不必手动组合 --group-by 和 --aggregation 参数。该脚本返回最多 200 个不同取值,按出现频次从高到低排列。
这在以下场景特别有用:
- 构造
eq或in过滤条件前,需要知道字段的准确取值(如reg_status到底是"存续"还是"在业") - 想快速了解某个分类字段(如
category、region_level、company_org_type)有哪些选项 - 需要在特定条件范围内(如某条产业链内)查看字段的取值分布
# 基本用法:查看某个字段有哪些取值
python3 scripts/get_field_enums.py \
--datasource-id <数据源ID> \
--field <字段名>
# 带过滤条件:只看满足条件的数据中该字段有哪些取值
python3 scripts/get_field_enums.py \
--datasource-id <数据源ID> \
--field <字段名> \
--filters '<过滤条件>'参数说明:
| 参数 | 必填 | 说明 |
|---|---|---|
--datasource-id | 是 | 数据源 ID |
--field | 是 | 要查询枚举值的字段名 |
--filters | 否 | 前置过滤条件,格式同 query_datasource |
--limit | 否 | 最多返回的枚举值数量(默认 20,上限 50) |
输出示例:
数据源: enterprise_basic_wide
字段: reg_status
共找到 8 个不同取值(最多显示 200 个):
1. 存续 (5832174 条)
2. 注销 (3021487 条)
3. 在业 (1245633 条)
4. 吊销 (412056 条)
...
[JSON] ["存续", "注销", "在业", "吊销", ...]最后一行的 [JSON] 行是机器可读格式,方便程序化提取枚举值列表。
常见示例:
# 查看企业状态有哪些取值
python3 scripts/get_field_enums.py \
--datasource-id enterprise_basic_wide --field reg_status
# 查看所属行业分类有哪些
python3 scripts/get_field_enums.py \
--datasource-id enterprise_basic_wide --field category
# 查看产业链区域指标中 region_level 的取值
python3 scripts/get_field_enums.py \
--datasource-id industry_chain_node_region_metric --field region_level
# 在"新能源汽车"产业链范围内,查看企业省份分布
python3 scripts/get_field_enums.py \
--datasource-id industry_chain_company_info --field base_name \
--filters 'chain_name:like:新能源汽车'
# 查看纳税人类型有哪些
python3 scripts/get_field_enums.py \
--datasource-id enterprise_basic_wide --field taxpayer_type提示:拿到枚举值后,就可以在正式查询中使用eq或in精确过滤了。比如确认取值为"存续"后,就可以用reg_status:eq:存续过滤。
第二步:查询数据
根据第一步获取的字段信息构造查询命令:
python3 scripts/query_datasource.py \
--datasource-id <数据源ID> \
--filters '<过滤条件>' \
--page 1完整参数说明:
| 参数 | 必填 | 说明 |
|---|---|---|
--datasource-id | 是 | 数据源 ID,从第一步获取 |
--filters | 否 | 过滤条件,格式见下方,多个条件用 ; 分隔 |
--aggregation | 否 | 聚合操作:count(总数统计)、field:count(字段计数)、field:distinct(去重计数)、field:sum/avg/max/min |
--group-by | 否 | 分组字段,逗号分隔,需配合 --aggregation 使用 |
--sort-field | 否 | 排序字段名,不填使用默认排序 |
--sort-order | 否 | asc 或 desc(默认 desc) |
--page | 否 | 页码,从 1 开始(默认 1) |
过滤条件格式
格式:字段名:操作符:值,多个条件用 ; 分隔。
| 操作符 | 含义 | 示例 | 适用字段类型 |
|---|---|---|---|
eq | 精确匹配 | reg_status:eq:存续 | keyword, date, 数值 |
like | 模糊匹配(短语匹配) | company_name:like:字节跳动 | text |
in | 多值匹配(逗号分隔) | reg_status:in:存续,在业 | keyword |
not_in | 排除匹配(逗号分隔) | reg_status:not_in:注销,吊销 | keyword |
between | 日期范围(起始,结束) | estiblish_time:between:2020-01-01,2025-12-31 | date, datetime |
range | 数值范围(min,max;半开区间用 ,100 或 50,) | company_total_count:range:100, | long, integer, float, double |
keyword | 全文搜索 | keyword:keyword:新能源补贴 | text |
常见错误:
text类型字段(如company_name)不能用eq,须用like或keywordkeyword类型字段(如reg_status)不能用like,须用eq/in/not_in- 如果查询报错"字段不支持操作符",回到第一步检查字段 type
多条件组合示例:
company_name:like:科技;region_province_name:like:广东;estiblish_time:between:2020-01-01,2025-12-31当前已知数据源速查
以下为常见数据源,完整清单请调用 describe_datasource.py --datasource-id all 获取:
| 数据源ID | 名称 | 典型用途 | 关键注意 |
|---|---|---|---|
enterprise_basic_wide | 企业基本信息 | 工商注册信息、法人、注册资本、经营范围、股东融资等 | company_name 是 text 类型→用 like;max_page_size=5 |
industry_chain_company_info | 产业链企业信息 | 产业链下的企业、按标签筛选(龙头/高新/专精特新/独角兽等) | chain_name/node_name 是 text→用 like;标签字段(is_longtou_flag 等)是 long→用 eq:1 |
industry_chain_node_region_metric | 产业链节点区域指标 | 产业链的区域分布统计、企业数量、财务指标、知识产权等 | 支持聚合统计,max_page_size=50 |
stock_company_brief | 上市公司简况 | A 股上市公司基本面、F10、IPO、分红、高管等 | 必须用 code:eq:证券编码 查询(如 code:eq:000001);max_page_size=5 |
常见查询示例
企业工商信息查询
# 模糊搜索企业
python3 scripts/query_datasource.py \
--datasource-id enterprise_basic_wide \
--filters 'company_name:like:字节跳动'
# 按省份+行业筛选企业
python3 scripts/query_datasource.py \
--datasource-id enterprise_basic_wide \
--filters 'region_province_name:eq:广东;category:eq:信息传输、软件和信息技术服务业'
# 查询特定日期后成立的企业
python3 scripts/query_datasource.py \
--datasource-id enterprise_basic_wide \
--filters 'company_name:like:科技;estiblish_time:between:2023-01-01,2025-12-31'产业链企业查询
# 查某产业链下的龙头企业
python3 scripts/query_datasource.py \
--datasource-id industry_chain_company_info \
--filters 'chain_name:like:新能源汽车;is_longtou_flag:eq:1'
# 查某产业链下的专精特新企业(按省份筛选)
python3 scripts/query_datasource.py \
--datasource-id industry_chain_company_info \
--filters 'chain_name:like:半导体;is_ssdi_flag:eq:1;base_name:eq:广东'产业链区域指标查询
# 查某产业链在各省份的企业数量
python3 scripts/query_datasource.py \
--datasource-id industry_chain_node_region_metric \
--filters 'chain_name:like:新能源;region_level:eq:省' \
--sort-field company_total_count \
--sort-order desc
# 聚合统计某产业链的总企业数
python3 scripts/query_datasource.py \
--datasource-id industry_chain_node_region_metric \
--filters 'chain_name:like:人工智能' \
--aggregation 'company_total_count:sum'上市公司查询
# 按证券编码查询上市公司简况(code 必传)
python3 scripts/query_datasource.py \
--datasource-id stock_company_brief \
--filters 'code:eq:000001'探查字段取值后再过滤
# 不确定 category(所属行业)有哪些取值?用枚举值查询一步搞定
python3 scripts/get_field_enums.py \
--datasource-id enterprise_basic_wide --field category
# 看到取值后,用精确值过滤
python3 scripts/query_datasource.py \
--datasource-id enterprise_basic_wide \
--filters 'category:eq:信息传输、软件和信息技术服务业'聚合统计查询
# 统计符合条件的总记录数
python3 scripts/query_datasource.py \
--datasource-id industry_chain_company_info \
--filters 'chain_name:like:新能源汽车' \
--aggregation 'count'
# 按省份分组统计企业数量
python3 scripts/query_datasource.py \
--datasource-id industry_chain_company_info \
--filters 'chain_name:like:新能源汽车' \
--group-by 'base_name' \
--aggregation 'company_id:count'
# 按省份分组统计并去重
python3 scripts/query_datasource.py \
--datasource-id industry_chain_company_info \
--filters 'chain_name:like:新能源汽车' \
--group-by 'base_name' \
--aggregation 'company_id:distinct'调用限制
- 频率限制:每分钟最多 10 次调用
- 每日上限:每天最多 200 次调用
- 如需更多调用次数,请购买 火山引擎-高质量数据集
错误处理
| 错误类型 | 原因 | 解决方案 |
|---|---|---|
| 认证失败 | 环境变量未设置或凭证无效 | 检查 VOLCENGINE_ACCESS_KEY / VOLCENGINE_SECRET_KEY,参考 用户指南 获取 AK/SK |
| 数据源不存在 | datasource_id 错误 | 调用 describe_datasource.py --datasource-id all 确认 |
| 字段不支持操作符 | 操作符与字段类型不匹配 | 检查字段 type 并参照"字段类型与操作符对照表" |
| 字段不可用 | 字段名拼写错误或不可过滤 | 调用 describe_datasource.py --datasource-id <ID> 确认 |
| 无结果返回 | 过滤条件过严或字段取值不对 | 先用 get_field_enums.py 探查字段实际取值,或放宽条件(如 eq 改为 like) |
| 频率限制 | 调用次数超限 | 分钟限流等 1 分钟,日限流等次日 |
环境变量配置方式:
export VOLCENGINE_ACCESS_KEY="your-access-key"
export VOLCENGINE_SECRET_KEY="your-secret-key"
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.#!/usr/bin/env python3
"""Describe datasource tool — retrieves metadata for available data sources.
Usage:
python3 scripts/describe_datasource.py --datasource-id all
python3 scripts/describe_datasource.py --datasource-id enterprise_basic_wide
"""
import argparse
import json
import sys
from mcp_gateway_client import (
DEFAULT_MCP_GATEWAY_URL,
call_mcp_tool,
load_credentials,
pretty_print_mcp_result,
)
def main() -> None:
parser = argparse.ArgumentParser(
description="查询数据源元数据(维度、字段类型、可用过滤操作符)",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
示例:
# 列出所有数据源摘要
python3 scripts/describe_datasource.py --datasource-id all
# 查看企业基本信息宽表的完整字段定义
python3 scripts/describe_datasource.py --datasource-id enterprise_basic_wide
# 查看产业链企业信息字段
python3 scripts/describe_datasource.py --datasource-id industry_chain_company_info
""",
)
parser.add_argument(
"--datasource-id",
default="all",
help="数据源 ID,或 'all' 列出全部(默认: all)",
)
parser.add_argument(
"--locale",
default="zh-CN",
help="字段描述语言(默认: zh-CN)",
)
parser.add_argument(
"--url",
default=DEFAULT_MCP_GATEWAY_URL,
help="MCP Gateway URL",
)
parser.add_argument(
"--access-key",
default=None,
help="VOLCENGINE_ACCESS_KEY(可选,覆盖环境变量)",
)
parser.add_argument(
"--secret-key",
default=None,
help="VOLCENGINE_SECRET_KEY(可选,覆盖环境变量)",
)
parser.add_argument(
"--raw-response",
action="store_true",
help="输出完整 MCP JSON-RPC 响应",
)
args = parser.parse_args()
try:
ak, sk = load_credentials(args.access_key, args.secret_key)
resp = call_mcp_tool(
url=args.url,
access_key=ak,
secret_key=sk,
tool_name="describe_datasource",
arguments={"datasource_id": args.datasource_id, "locale": args.locale},
)
if args.raw_response:
print(json.dumps(resp, ensure_ascii=False, indent=2))
else:
pretty_print_mcp_result(resp)
except Exception as exc:
print(f"Error: {exc}", file=sys.stderr)
sys.exit(1)
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""Get field enum values — quickly discover the distinct values a field can take.
Returns up to 200 distinct values for a given field in a datasource,
sorted by frequency (most common first). This is essential when you need
to construct exact-match filters (eq/in) but aren't sure what values
the field actually contains.
Usage:
python3 scripts/get_field_enums.py --datasource-id enterprise_basic_wide --field reg_status
python3 scripts/get_field_enums.py --datasource-id industry_chain_company_info --field base_name --filters 'chain_name:like:新能源汽车'
"""
import argparse
import json
import sys
from mcp_gateway_client import (
DEFAULT_MCP_GATEWAY_URL,
call_mcp_tool,
extract_tool_text,
load_credentials,
)
def main() -> None:
parser = argparse.ArgumentParser(
description="查询某个数据源中指定字段的枚举值(最多返回 200 个不同取值,按出现频次降序排列)",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
示例:
# 查看企业状态有哪些取值
python3 scripts/get_field_enums.py \\
--datasource-id enterprise_basic_wide --field reg_status
# 查看某产业链下企业的省份分布
python3 scripts/get_field_enums.py \\
--datasource-id industry_chain_company_info --field base_name \\
--filters 'chain_name:like:新能源汽车'
# 查看产业链区域指标中 region_level 有哪些取值
python3 scripts/get_field_enums.py \\
--datasource-id industry_chain_node_region_metric --field region_level
""",
)
parser.add_argument(
"--datasource-id",
required=True,
help="数据源 ID(必填)",
)
parser.add_argument(
"--field",
required=True,
help="要查询枚举值的字段名(必填)",
)
parser.add_argument(
"--filters",
default=None,
help="可选的前置过滤条件,格式同 query_datasource(如 'chain_name:like:新能源汽车')",
)
parser.add_argument(
"--limit",
type=int,
default=200,
help="最多返回的枚举值数量(默认: 200,上限: 300)",
)
parser.add_argument(
"--url",
default=DEFAULT_MCP_GATEWAY_URL,
help="MCP Gateway URL",
)
parser.add_argument(
"--access-key",
default=None,
help="VOLCENGINE_ACCESS_KEY(可选,覆盖环境变量)",
)
parser.add_argument(
"--secret-key",
default=None,
help="VOLCENGINE_SECRET_KEY(可选,覆盖环境变量)",
)
args = parser.parse_args()
# Cap limit at 50 (server max page_size)
limit = min(args.limit, 300)
try:
ak, sk = load_credentials(args.access_key, args.secret_key)
# Use group_by + aggregation to get distinct values via terms aggregation.
# The server translates group_by into an ES terms aggregation whose
# bucket size equals page_size, so we set page_size = limit.
arguments = {
"datasource_id": args.datasource_id,
"group_by": args.field,
"aggregation": f"{args.field}:count",
"page_size": limit,
}
if args.filters:
arguments["filters"] = args.filters
resp = call_mcp_tool(
url=args.url,
access_key=ak,
secret_key=sk,
tool_name="query_datasource",
arguments=arguments,
)
# --- Parse response ---
tool_text = extract_tool_text(resp)
if tool_text is None:
# MCP-level error
print(json.dumps(resp, ensure_ascii=False, indent=2), file=sys.stderr)
sys.exit(1)
parsed = json.loads(tool_text)
# Check for server-side error
if "error" in parsed:
err = parsed["error"]
print(f"Error [{err.get('code', 'UNKNOWN')}]: {err.get('message', '')}", file=sys.stderr)
if "suggestion" in err:
print(f"Suggestion: {err['suggestion']}", file=sys.stderr)
sys.exit(1)
# Extract enum values from aggregation_result
agg_result = parsed.get("aggregation_result", [])
if not agg_result:
print(f"字段 '{args.field}' 在数据源 '{args.datasource_id}' 中未找到任何取值。", file=sys.stderr)
print(f"请确认字段名是否正确(可运行: python3 scripts/describe_datasource.py --datasource-id {args.datasource_id})", file=sys.stderr)
sys.exit(0)
# Each bucket in agg_result is a dict like:
# { "<field>": "<value>", "doc_count": 123, "<field>_count": 123 }
values = []
for bucket in agg_result:
val = bucket.get(args.field)
count = bucket.get("doc_count", 0)
if val is not None:
values.append({"value": val, "count": count})
# Output
print(f"数据源: {args.datasource_id}")
print(f"字段: {args.field}")
if args.filters:
print(f"过滤: {args.filters}")
print(f"共找到 {len(values)} 个不同取值(最多显示 {limit} 个):\n")
for i, item in enumerate(values, 1):
print(f" {i:>2}. {item['value']} ({item['count']} 条)")
# Also output a machine-readable JSON line for programmatic consumption
print(f"\n[JSON] {json.dumps([v['value'] for v in values], ensure_ascii=False)}")
except Exception as exc:
print(f"Error: {exc}", file=sys.stderr)
sys.exit(1)
if __name__ == "__main__":
main()
import json
import os
from typing import Any, Dict, Optional, Tuple
import requests
DEFAULT_MCP_GATEWAY_URL = (
"https://sd6k08f59gqcea6qe13vg.apigateway-cn-beijing.volceapi.com/mcp"
)
def _has_ark_skill_env() -> bool:
"""Check if ARK_SKILL_API_BASE and ARK_SKILL_API_KEY are both set.
This is an internal authentication shortcut — when the runtime environment
provides these variables the script can authenticate without requiring the
user to configure Volcengine AK/SK. The variables are never surfaced to
end-users.
"""
return bool(os.getenv("ARK_SKILL_API_BASE")) and bool(
os.getenv("ARK_SKILL_API_KEY")
)
def _get_skill_account_token() -> str:
"""Obtain a SKILL_ACCOUNT_TOKEN via the ARK GetSkillAuth API.
Calls:
GET {ARK_SKILL_API_BASE}?Version=2026-01-01&Action=GetSkillAuth
Headers:
Authorization: Bearer {ARK_SKILL_API_KEY}
ServiceName: hqd
Returns:
The token string from response ``data`` field.
Raises:
RuntimeError: If the request fails or the response indicates an error.
"""
ark_api_base = os.getenv("ARK_SKILL_API_BASE", "")
ark_api_key = os.getenv("ARK_SKILL_API_KEY", "")
auth_url = f"{ark_api_base}?Version=2026-01-01&Action=GetSkillAuth"
headers = {
"Authorization": f"Bearer {ark_api_key}",
"ServiceName": "hqd",
}
resp = requests.get(auth_url, headers=headers, timeout=15, allow_redirects=True)
resp.raise_for_status()
body = resp.json()
if body.get("code") != 0:
raise RuntimeError(
f"GetSkillAuth failed: code={body.get('code')}, msg={body.get('msg')}"
)
token = body.get("data")
if not token:
raise RuntimeError("GetSkillAuth returned empty token")
return token
def load_credentials(
access_key: Optional[str] = None, secret_key: Optional[str] = None
) -> Tuple[str, str]:
"""Load credentials from parameters or environment variables.
Authentication is resolved in the following order:
1. **Internal ARK Skill mode** — if both ``ARK_SKILL_API_BASE`` and
``ARK_SKILL_API_KEY`` are present in the environment, a
``SKILL_ACCOUNT_TOKEN`` is obtained from the ARK GetSkillAuth API.
AK/SK is **not** required in this case, so this function returns
``("", "")``.
2. **Volcengine AK/SK mode** — falls back to ``VOLCENGINE_ACCESS_KEY`` /
``VOLCENGINE_SECRET_KEY`` (or the shorter ``VOLC_ACCESS_KEY`` /
``VOLC_SECRET_KEY`` aliases).
Returns:
A ``(access_key, secret_key)`` tuple. Both values are empty strings
when internal ARK Skill mode is active.
Raises:
ValueError: If no valid credentials are found through either mode.
"""
# --- Internal: ARK Skill env (transparent, user does not need to know) ---
if _has_ark_skill_env():
return "", ""
# --- User-facing: Volcengine AK/SK ---
ak = (
access_key
or os.getenv("VOLCENGINE_ACCESS_KEY")
or os.getenv("VOLC_ACCESS_KEY")
or ""
)
sk = (
secret_key
or os.getenv("VOLCENGINE_SECRET_KEY")
or os.getenv("VOLC_SECRET_KEY")
or ""
)
if not ak or not sk:
raise ValueError(
"Missing credentials. Please set environment variables:\n"
" export VOLCENGINE_ACCESS_KEY='your-access-key'\n"
" export VOLCENGINE_SECRET_KEY='your-secret-key'\n"
"Get your AK/SK from: https://www.volcengine.com/docs/6291/65568"
)
return ak, sk
def call_mcp_tool(
*,
url: str,
access_key: str,
secret_key: str,
tool_name: str,
arguments: Dict[str, Any],
request_id: int = 1,
timeout_seconds: int = 30,
) -> Dict[str, Any]:
"""Call an MCP tool via the MCP Gateway.
All requests are sent to ``DEFAULT_MCP_GATEWAY_URL`` regardless of
authentication mode.
Authentication modes
--------------------
* **ARK Skill mode** (``ARK_SKILL_API_BASE`` + ``ARK_SKILL_API_KEY`` set):
1. Calls ``GetSkillAuth`` to obtain a ``SKILL_ACCOUNT_TOKEN``.
2. Sends the MCP request to ``url`` (= ``DEFAULT_MCP_GATEWAY_URL``)
with header ``Skill-Account-Token: <token>``.
* **Volcengine AK/SK mode** (fallback):
Sends the MCP request to ``url`` with ``Volc-Access-Key`` /
``Volc-Secret-Key`` headers.
Args:
url: MCP Gateway endpoint URL (used as the target in all modes)
access_key: Volcengine Access Key (used only in AK/SK mode)
secret_key: Volcengine Secret Key (used only in AK/SK mode)
tool_name: Name of the MCP tool to invoke
arguments: Tool arguments dict
request_id: JSON-RPC request ID
timeout_seconds: Request timeout in seconds
Returns:
JSON-RPC response dict
Raises:
requests.HTTPError: If the HTTP request fails
requests.Timeout: If the request times out
RuntimeError: If GetSkillAuth fails in ARK Skill mode
"""
payload = {
"jsonrpc": "2.0",
"id": request_id,
"method": "tools/call",
"params": {"name": tool_name, "arguments": arguments},
}
# Always use the passed url (which defaults to DEFAULT_MCP_GATEWAY_URL in
# all callers) — both auth modes hit the same gateway.
target_url = url
if _has_ark_skill_env():
# --- ARK Skill mode ---
# Step 1: Obtain SKILL_ACCOUNT_TOKEN via GetSkillAuth
token = _get_skill_account_token()
# Step 2: Call MCP Gateway with the token
headers = {
"Content-Type": "application/json",
"Accept": "application/json",
"Skill-Account-Token": token,
}
else:
# --- Volcengine AK/SK mode ---
headers = {
"Content-Type": "application/json",
"Accept": "application/json",
"Volc-Access-Key": access_key,
"Volc-Secret-Key": secret_key,
}
response = requests.post(
target_url,
headers=headers,
data=json.dumps(payload, ensure_ascii=False).encode("utf-8"),
timeout=timeout_seconds,
)
response.raise_for_status()
return response.json()
def extract_tool_text(mcp_response: Dict[str, Any]) -> Optional[str]:
"""Extract text content from MCP tool response.
Args:
mcp_response: Raw JSON-RPC response from MCP Gateway
Returns:
Concatenated text content, or None if error or no text found
"""
if "error" in mcp_response and mcp_response["error"]:
return None
result = mcp_response.get("result")
if isinstance(result, dict):
content = result.get("content")
if isinstance(content, list):
texts = []
for item in content:
if (
isinstance(item, dict)
and item.get("type") == "text"
and isinstance(item.get("text"), str)
):
texts.append(item["text"])
if texts:
return "\n".join(texts)
if isinstance(result, str):
return result
return None
def pretty_print_mcp_result(mcp_response: Dict[str, Any]) -> None:
"""Pretty-print an MCP tool response.
For text results, attempts JSON parsing for formatted output.
Falls back to raw JSON-RPC dump on error or non-text responses.
"""
if "error" in mcp_response and mcp_response["error"]:
print(json.dumps(mcp_response, ensure_ascii=False, indent=2))
return
tool_text = extract_tool_text(mcp_response)
if tool_text is None:
print(json.dumps(mcp_response, ensure_ascii=False, indent=2))
return
try:
parsed = json.loads(tool_text)
print(json.dumps(parsed, ensure_ascii=False, indent=2))
except Exception:
print(tool_text)
#!/usr/bin/env python3
"""Query datasource tool — retrieves actual data from a data source.
Usage:
python3 scripts/query_datasource.py --datasource-id enterprise_basic_wide --filters 'company_name:like:字节跳动'
python3 scripts/query_datasource.py --datasource-id industry_chain_company_info --filters 'chain_name:like:新能源' --aggregation 'count'
"""
import argparse
import json
import sys
from mcp_gateway_client import (
DEFAULT_MCP_GATEWAY_URL,
call_mcp_tool,
load_credentials,
pretty_print_mcp_result,
)
def main() -> None:
parser = argparse.ArgumentParser(
description="查询数据源数据(支持过滤、聚合、分组、排序、分页)",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
示例:
# 模糊搜索企业
python3 scripts/query_datasource.py \\
--datasource-id enterprise_basic_wide \\
--filters 'company_name:like:字节跳动'
# 聚合统计
python3 scripts/query_datasource.py \\
--datasource-id industry_chain_company_info \\
--filters 'chain_name:like:新能源汽车' \\
--aggregation 'count'
# 分组统计
python3 scripts/query_datasource.py \\
--datasource-id industry_chain_company_info \\
--filters 'chain_name:like:新能源汽车' \\
--group-by 'base_name' \\
--aggregation 'company_id:count'
# 按证券编码查上市公司
python3 scripts/query_datasource.py \\
--datasource-id stock_company_brief \\
--filters 'code:eq:000001'
""",
)
parser.add_argument("--datasource-id", required=True, help="数据源 ID(必填)")
parser.add_argument(
"--select-fields",
default=None,
help="逗号分隔的返回字段(可选,服务端默认返回所有非屏蔽维度字段)",
)
parser.add_argument(
"--filters",
default=None,
help="过滤条件,格式: 'field:op:value',多个用 ';' 分隔",
)
parser.add_argument(
"--aggregation",
default=None,
help="聚合操作: 'count' | 'field:count' | 'field:distinct' | 'field:sum/avg/max/min'",
)
parser.add_argument(
"--group-by",
default=None,
help="分组字段,逗号分隔(需配合 --aggregation 使用)",
)
parser.add_argument("--sort-field", default=None, help="排序字段名")
parser.add_argument(
"--sort-order",
default="desc",
choices=["asc", "desc"],
help="排序方向(默认: desc)",
)
parser.add_argument("--page", type=int, default=1, help="页码,从 1 开始(默认: 1)")
parser.add_argument(
"--url",
default=DEFAULT_MCP_GATEWAY_URL,
help="MCP Gateway URL",
)
parser.add_argument(
"--access-key",
default=None,
help="VOLCENGINE_ACCESS_KEY(可选,覆盖环境变量)",
)
parser.add_argument(
"--secret-key",
default=None,
help="VOLCENGINE_SECRET_KEY(可选,覆盖环境变量)",
)
parser.add_argument(
"--raw-response",
action="store_true",
help="输出完整 MCP JSON-RPC 响应",
)
args = parser.parse_args()
if args.page < 1:
raise SystemExit("--page must be >= 1")
try:
ak, sk = load_credentials(args.access_key, args.secret_key)
arguments = {
"datasource_id": args.datasource_id,
"select_fields": args.select_fields,
"filters": args.filters,
"aggregation": args.aggregation,
"group_by": args.group_by,
"sort_field": args.sort_field,
"sort_order": args.sort_order,
"page": args.page,
}
# Remove None values to avoid sending unnecessary parameters
arguments = {k: v for k, v in arguments.items() if v is not None}
resp = call_mcp_tool(
url=args.url,
access_key=ak,
secret_key=sk,
tool_name="query_datasource",
arguments=arguments,
)
if args.raw_response:
print(json.dumps(resp, ensure_ascii=False, indent=2))
else:
pretty_print_mcp_result(resp)
except Exception as exc:
print(f"Error: {exc}", file=sys.stderr)
sys.exit(1)
if __name__ == "__main__":
main()