
Building Ioc Enrichment Pipeline With Opencti
- 138 installs
- 27.3k repo stars
- Updated August 2, 2026
- mukul975/anthropic-cybersecurity-skills
Helps with ai & agent building tasks.
About
building-ioc-enrichment-pipeline-with-opencti is a Claude Code skill in the AI & Agent Building category.
- building-ioc-enrichment-pipeline-with-opencti
- AI & Agent Building
- AI-coding skill
Building Ioc Enrichment Pipeline With Opencti by the numbers
- 138 all-time installs (skills.sh)
- +4 installs in the week ending Aug 4, 2026 (Skillselion tracking)
- Ranked #3,511 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/mukul975/anthropic-cybersecurity-skills --skill building-ioc-enrichment-pipeline-with-openctiAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 138 |
|---|---|
| repo stars | ★ 27.3k |
| Last updated | August 2, 2026 |
| Repository | mukul975/anthropic-cybersecurity-skills ↗ |
What it does
Helps with ai & agent building tasks.
Files
Building IOC Enrichment Pipeline with OpenCTI
Overview
OpenCTI is an open-source platform for managing cyber threat intelligence knowledge, built on STIX 2.1 as its native data model. This skill covers building an automated IOC enrichment pipeline using OpenCTI's connector ecosystem to enrich indicators with context from VirusTotal, Shodan, AbuseIPDB, GreyNoise, and other sources. The pipeline automatically enriches newly ingested indicators, correlates them with known threat actors and campaigns, and scores them for analyst prioritization.
When to Use
- When deploying or configuring building ioc enrichment pipeline with opencti capabilities in your environment
- When establishing security controls aligned to compliance requirements
- When building or improving security architecture for this domain
- When conducting security assessments that require this implementation
Prerequisites
- Docker and Docker Compose for OpenCTI deployment
- Python 3.9+ with
pyctilibrary - API keys for enrichment services: VirusTotal, Shodan, AbuseIPDB, GreyNoise
- Understanding of STIX 2.1 data model and relationships
- ElasticSearch or OpenSearch for OpenCTI backend
- RabbitMQ or Redis for connector messaging
Key Concepts
OpenCTI Architecture
OpenCTI uses a GraphQL API frontend backed by ElasticSearch for storage and Redis/RabbitMQ for connector communication. Data is natively stored as STIX 2.1 objects with relationships. Connectors are categorized as: External Import (feed ingestion), Internal Import (file parsing), Internal Enrichment (context addition), and Stream (real-time export).
Enrichment Connector Model
Internal enrichment connectors are triggered automatically when new observables are created or manually by analysts. Each connector receives STIX objects, queries external services, and returns STIX 2.1 bundles that augment the original observable with additional context, labels, and relationships.
Confidence Scoring
OpenCTI uses a 0-100 confidence scale for indicators. Enrichment connectors can update confidence scores based on external validation: VirusTotal detection ratios, Shodan exposure data, AbuseIPDB report counts, and GreyNoise classification results.
Workflow
Step 1: Deploy OpenCTI with Docker Compose
# docker-compose.yml (key services)
version: '3'
services:
opencti:
image: opencti/platform:6.4.4
environment:
- APP__PORT=8080
- APP__ADMIN__EMAIL=admin@opencti.io
- APP__ADMIN__PASSWORD=ChangeMeNow
- APP__ADMIN__TOKEN=your-admin-token-uuid
- ELASTICSEARCH__URL=http://elasticsearch:9200
- MINIO__ENDPOINT=minio
- RABBITMQ__HOSTNAME=rabbitmq
ports:
- "8080:8080"
depends_on:
- elasticsearch
- minio
- rabbitmq
- redis
connector-virustotal:
image: opencti/connector-virustotal:6.4.4
environment:
- OPENCTI_URL=http://opencti:8080
- OPENCTI_TOKEN=your-admin-token-uuid
- CONNECTOR_ID=connector-virustotal-id
- CONNECTOR_NAME=VirusTotal
- CONNECTOR_SCOPE=StixFile,Artifact,IPv4-Addr,Domain-Name,Url
- CONNECTOR_AUTO=true
- VIRUSTOTAL_TOKEN=your-vt-api-key
- VIRUSTOTAL_MAX_TLP=TLP:AMBER
connector-shodan:
image: opencti/connector-shodan:6.4.4
environment:
- OPENCTI_URL=http://opencti:8080
- OPENCTI_TOKEN=your-admin-token-uuid
- CONNECTOR_ID=connector-shodan-id
- CONNECTOR_NAME=Shodan
- CONNECTOR_SCOPE=IPv4-Addr
- CONNECTOR_AUTO=true
- SHODAN_TOKEN=your-shodan-api-key
- SHODAN_MAX_TLP=TLP:AMBER
connector-abuseipdb:
image: opencti/connector-abuseipdb:6.4.4
environment:
- OPENCTI_URL=http://opencti:8080
- OPENCTI_TOKEN=your-admin-token-uuid
- CONNECTOR_ID=connector-abuseipdb-id
- CONNECTOR_NAME=AbuseIPDB
- CONNECTOR_SCOPE=IPv4-Addr
- CONNECTOR_AUTO=true
- ABUSEIPDB_API_KEY=your-abuseipdb-keyStep 2: Build Custom Enrichment Connector
import os
from pycti import OpenCTIConnectorHelper, get_config_variable
from stix2 import (
Bundle, Indicator, Note, Relationship,
IPv4Address, DomainName
)
import requests
class CustomEnrichmentConnector:
def __init__(self):
config = {
"opencti": {
"url": os.environ.get("OPENCTI_URL"),
"token": os.environ.get("OPENCTI_TOKEN"),
},
"connector": {
"id": os.environ.get("CONNECTOR_ID"),
"name": "CustomEnrichment",
"scope": "IPv4-Addr,Domain-Name,Url",
"auto": True,
"type": "INTERNAL_ENRICHMENT",
},
}
self.helper = OpenCTIConnectorHelper(config)
self.helper.listen(self._process_message)
def _process_message(self, data):
entity_id = data["entity_id"]
stix_object = self.helper.api.stix_cyber_observable.read(id=entity_id)
if not stix_object:
return "Observable not found"
observable_type = stix_object["entity_type"]
observable_value = stix_object.get("value", "")
enrichment_results = []
if observable_type == "IPv4-Addr":
enrichment_results = self._enrich_ip(observable_value, entity_id)
elif observable_type == "Domain-Name":
enrichment_results = self._enrich_domain(observable_value, entity_id)
if enrichment_results:
bundle = Bundle(objects=enrichment_results, allow_custom=True)
self.helper.send_stix2_bundle(bundle.serialize())
return "Enrichment completed"
def _enrich_ip(self, ip_address, entity_id):
"""Enrich IP address with GreyNoise, AbuseIPDB context."""
objects = []
# GreyNoise Community API
try:
gn_response = requests.get(
f"https://api.greynoise.io/v3/community/{ip_address}",
headers={"key": os.environ.get("GREYNOISE_API_KEY")},
timeout=30,
)
if gn_response.status_code == 200:
gn_data = gn_response.json()
classification = gn_data.get("classification", "unknown")
noise = gn_data.get("noise", False)
riot = gn_data.get("riot", False)
note_content = (
f"## GreyNoise Enrichment\n"
f"- Classification: {classification}\n"
f"- Internet Noise: {noise}\n"
f"- RIOT (Benign Service): {riot}\n"
f"- Name: {gn_data.get('name', 'N/A')}\n"
f"- Last Seen: {gn_data.get('last_seen', 'N/A')}"
)
note = Note(
content=note_content,
object_refs=[entity_id],
abstract=f"GreyNoise: {classification}",
allow_custom=True,
)
objects.append(note)
# Add labels based on classification
if classification == "malicious":
self.helper.api.stix_cyber_observable.add_label(
id=entity_id, label_name="greynoise:malicious"
)
elif riot:
self.helper.api.stix_cyber_observable.add_label(
id=entity_id, label_name="greynoise:benign-service"
)
except Exception as e:
self.helper.log_error(f"GreyNoise enrichment failed: {e}")
return objects
def _enrich_domain(self, domain, entity_id):
"""Enrich domain with WHOIS and DNS context."""
objects = []
try:
# Use SecurityTrails API for domain enrichment
st_response = requests.get(
f"https://api.securitytrails.com/v1/domain/{domain}",
headers={"APIKEY": os.environ.get("SECURITYTRAILS_API_KEY")},
timeout=30,
)
if st_response.status_code == 200:
st_data = st_response.json()
current_dns = st_data.get("current_dns", {})
a_records = [
r.get("ip") for r in current_dns.get("a", {}).get("values", [])
]
note_content = (
f"## SecurityTrails Enrichment\n"
f"- A Records: {', '.join(a_records)}\n"
f"- Alexa Rank: {st_data.get('alexa_rank', 'N/A')}\n"
f"- Hostname: {st_data.get('hostname', 'N/A')}"
)
note = Note(
content=note_content,
object_refs=[entity_id],
abstract=f"SecurityTrails: {domain}",
allow_custom=True,
)
objects.append(note)
except Exception as e:
self.helper.log_error(f"SecurityTrails enrichment failed: {e}")
return objects
if __name__ == "__main__":
connector = CustomEnrichmentConnector()
IOC Enrichment Report Template
Report Metadata
| Field | Value |
|---|---|
| Report ID | ENRICH-YYYY-NNNN |
| Date | YYYY-MM-DD HH:MM UTC |
| Platform | OpenCTI v6.x |
| Analyst | [Analyst Name] |
| Classification | TLP:AMBER |
Observable Summary
| Observable | Type | Initial Score | Enriched Score | Priority |
|---|---|---|---|---|
| x.x.x.x | IPv4-Addr | 0 | 85 | Critical |
| evil.com | Domain-Name | 0 | 62 | High |
Enrichment Results
Observable: [Value]
Type: IPv4-Addr / Domain-Name / StixFile
VirusTotal
| Metric | Value |
|---|---|
| Malicious Detections | X / Y engines |
| Suspicious | X |
| Reputation Score | X |
| AS Owner | |
| Country |
Shodan
| Metric | Value |
|---|---|
| Open Ports | |
| Known Vulnerabilities | |
| ISP | |
| Organization | |
| Operating System |
AbuseIPDB
| Metric | Value |
|---|---|
| Abuse Confidence Score | X% |
| Total Reports | X |
| Distinct Reporters | X |
| Is Tor Exit Node | Yes/No |
| Usage Type |
GreyNoise
| Metric | Value |
|---|---|
| Classification | malicious/benign/unknown |
| Internet Noise | Yes/No |
| RIOT (Benign Service) | Yes/No |
| Name | |
| Last Seen |
Confidence Scoring Breakdown
| Source | Weight | Score Contribution |
|---|---|---|
| VirusTotal | 30% | X points |
| AbuseIPDB | 30% | X points |
| GreyNoise | 20% | X points |
| Shodan | 20% | X points |
| Total | 100% | X / 100 |
Recommended Actions
| Priority | Action | Observable | Reason |
|---|---|---|---|
| Critical | Block immediately | Score > 80 | |
| High | Add to watchlist | Score 50-79 | |
| Medium | Monitor | Score 20-49 | |
| Low | No action needed | Score < 20 |
STIX Relationships Created
| Source | Relationship | Target |
|---|---|---|
| [Observable] | indicates | [Malware/Campaign] |
| [Observable] | related-to | [Infrastructure] |
| [Threat Actor] | uses | [Observable] |
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to the Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by the Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding any notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. Please do not remove or change
the license header comment from a contributed file except when
necessary.
Copyright 2026 mukul975
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
API Reference: IOC Enrichment Pipeline with OpenCTI
pycti — OpenCTI Python Client
Installation
pip install pyctiClient Initialization
from pycti import OpenCTIApiClient
client = OpenCTIApiClient(
url="http://localhost:8080",
token=os.environ.get("OPENCTI_TOKEN", "")
)Indicator Operations
# List indicators with filter
filters = {
"mode": "and",
"filters": [{"key": "value", "values": ["198.51.100.42"]}],
"filterGroups": []
}
indicators = client.indicator.list(filters=filters)
# Create indicator
client.indicator.create(
name="Malicious IP",
pattern="[ipv4-addr:value = '198.51.100.42']",
pattern_type="stix",
x_opencti_score=80,
valid_from="2025-01-01T00:00:00Z"
)Observable Operations
# Search observables
obs = client.stix_cyber_observable.list(filters=filters)
# Create observable
client.stix_cyber_observable.create(
observableData={
"type": "ipv4-addr",
"value": "198.51.100.42"
}
)Relationship Queries
# Get relationships from entity
rels = client.stix_core_relationship.list(
filters={
"mode": "and",
"filters": [{"key": "fromId", "values": [entity_id]}],
"filterGroups": []
}
)OpenCTI GraphQL API
Endpoint
POST /graphql
Authorization: Bearer <token>
Content-Type: application/jsonExample Query
query {
indicators(filters: {
mode: and
filters: [{ key: "value", values: ["198.51.100.42"] }]
filterGroups: []
}) {
edges {
node {
id
pattern
x_opencti_score
createdBy { name }
objectLabel { value }
}
}
}
}STIX Indicator Patterns
| Type | STIX Pattern |
|---|---|
| IPv4 | |
| Domain | |
| URL | |
| SHA-256 | |
| MD5 | |
Standards and Frameworks Reference
STIX 2.1 (Native Data Model for OpenCTI)
STIX Domain Objects (SDOs)
- Indicator: Contains detection patterns (STIX patterning, YARA, Sigma)
- Malware: Represents malware families and variants
- Threat Actor: Describes adversary groups and individuals
- Campaign: Groups related intrusion activity
- Attack Pattern: Maps to MITRE ATT&CK techniques
- Infrastructure: Represents adversary-owned systems (C2, exploit kits)
- Tool: Legitimate software used by adversaries
STIX Cyber Observables (SCOs)
- IPv4-Addr / IPv6-Addr: Network addresses
- Domain-Name: DNS domain names
- URL: Full URL indicators
- StixFile: File hashes (MD5, SHA-1, SHA-256)
- Email-Addr: Email addresses
- Artifact: Binary content (malware samples)
- Process: Running process information
- Network-Traffic: Network flow data
STIX Relationship Objects (SROs)
- Relationship: Connects two SDOs (e.g., Threat Actor "uses" Malware)
- Sighting: Records observation of an indicator or malware
OpenCTI Connector Standards
Connector Types
1. EXTERNAL_IMPORT: Ingest data from external sources (MISP, TAXII feeds) 2. INTERNAL_IMPORT_FILE: Parse uploaded files (PDF reports, STIX bundles) 3. INTERNAL_ENRICHMENT: Enrich existing observables with external data 4. INTERNAL_ANALYSIS: Analyze content for indicators 5. STREAM: Real-time export to external systems (SIEM, SOAR)
Connector Communication Protocol
- Connectors communicate via RabbitMQ message queues
- Messages contain STIX 2.1 bundles in JSON format
- Enrichment connectors receive entity_id and return STIX bundles
- Rate limiting and retry logic handled by connector framework
Enrichment Service APIs
VirusTotal v3 API
- Endpoint:
https://www.virustotal.com/api/v3/ - Resources: files, urls, domains, ip_addresses
- Rate limits: 4 requests/minute (free), 1000/minute (premium)
- Returns: detection ratios, behavioral analysis, relationships
Shodan API
- Endpoint:
https://api.shodan.io/ - Resources: host/{ip}, dns/resolve, search
- Returns: open ports, services, banners, vulnerabilities, ASN info
AbuseIPDB v2 API
- Endpoint:
https://api.abuseipdb.com/api/v2/ - Resources: check, reports, blacklist
- Returns: abuse confidence score, total reports, categories, country
GreyNoise v3 API
- Endpoint:
https://api.greynoise.io/v3/ - Resources: community/{ip}, noise/context/{ip}
- Returns: classification (benign/malicious/unknown), RIOT status, tags
MITRE ATT&CK Framework
- OpenCTI maps Attack Patterns to ATT&CK techniques
- Supports Enterprise, Mobile, and ICS matrices
- Technique relationships enable campaign-level analysis
- Sub-technique granularity (e.g., T1059.001 - PowerShell)
References
OpenCTI IOC Enrichment Workflows
Workflow 1: Automatic Enrichment Pipeline
[New Observable Created] --> [RabbitMQ Queue] --> [Enrichment Connectors]
|
+-----------+-----------+
| | |
v v v
[VirusTotal] [Shodan] [AbuseIPDB]
| | |
v v v
[STIX Bundle] [STIX Bundle] [STIX Bundle]
| | |
+-----------+-----------+
|
v
[Merged into OpenCTI]
|
v
[Confidence Updated]Steps:
1. Observable Ingestion: New IP/domain/hash created via feed import or manual entry 2. Queue Distribution: OpenCTI sends observable to enrichment connector queues 3. Parallel Enrichment: Each connector queries its respective external API 4. STIX Bundle Generation: Connectors produce STIX 2.1 bundles with notes, labels, relationships 5. Merge: Enrichment results merged into the observable's knowledge graph 6. Scoring: Confidence score updated based on aggregated enrichment data
Workflow 2: Analyst-Triggered Enrichment
[Analyst Selects Observable] --> [Manual Enrichment Request] --> [Selected Connectors]
| |
v v
[Review Results] <-- [Enrichment Dashboard] <-- [Results Returned]
|
v
[Update Tags/Labels] --> [Add to Investigation]Steps:
1. Selection: Analyst identifies observable requiring additional context 2. Connector Choice: Select specific enrichment connectors to run 3. Execution: Connectors query external services with observable value 4. Review: Analyst reviews enrichment results in observable detail view 5. Curation: Analyst updates labels, confidence, and adds notes 6. Investigation: Link enriched observable to ongoing investigation case
Workflow 3: Bulk Enrichment Pipeline
[STIX Import] --> [Observable Extraction] --> [Batch Queue] --> [Rate-Limited Enrichment]
|
v
[Progress Tracking]
|
v
[Enrichment Report]Steps:
1. Bulk Import: Import STIX bundle with hundreds of observables 2. Extraction: OpenCTI extracts unique observables from imported data 3. Queue Management: Observables queued for enrichment with rate limiting 4. Progressive Enrichment: Connectors process queue respecting API rate limits 5. Monitoring: Track enrichment progress via connector status dashboard 6. Reporting: Generate enrichment summary with coverage statistics
Workflow 4: Enrichment-Driven Scoring
[Raw IOC (Score: 0)] --> [VirusTotal] --> [Score += VT_detections/total * 30]
|
v
[AbuseIPDB] --> [Score += abuse_confidence * 0.3]
|
v
[GreyNoise] --> [Score += classification_weight]
|
v
[Shodan] --> [Score += open_ports_risk]
|
v
[Final Score (0-100)] --> [Priority Classification]
|
+---------+---------+
| | |
v v v
[Critical] [High] [Low]
(80-100) (50-79) (0-49)Steps:
1. Baseline: Observable starts with confidence score of 0 2. VT Score: VirusTotal detection ratio contributes up to 30 points 3. Abuse Score: AbuseIPDB confidence contributes up to 30 points 4. Classification: GreyNoise malicious/benign classification adds/subtracts points 5. Exposure: Shodan data on open ports and known vulnerabilities adds risk points 6. Final Priority: Aggregated score determines analyst priority queue placement
#!/usr/bin/env python3
"""IOC enrichment pipeline using OpenCTI and the pycti Python client.
Queries OpenCTI's GraphQL API to enrich indicators of compromise with
threat context, relationships, and scoring from connected intelligence sources.
"""
import os
import sys
import json
import datetime
import hashlib
import re
try:
from pycti import OpenCTIApiClient
HAS_PYCTI = True
except ImportError:
HAS_PYCTI = False
try:
import requests
HAS_REQUESTS = True
except ImportError:
HAS_REQUESTS = False
def init_client(url=None, token=None):
"""Initialize OpenCTI API client."""
url = url or os.environ.get("OPENCTI_URL", "http://localhost:8080")
token = token or os.environ.get("OPENCTI_TOKEN", "")
if not HAS_PYCTI:
return None
return OpenCTIApiClient(url, token)
def enrich_indicator(client, indicator_value):
"""Enrich a single indicator via OpenCTI GraphQL API."""
if not client:
return {"error": "pycti not available"}
filters = {
"mode": "and",
"filters": [{"key": "value", "values": [indicator_value]}],
"filterGroups": [],
}
results = client.indicator.list(filters=filters)
enriched = []
for ind in results:
entry = {
"id": ind.get("id"),
"pattern": ind.get("pattern"),
"name": ind.get("name"),
"valid_from": ind.get("valid_from"),
"valid_until": ind.get("valid_until"),
"score": ind.get("x_opencti_score"),
"created_by": ind.get("createdBy", {}).get("name", "Unknown") if ind.get("createdBy") else "Unknown",
"labels": [l.get("value") for l in ind.get("objectLabel", [])],
"kill_chain_phases": [
f"{k.get('kill_chain_name')}:{k.get('phase_name')}"
for k in ind.get("killChainPhases", [])
],
}
enriched.append(entry)
return enriched
def enrich_observable(client, observable_value):
"""Enrich a STIX Cyber Observable via OpenCTI."""
if not client:
return {"error": "pycti not available"}
filters = {
"mode": "and",
"filters": [{"key": "value", "values": [observable_value]}],
"filterGroups": [],
}
results = client.stix_cyber_observable.list(filters=filters)
enriched = []
for obs in results:
entry = {
"id": obs.get("id"),
"entity_type": obs.get("entity_type"),
"value": obs.get("observable_value"),
"score": obs.get("x_opencti_score"),
"labels": [l.get("value") for l in obs.get("objectLabel", [])],
"created_by": obs.get("createdBy", {}).get("name", "Unknown") if obs.get("createdBy") else "Unknown",
}
enriched.append(entry)
return enriched
def get_relationships(client, entity_id, relationship_type=None):
"""Get STIX relationships for an entity."""
if not client:
return []
filters = {
"mode": "and",
"filters": [{"key": "fromId", "values": [entity_id]}],
"filterGroups": [],
}
if relationship_type:
filters["filters"].append({"key": "relationship_type", "values": [relationship_type]})
rels = client.stix_core_relationship.list(filters=filters)
return [
{
"type": r.get("relationship_type"),
"target": r.get("to", {}).get("name", r.get("to", {}).get("observable_value", "?")),
"confidence": r.get("confidence"),
"start_time": r.get("start_time"),
}
for r in rels
]
def classify_ioc(value):
"""Classify IOC type from value string."""
if re.match(r"^[0-9]{1,3}(\.[0-9]{1,3}){3}$", value):
return "IPv4-Addr"
if re.match(r"^[a-fA-F0-9]{32}$", value):
return "MD5"
if re.match(r"^[a-fA-F0-9]{40}$", value):
return "SHA-1"
if re.match(r"^[a-fA-F0-9]{64}$", value):
return "SHA-256"
if re.match(r"^[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$", value):
return "Domain-Name"
if value.startswith("http://") or value.startswith("https://"):
return "Url"
return "Unknown"
def build_enrichment_report(client, iocs):
"""Build enrichment report for a list of IOCs."""
report = {"timestamp": datetime.datetime.utcnow().isoformat() + "Z", "iocs": []}
for ioc in iocs:
ioc_type = classify_ioc(ioc)
entry = {"value": ioc, "type": ioc_type, "indicators": [], "observables": [], "relationships": []}
if client:
entry["indicators"] = enrich_indicator(client, ioc)
entry["observables"] = enrich_observable(client, ioc)
for ind in entry["indicators"]:
if ind.get("id"):
entry["relationships"].extend(get_relationships(client, ind["id"]))
report["iocs"].append(entry)
return report
if __name__ == "__main__":
print("=" * 60)
print("IOC Enrichment Pipeline with OpenCTI")
print("pycti GraphQL client for indicator and observable enrichment")
print("=" * 60)
print(f" pycti available: {HAS_PYCTI}")
demo_iocs = ["198.51.100.42", "evil-domain.example.com", "d41d8cd98f00b204e9800998ecf8427e"]
if len(sys.argv) > 1:
demo_iocs = sys.argv[1:]
client = init_client() if HAS_PYCTI else None
if not client:
print("\n[DEMO] No OpenCTI connection. Showing classification only.")
report = build_enrichment_report(client, demo_iocs)
for ioc in report["iocs"]:
print(f"\n IOC: {ioc['value']} Type: {ioc['type']}")
print(f" Indicators found: {len(ioc['indicators'])}")
print(f" Observables found: {len(ioc['observables'])}")
print(f" Relationships: {len(ioc['relationships'])}")
summary = json.dumps({"total_iocs": len(report["iocs"])}, indent=2)
print(f"\n{summary}")
#!/usr/bin/env python3
"""
OpenCTI IOC Enrichment Pipeline Script
Automates IOC enrichment using OpenCTI's pycti library and external APIs:
- Queries VirusTotal, Shodan, AbuseIPDB, GreyNoise for IOC context
- Creates STIX 2.1 bundles with enrichment results
- Updates OpenCTI observables with enrichment data
- Generates enrichment reports with confidence scoring
Requirements:
pip install pycti stix2 requests
Usage:
python process.py --url http://localhost:8080 --token YOUR_TOKEN --enrich-ip 1.2.3.4
python process.py --url http://localhost:8080 --token YOUR_TOKEN --enrich-domain evil.com
python process.py --url http://localhost:8080 --token YOUR_TOKEN --bulk-enrich --days 1
"""
import argparse
import json
import sys
import os
from datetime import datetime, timedelta
from typing import Optional
try:
from pycti import OpenCTIApiClient
except ImportError:
print("ERROR: pycti not installed. Run: pip install pycti")
sys.exit(1)
import requests
class OpenCTIEnrichmentPipeline:
"""Automated IOC enrichment pipeline for OpenCTI."""
def __init__(self, url: str, token: str):
self.client = OpenCTIApiClient(url, token)
self.vt_key = os.environ.get("VIRUSTOTAL_API_KEY", "")
self.shodan_key = os.environ.get("SHODAN_API_KEY", "")
self.abuseipdb_key = os.environ.get("ABUSEIPDB_API_KEY", "")
self.greynoise_key = os.environ.get("GREYNOISE_API_KEY", "")
self.stats = {
"enriched": 0,
"failed": 0,
"skipped": 0,
"vt_queries": 0,
"shodan_queries": 0,
"abuseipdb_queries": 0,
"greynoise_queries": 0,
}
def enrich_ip(self, ip_address: str) -> dict:
"""Enrich an IP address with multiple external sources."""
results = {"ip": ip_address, "sources": {}, "confidence_score": 0}
# VirusTotal enrichment
if self.vt_key:
vt_data = self._query_virustotal_ip(ip_address)
if vt_data:
results["sources"]["virustotal"] = vt_data
malicious = vt_data.get("malicious_count", 0)
total = vt_data.get("total_engines", 0)
if total > 0:
results["confidence_score"] += int((malicious / total) * 30)
# Shodan enrichment
if self.shodan_key:
shodan_data = self._query_shodan(ip_address)
if shodan_data:
results["sources"]["shodan"] = shodan_data
vulns = len(shodan_data.get("vulns", []))
results["confidence_score"] += min(vulns * 5, 20)
# AbuseIPDB enrichment
if self.abuseipdb_key:
abuse_data = self._query_abuseipdb(ip_address)
if abuse_data:
results["sources"]["abuseipdb"] = abuse_data
abuse_score = abuse_data.get("abuse_confidence_score", 0)
results["confidence_score"] += int(abuse_score * 0.3)
# GreyNoise enrichment
if self.greynoise_key:
gn_data = self._query_greynoise(ip_address)
if gn_data:
results["sources"]["greynoise"] = gn_data
classification = gn_data.get("classification", "unknown")
if classification == "malicious":
results["confidence_score"] += 20
elif classification == "benign":
results["confidence_score"] = max(0, results["confidence_score"] - 20)
results["confidence_score"] = min(100, results["confidence_score"])
self.stats["enriched"] += 1
return results
def enrich_domain(self, domain: str) -> dict:
"""Enrich a domain with VirusTotal context."""
results = {"domain": domain, "sources": {}, "confidence_score": 0}
if self.vt_key:
vt_data = self._query_virustotal_domain(domain)
if vt_data:
results["sources"]["virustotal"] = vt_data
malicious = vt_data.get("malicious_count", 0)
total = vt_data.get("total_engines", 0)
if total > 0:
results["confidence_score"] += int((malicious / total) * 50)
results["confidence_score"] = min(100, results["confidence_score"])
self.stats["enriched"] += 1
return results
def enrich_hash(self, file_hash: str) -> dict:
"""Enrich a file hash with VirusTotal context."""
results = {"hash": file_hash, "sources": {}, "confidence_score": 0}
if self.vt_key:
vt_data = self._query_virustotal_hash(file_hash)
if vt_data:
results["sources"]["virustotal"] = vt_data
malicious = vt_data.get("malicious_count", 0)
total = vt_data.get("total_engines", 0)
if total > 0:
results["confidence_score"] += int((malicious / total) * 80)
results["confidence_score"] = min(100, results["confidence_score"])
self.stats["enriched"] += 1
return results
def _query_virustotal_ip(self, ip: str) -> Optional[dict]:
"""Query VirusTotal for IP address information."""
try:
resp = requests.get(
f"https://www.virustotal.com/api/v3/ip_addresses/{ip}",
headers={"x-apikey": self.vt_key},
timeout=30,
)
self.stats["vt_queries"] += 1
if resp.status_code == 200:
data = resp.json().get("data", {}).get("attributes", {})
stats = data.get("last_analysis_stats", {})
return {
"malicious_count": stats.get("malicious", 0),
"suspicious_count": stats.get("suspicious", 0),
"harmless_count": stats.get("harmless", 0),
"total_engines": sum(stats.values()) if stats else 0,
"as_owner": data.get("as_owner", ""),
"country": data.get("country", ""),
"reputation": data.get("reputation", 0),
}
except Exception as e:
print(f"[-] VirusTotal query failed for {ip}: {e}")
return None
def _query_virustotal_domain(self, domain: str) -> Optional[dict]:
"""Query VirusTotal for domain information."""
try:
resp = requests.get(
f"https://www.virustotal.com/api/v3/domains/{domain}",
headers={"x-apikey": self.vt_key},
timeout=30,
)
self.stats["vt_queries"] += 1
if resp.status_code == 200:
data = resp.json().get("data", {}).get("attributes", {})
stats = data.get("last_analysis_stats", {})
return {
"malicious_count": stats.get("malicious", 0),
"suspicious_count": stats.get("suspicious", 0),
"total_engines": sum(stats.values()) if stats else 0,
"registrar": data.get("registrar", ""),
"creation_date": data.get("creation_date", ""),
"reputation": data.get("reputation", 0),
"categories": data.get("categories", {}),
}
except Exception as e:
print(f"[-] VirusTotal query failed for {domain}: {e}")
return None
def _query_virustotal_hash(self, file_hash: str) -> Optional[dict]:
"""Query VirusTotal for file hash information."""
try:
resp = requests.get(
f"https://www.virustotal.com/api/v3/files/{file_hash}",
headers={"x-apikey": self.vt_key},
timeout=30,
)
self.stats["vt_queries"] += 1
if resp.status_code == 200:
data = resp.json().get("data", {}).get("attributes", {})
stats = data.get("last_analysis_stats", {})
return {
"malicious_count": stats.get("malicious", 0),
"suspicious_count": stats.get("suspicious", 0),
"total_engines": sum(stats.values()) if stats else 0,
"type_description": data.get("type_description", ""),
"size": data.get("size", 0),
"names": data.get("names", [])[:5],
"tags": data.get("tags", [])[:10],
}
except Exception as e:
print(f"[-] VirusTotal query failed for {file_hash}: {e}")
return None
def _query_shodan(self, ip: str) -> Optional[dict]:
"""Query Shodan for IP host information."""
try:
resp = requests.get(
f"https://api.shodan.io/shodan/host/{ip}?key={self.shodan_key}",
timeout=30,
)
self.stats["shodan_queries"] += 1
if resp.status_code == 200:
data = resp.json()
return {
"ports": data.get("ports", []),
"vulns": data.get("vulns", []),
"os": data.get("os"),
"isp": data.get("isp", ""),
"org": data.get("org", ""),
"country": data.get("country_name", ""),
"city": data.get("city", ""),
"hostnames": data.get("hostnames", []),
"tags": data.get("tags", []),
}
except Exception as e:
print(f"[-] Shodan query failed for {ip}: {e}")
return None
def _query_abuseipdb(self, ip: str) -> Optional[dict]:
"""Query AbuseIPDB for IP reputation."""
try:
resp = requests.get(
"https://api.abuseipdb.com/api/v2/check",
headers={
"Key": self.abuseipdb_key,
"Accept": "application/json",
},
params={"ipAddress": ip, "maxAgeInDays": 90, "verbose": True},
timeout=30,
)
self.stats["abuseipdb_queries"] += 1
if resp.status_code == 200:
data = resp.json().get("data", {})
return {
"abuse_confidence_score": data.get("abuseConfidenceScore", 0),
"total_reports": data.get("totalReports", 0),
"distinct_users": data.get("numDistinctUsers", 0),
"country": data.get("countryCode", ""),
"isp": data.get("isp", ""),
"usage_type": data.get("usageType", ""),
"is_tor": data.get("isTor", False),
"is_whitelisted": data.get("isWhitelisted", False),
"last_reported": data.get("lastReportedAt", ""),
}
except Exception as e:
print(f"[-] AbuseIPDB query failed for {ip}: {e}")
return None
def _query_greynoise(self, ip: str) -> Optional[dict]:
"""Query GreyNoise for IP classification."""
try:
resp = requests.get(
f"https://api.greynoise.io/v3/community/{ip}",
headers={"key": self.greynoise_key},
timeout=30,
)
self.stats["greynoise_queries"] += 1
if resp.status_code == 200:
data = resp.json()
return {
"classification": data.get("classification", "unknown"),
"noise": data.get("noise", False),
"riot": data.get("riot", False),
"name": data.get("name", ""),
"last_seen": data.get("last_seen", ""),
"message": data.get("message", ""),
}
except Exception as e:
print(f"[-] GreyNoise query failed for {ip}: {e}")
return None
def update_opencti_observable(self, observable_value: str,
enrichment: dict) -> bool:
"""Update OpenCTI observable with enrichment results."""
try:
# Search for existing observable
observables = self.client.stix_cyber_observable.list(
filters={
"mode": "and",
"filters": [{"key": "value", "values": [observable_value]}],
"filterGroups": [],
}
)
if not observables:
print(f"[-] Observable {observable_value} not found in OpenCTI")
return False
obs_id = observables[0]["id"]
score = enrichment.get("confidence_score", 0)
# Update confidence score
self.client.stix_cyber_observable.update_field(
id=obs_id,
input={"key": "x_opencti_score", "value": str(score)},
)
# Add enrichment note
note_content = json.dumps(enrichment["sources"], indent=2)
self.client.note.create(
content=f"## Automated Enrichment Results\n```json\n{note_content}\n```",
abstract=f"Enrichment Score: {score}/100",
objects=[obs_id],
)
# Add labels based on score
if score >= 80:
self.client.stix_cyber_observable.add_label(
id=obs_id, label_name="enrichment:critical"
)
elif score >= 50:
self.client.stix_cyber_observable.add_label(
id=obs_id, label_name="enrichment:high"
)
print(f"[+] Updated {observable_value} in OpenCTI (score: {score})")
return True
except Exception as e:
print(f"[-] Failed to update OpenCTI: {e}")
self.stats["failed"] += 1
return False
def bulk_enrich_recent(self, days: int = 1, max_items: int = 100):
"""Bulk enrich recently created observables."""
date_from = (datetime.now() - timedelta(days=days)).strftime(
"%Y-%m-%dT00:00:00.000Z"
)
observables = self.client.stix_cyber_observable.list(
first=max_items,
filters={
"mode": "and",
"filters": [
{"key": "created_at", "values": [date_from], "operator": "gt"}
],
"filterGroups": [],
},
)
print(f"[+] Found {len(observables)} observables to enrich")
for obs in observables:
entity_type = obs.get("entity_type", "")
value = obs.get("observable_value", "")
if not value:
continue
if entity_type == "IPv4-Addr":
enrichment = self.enrich_ip(value)
elif entity_type == "Domain-Name":
enrichment = self.enrich_domain(value)
elif entity_type in ("StixFile", "Artifact"):
hashes = obs.get("hashes", {})
sha256 = hashes.get("SHA-256", "")
if sha256:
enrichment = self.enrich_hash(sha256)
else:
continue
else:
self.stats["skipped"] += 1
continue
self.update_opencti_observable(value, enrichment)
def print_stats(self):
"""Print enrichment statistics."""
print("\n=== Enrichment Pipeline Statistics ===")
for key, value in self.stats.items():
print(f" {key.replace('_', ' ').title()}: {value}")
print("=====================================\n")
def main():
parser = argparse.ArgumentParser(
description="OpenCTI IOC Enrichment Pipeline"
)
parser.add_argument("--url", required=True, help="OpenCTI instance URL")
parser.add_argument("--token", required=True, help="OpenCTI API token")
parser.add_argument("--enrich-ip", help="Enrich a single IP address")
parser.add_argument("--enrich-domain", help="Enrich a single domain")
parser.add_argument("--enrich-hash", help="Enrich a single file hash")
parser.add_argument(
"--bulk-enrich", action="store_true", help="Bulk enrich recent observables"
)
parser.add_argument("--days", type=int, default=1, help="Lookback days for bulk")
parser.add_argument("--max-items", type=int, default=100, help="Max items for bulk")
parser.add_argument(
"--update-opencti", action="store_true",
help="Update results back to OpenCTI",
)
parser.add_argument("--output", help="Output file for enrichment results")
args = parser.parse_args()
pipeline = OpenCTIEnrichmentPipeline(args.url, args.token)
results = None
if args.enrich_ip:
results = pipeline.enrich_ip(args.enrich_ip)
if args.update_opencti:
pipeline.update_opencti_observable(args.enrich_ip, results)
elif args.enrich_domain:
results = pipeline.enrich_domain(args.enrich_domain)
if args.update_opencti:
pipeline.update_opencti_observable(args.enrich_domain, results)
elif args.enrich_hash:
results = pipeline.enrich_hash(args.enrich_hash)
if args.update_opencti:
pipeline.update_opencti_observable(args.enrich_hash, results)
elif args.bulk_enrich:
pipeline.bulk_enrich_recent(days=args.days, max_items=args.max_items)
if results:
print(json.dumps(results, indent=2, default=str))
if args.output:
with open(args.output, "w") as f:
json.dump(results, f, indent=2, default=str)
print(f"[+] Results saved to {args.output}")
pipeline.print_stats()
if __name__ == "__main__":
main()