
Tinybird Python Sdk Guidelines
- 394 installs
- 20 repo stars
- Updated July 29, 2026
- tinybirdco/tinybird-agent-skills
tinybird-python-sdk-guidelines is an agent skill that guides tinybird-sdk usage to define datasources, pipes, endpoints, and queries in Python for developers building Tinybird analytics ingestion and API layers.
About
tinybird-python-sdk-guidelines is an agent skill in tinybirdco/tinybird-agent-skills for the tinybird-sdk Python package, covering datasource, pipe, and endpoint definitions, client creation, ingestion, queries, materialized views, copy and sink pipes, and Kafka, S3, or GCS connections. The SKILL.md bundles eleven rule reference files—from getting-started and configuration through defining datasources, endpoints, low-level API, CLI commands, connections, materialized views, copy-sink pipes, and tokens—and documents seven primary CLI flows: pip install tinybird-sdk, tinybird init, tinybird dev, tinybird build, tinybird deploy, tinybird preview, and tinybird migrate from legacy .datasource/.pipe files. Developers reach for this skill when migrating file-based Tinybird projects to Python, wiring server-side ingestion, or authoring materialized views without exposing tokens to browsers. Use during Tinybird Python project setup or pipeline refactors. Skip for non-Tinybird warehouses, client-side analytics SDKs, or teams staying on legacy .datasource/.pipe files with no Python migration goal.
- tinybird-python-sdk-guidelines
- Python
- AI-coding skill
Tinybird Python Sdk Guidelines by the numbers
- 394 all-time installs (skills.sh)
- +17 installs in the week ending Jul 27, 2026 (Skillselion tracking)
- Ranked #41 of 290 Python skills by installs in the Skillselion catalog
- Data as of Jul 30, 2026 (Skillselion catalog sync)
npx skills add https://github.com/tinybirdco/tinybird-agent-skills --skill tinybird-python-sdk-guidelinesAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 394 |
|---|---|
| repo stars | ★ 20 |
| Last updated | July 29, 2026 |
| Repository | tinybirdco/tinybird-agent-skills ↗ |
How do you define Tinybird pipes in Python SDK?
Helps with python tasks.
Who is it for?
Python developers building Tinybird real-time analytics who need tinybird-sdk datasource and pipe definitions plus dev, build, and deploy CLI workflows.
Skip if: Non-Tinybird analytics stacks, browser-exposed token usage, or legacy .datasource/.pipe projects with no Python migration planned.
When should I use this skill?
User works with tinybird-sdk, defines Tinybird datasources or pipes in Python, migrates .datasource files, or runs tinybird dev, build, or deploy commands.
What you get
Python tinybird-sdk resource definitions, CLI-ready project layout, connection configs, and deployable datasource and pipe artifacts.
- Python pipe and datasource definitions
- CLI deployment configuration
- Connection and token setup notes
By the numbers
- Bundles 11 rule reference files in the skill structure
- Documents 7 Tinybird CLI commands in the quick reference
- Supports migration from legacy .datasource and .pipe file formats
Files
Tinybird Python SDK Guidelines
Guidance for using the tinybird-sdk package to define Tinybird resources in Python.
When to Apply
- Installing or configuring tinybird-sdk
- Defining datasources, pipes, or endpoints in Python
- Creating Tinybird clients in Python
- Using data ingestion or queries in Python
- Running tinybird dev/build/deploy commands for Python projects
- Migrating from legacy .datasource/.pipe files to Python
- Defining connections (Kafka, S3, GCS)
- Creating materialized views, copy pipes, or sink pipes
Rule Files
rules/getting-started.mdrules/configuration.mdrules/defining-datasources.mdrules/defining-endpoints.mdrules/client.mdrules/low-level-api.mdrules/cli-commands.mdrules/connections.mdrules/materialized-views.mdrules/copy-sink-pipes.mdrules/tokens.md
Quick Reference
- Install:
pip install tinybird-sdk - Initialize:
tinybird init - Dev mode:
tinybird dev(uses configureddev_mode, typically branch) - Build:
tinybird build(builds against configured dev target) - Deploy:
tinybird deploy(deploys to main/production) - Preview in CI:
tinybird preview - Migrate:
tinybird migrate(convert .datasource/.pipe files to Python) - Server-side only; never expose tokens in browsers
SDK CLI Commands
The SDK installs tinybird as a runtime dependency. Some commands are handled by the SDK; others delegate to the Tinybird CLI.
CLI 4.0 Build/Deploy Model
- Configure your default development target once in
tinybird.config.*(dev_mode). - Run
tinybird buildwithout environment flags for normal workflows. - Run
tinybird deployto publish to Tinybird Cloud main. - Use
--local/--branchonly as explicit overrides.
tinybird init
Initialize a new Tinybird project:
tinybird init
tinybird init --force # Overwrite existing files
tinybird init --skip-login # Skip browser authenticationCreates lib/datasources.py, lib/pipes.py, lib/client.py, and tinybird.config.json.
tinybird migrate
Migrate legacy datafiles to Python definitions:
tinybird migrate "tinybird/**/*.datasource" "tinybird/**/*.pipe" "tinybird/**/*.connection"
tinybird migrate tinybird/legacy --out ./tinybird.migration.py
tinybird migrate tinybird --dry-runConverts .datasource, .pipe, and .connection files into a Python definitions file.
tinybird dev
Watch schema files and auto-sync to Tinybird:
tinybird dev # Watch and sync using configured dev_mode
tinybird dev --local # Sync with local container
tinybird dev --branch # Force branch mode for this runImportant: In branch mode, feature branches are expected; main/master are blocked to prevent accidental production changes.
tinybird build
Build and validate resources using your configured development target:
tinybird build # Build to dev_mode target (branch or local)
tinybird build --dry-run # Preview build operations
tinybird build --local # Build to local container
tinybird build --branch # Build to branch for this runUse tinybird build for iterative development; it does not publish to production.
tinybird deploy
Deploy resources to the main workspace (production):
tinybird deploy # Deploy to main/production
tinybird deploy --dry-run # Preview without deploying
tinybird deploy --check # Validate without deploying
tinybird deploy --wait # Wait for deployment completion
tinybird deploy --allow-destructive-operations # Allow breaking changesThis is the only way to deploy to main.
tinybird preview
Create or refresh a CI preview environment for the current branch:
tinybird previewUse this in pull request workflows so preview apps query isolated Tinybird preview branches.
tinybird pull
Pull resources from remote workspace:
tinybird pull # Pull to default location
tinybird pull --output-dir ./tinybird-datafiles
tinybird pull --force # Overwrite existing filestinybird login
Authenticate via browser:
tinybird loginUseful for existing projects or token refresh.
tinybird branch
Manage branches:
tinybird branch list # List all branches
tinybird branch status # Show current branch status
tinybird branch delete <name> # Delete a branchtinybird info
Display workspace, local, and project configuration:
tinybird info # Show configuration
tinybird info --json # Output as JSONDevelopment Workflow
1. tinybird init - Initialize project 2. Define datasources and pipes in Python 3. tinybird build or tinybird dev - Iterate against configured dev target 4. tinybird preview in CI - Create preview branch environment per PR 5. tinybird deploy - Deploy to production after merge
Migration Workflow
1. tinybird migrate "path/to/*.datasource" "path/to/*.pipe" - Convert legacy files 2. Review generated Python file 3. Move definitions to lib/datasources.py and lib/pipes.py 4. Update tinybird.config.json to include Python files 5. tinybird dev - Verify sync works
Important Notes
- The CLI auto-generates datafiles from Python definitions before
build,deploy, andpreview - Use
--check/--dry-runbefore production deploys when in doubt - The CLI automatically loads
.env.localand.envfiles
Creating the Tinybird Client
Client Setup
# lib/client.py
from tinybird_sdk import Tinybird
from .datasources import page_views
from .pipes import top_pages
tinybird = Tinybird(
{
"datasources": {"page_views": page_views},
"pipes": {"top_pages": top_pages},
}
)
__all__ = ["tinybird", "page_views", "top_pages"]Using the Client
Data Ingestion
from lib.client import tinybird
# Ingest one row
tinybird.page_views.ingest(
{
"timestamp": "2024-01-15 10:30:00",
"pathname": "/home",
"session_id": "abc123",
"country": "US",
}
)
# Batch ingestion (list of rows)
tinybird.page_views.ingest([
{"timestamp": "2024-01-15 10:30:00", "pathname": "/home", "session_id": "abc", "country": "US"},
{"timestamp": "2024-01-15 10:31:00", "pathname": "/about", "session_id": "abc", "country": "US"},
])Querying Endpoints
from lib.client import tinybird
result = tinybird.top_pages.query(
{
"start_date": "2024-01-01 00:00:00",
"end_date": "2024-01-31 23:59:59",
"limit": 5,
}
)
# Access result data
for row in result["data"]:
print(f"{row['pathname']}: {row['views']} views")Datasource Operations
The client provides several operations for managing datasource data:
Append from URL
tinybird.page_views.append(
{
"url": "https://example.com/page_views.csv",
}
)Replace (Full Snapshot)
tinybird.page_views.replace(
{
"url": "https://example.com/page_views_full_snapshot.csv",
}
)Delete Rows
# Delete matching rows
tinybird.page_views.delete(
{
"delete_condition": "country = 'XX'",
}
)
# Dry run to preview deletions
tinybird.page_views.delete(
{
"delete_condition": "country = 'XX'",
"dry_run": True,
}
)Truncate
tinybird.page_views.truncate()Client Benefits
- Convenience: Access datasources and pipes as attributes
- Consistency: All operations use the same pattern
- Organization: Keep definitions and client in dedicated modules
Python App Integration
For Python web apps (FastAPI, Django, Flask), import from a dedicated module:
# In your FastAPI app
from lib.client import tinybird
@app.get("/analytics")
async def get_analytics():
result = tinybird.top_pages.query({"start_date": "2024-01-01", "end_date": "2024-01-31"})
return result["data"]SDK Configuration
Configuration File
Create a configuration file in your project root. Supported formats (in priority order):
1. tinybird.config.py - Python config with dynamic logic 2. tinybird_config.py - Python config alias 3. tinybird.config.json - Standard JSON (default) 4. tinybird.json - Legacy format
JSON Configuration
{
"include": [
"lib/*.py",
"tinybird/**/*.datasource",
"tinybird/**/*.pipe",
"tinybird/**/*.connection"
],
"token": "${TINYBIRD_TOKEN}",
"base_url": "https://api.tinybird.co",
"dev_mode": "branch"
}Python Configuration
# tinybird.config.py
config = {
"include": ["lib/*.py"],
"token": "${TINYBIRD_TOKEN}",
"base_url": "https://api.tinybird.co",
"dev_mode": "branch",
}For Python configs, export one of:
configdictCONFIGdictdefaultdictget_config()returning a dict
Configuration Options
| Option | Type | Default | Description |
|---|---|---|---|
include | list[str] | required | File paths or glob patterns for Python and raw datafiles |
token | str | required | API token; supports ${ENV_VAR} interpolation |
base_url | str | "https://api.tinybird.co" | Tinybird API URL |
dev_mode | "branch" \ | "local" | "branch" |
Token Resolution
If token is omitted, SDK resolves from: 1. TINYBIRD_TOKEN environment variable 2. .tinyb file
Base URL Resolution
If base_url is omitted, SDK resolves from: 1. TINYBIRD_URL environment variable 2. TINYBIRD_HOST environment variable 3. .tinyb file (host field) 4. Default: https://api.tinybird.co
Mixed Formats
Combine Python files with legacy .datasource, .pipe, and .connection files:
{
"include": [
"lib/datasources.py",
"lib/pipes.py",
"legacy/events.datasource",
"legacy/analytics.pipe"
]
}Local Development Mode
Use a local Tinybird container:
1. Start the container:
docker run -d -p 7181:7181 --name tinybird-local tinybirdco/tinybird-local:latest2. Configure your project:
{
"dev_mode": "local"
}Or use CLI flag:
tinybird dev --localDefining Connections
Connections define external data sources that Tinybird can integrate with.
Kafka Connection
from tinybird_sdk import define_kafka_connection, secret
events_kafka = define_kafka_connection(
"events_kafka",
{
"bootstrap_servers": "kafka.example.com:9092",
"security_protocol": "SASL_SSL",
"sasl_mechanism": "PLAIN",
"key": secret("KAFKA_KEY"),
"secret": secret("KAFKA_SECRET"),
},
)S3 Connection
from tinybird_sdk import define_s3_connection
landing_s3 = define_s3_connection(
"landing_s3",
{
"region": "us-east-1",
"arn": "arn:aws:iam::123456789012:role/tinybird-s3-access",
},
)GCS Connection
from tinybird_sdk import define_gcs_connection, secret
landing_gcs = define_gcs_connection(
"landing_gcs",
{
"service_account_credentials_json": secret("GCS_SERVICE_ACCOUNT_CREDENTIALS_JSON"),
},
)Using Secrets
The secret() function references secrets stored in Tinybird:
from tinybird_sdk import secret
# Reference a secret by name
api_key = secret("MY_API_KEY")Secrets must be created in Tinybird before deploying connections that use them.
Connection Configuration Options
Kafka Options
| Option | Description |
|---|---|
bootstrap_servers | Kafka broker addresses |
security_protocol | Protocol (e.g., SASL_SSL, PLAINTEXT) |
sasl_mechanism | SASL mechanism (e.g., PLAIN, SCRAM-SHA-256) |
key | SASL username (use secret()) |
secret | SASL password (use secret()) |
S3 Options
| Option | Description |
|---|---|
region | AWS region |
arn | IAM role ARN for cross-account access |
GCS Options
| Option | Description |
|---|---|
service_account_credentials_json | Service account JSON (use secret()) |
Using Connections in Sink Pipes
Connections are referenced when defining sink pipes:
from tinybird_sdk import define_sink_pipe, node
kafka_sink = define_sink_pipe(
"kafka_events_sink",
{
"sink": {
"connection": events_kafka, # Reference the connection
"topic": "events_export",
"schedule": "@on-demand",
},
"nodes": [
node({"name": "publish", "sql": "SELECT * FROM events"})
],
},
)Copy Pipes and Sink Pipes
Copy Pipes
Copy pipes execute SQL and write results to a datasource on a schedule or on-demand.
Scheduled Copy Pipe
from tinybird_sdk import define_copy_pipe, node
daily_snapshot = define_copy_pipe(
"daily_snapshot",
{
"datasource": events, # Target datasource
"copy_schedule": "0 0 * * *", # Cron: daily at midnight
"copy_mode": "append",
"nodes": [
node(
{
"name": "snapshot",
"sql": """
SELECT today() AS snapshot_date, event_name, count() AS events
FROM events
WHERE toDate(timestamp) = today() - 1
GROUP BY event_name
""",
}
)
],
},
)On-Demand Copy Pipe
manual_report = define_copy_pipe(
"manual_report",
{
"datasource": events,
"copy_schedule": "@on-demand",
"copy_mode": "replace",
"nodes": [
node(
{
"name": "report",
"sql": "SELECT * FROM events WHERE timestamp >= now() - interval 7 day",
}
)
],
},
)Copy Modes
| Mode | Description |
|---|---|
append | Add rows to existing data (default) |
replace | Replace all data in target datasource |
Schedule Options
| Schedule | Description |
|---|---|
"0 0 * * *" | Cron expression (daily at midnight) |
"*/5 * * * *" | Every 5 minutes |
"@on-demand" | Manual trigger only |
"@once" | Run once on deployment |
Sink Pipes
Sink pipes publish query results to external systems (Kafka, S3).
Kafka Sink
from tinybird_sdk import define_sink_pipe, node
kafka_events_sink = define_sink_pipe(
"kafka_events_sink",
{
"sink": {
"connection": events_kafka, # Kafka connection
"topic": "events_export",
"schedule": "@on-demand",
},
"nodes": [
node(
{
"name": "publish",
"sql": "SELECT timestamp, payload FROM kafka_events",
}
)
],
},
)S3 Sink
s3_events_sink = define_sink_pipe(
"s3_events_sink",
{
"sink": {
"connection": landing_s3, # S3 connection
"bucket_uri": "s3://my-bucket/exports/",
"file_template": "events_{date}",
"format": "csv",
"schedule": "@once",
"strategy": "create_new",
"compression": "gzip",
},
"nodes": [
node(
{
"name": "export",
"sql": "SELECT timestamp, session_id FROM s3_landing",
}
)
],
},
)S3 Sink Options
| Option | Description |
|---|---|
bucket_uri | S3 bucket and path prefix |
file_template | Filename template (supports {date}, {time}) |
format | Output format: csv, json, parquet |
schedule | Cron expression or @on-demand, @once |
strategy | create_new or overwrite |
compression | none, gzip, lz4 |
Defining Datasources
Basic Datasource Definition
from tinybird_sdk import define_datasource, t, engine
page_views = define_datasource(
"page_views",
{
"description": "Page view tracking data",
"schema": {
"timestamp": t.date_time(),
"pathname": t.string(),
"session_id": t.string(),
"country": t.string().low_cardinality().nullable(),
},
"engine": engine.merge_tree(
{
"sorting_key": ["pathname", "timestamp"],
}
),
},
)Schema Types
The t object provides type definitions:
String Types
t.string()- Variable-length stringt.fixed_string(n)- Fixed-length stringt.uuid()- UUID type
Numeric Types
t.int32(),t.int64()- Signed integerst.uint32(),t.uint64()- Unsigned integerst.float32(),t.float64()- Floating pointt.decimal(precision, scale)- Decimal type
Date/Time Types
t.date_time()- DateTime typet.date_time64(precision)- DateTime64 with precision (0-9)t.date()- Date type
Other Types
t.bool()- Boolean type (stored as UInt8)t.array(inner_type)- Array of any typet.map(key_type, value_type)- Map/dictionary type
Aggregate Types
t.simple_aggregate_function(func, inner_type)- For summing merge treet.aggregate_function(func, inner_type)- For aggregating merge tree
Type Modifiers
Chain modifiers on types:
.nullable()- Make column nullable.low_cardinality()- Use LowCardinality encoding for low-unique strings.default(value)- Set default value
Example:
schema = {
"tags": t.array(t.string()),
"country": t.string().low_cardinality().nullable(),
"score": t.float64().nullable(),
"status": t.string().default("pending"),
}Engine Configuration
MergeTree
engine.merge_tree(
{
"sorting_key": ["column1", "column2"],
"partition_key": "toYYYYMM(timestamp)", # optional
"ttl": "timestamp + INTERVAL 90 DAY", # optional
}
)ReplacingMergeTree
engine.replacing_merge_tree(
{
"sorting_key": ["id"],
"ver": "updated_at",
}
)SummingMergeTree
engine.summing_merge_tree(
{
"sorting_key": ["date", "category"],
"columns": ["count", "total"],
}
)AggregatingMergeTree
engine.aggregating_merge_tree(
{
"sorting_key": ["date", "dimension"],
}
)Schema Inference
Use the infer module to extract schemas:
from tinybird_sdk.infer import infer_row_schema
row_schema = infer_row_schema(page_views)
# Returns dict with column names and typesDefining Endpoints (Pipes)
Basic Endpoint Definition
from tinybird_sdk import define_endpoint, node, t, p
top_pages = define_endpoint(
"top_pages",
{
"description": "Get the most visited pages",
"params": {
"start_date": p.date_time(),
"end_date": p.date_time(),
"limit": p.int32().optional(10),
},
"nodes": [
node(
{
"name": "aggregated",
"sql": """
SELECT pathname, count() AS views
FROM page_views
WHERE timestamp >= {{DateTime(start_date)}}
AND timestamp <= {{DateTime(end_date)}}
GROUP BY pathname
ORDER BY views DESC
LIMIT {{Int32(limit, 10)}}
""",
}
)
],
"output": {
"pathname": t.string(),
"views": t.uint64(),
},
},
)Parameter Types
The p object provides parameter definitions:
p.string()- String parameterp.int32(),p.int64()- Integer parametersp.float32(),p.float64()- Float parametersp.date_time()- DateTime parameterp.date()- Date parameter
Parameter Modifiers
.optional(default_value)- Make parameter optional with a default.describe(text)- Add description for documentation
Example:
params = {
"limit": p.int32().optional(10),
"filter": p.string().optional(""),
"status": p.string().optional("active").describe("Filter by status"),
}Internal Pipes (Non-API)
Use define_pipe for pipes not exposed as API endpoints:
from tinybird_sdk import define_pipe, node, p
filtered_events = define_pipe(
"filtered_events",
{
"description": "Filter events by date range",
"params": {
"start_date": p.date_time(),
"end_date": p.date_time(),
},
"nodes": [
node(
{
"name": "filtered",
"sql": """
SELECT * FROM events
WHERE timestamp >= {{DateTime(start_date)}}
AND timestamp <= {{DateTime(end_date)}}
""",
}
)
],
},
)Multi-Node Pipes
Define multiple nodes for complex transformations:
nodes = [
node(
{
"name": "filtered",
"sql": """
SELECT * FROM events
WHERE timestamp >= {{DateTime(start_date)}}
""",
}
),
node(
{
"name": "aggregated",
"sql": """
SELECT date, count() as total
FROM filtered
GROUP BY date
""",
}
),
]SQL Templating
Use Tinybird templating in SQL:
{{Type(param_name)}}- Parameter with type{{Type(param_name, default)}}- Parameter with default value
WHERE user_id = {{String(user_id)}}
AND date >= {{Date(start_date, '2024-01-01')}}
LIMIT {{Int32(limit, 100)}}Schema Inference
from tinybird_sdk.infer import infer_params_schema, infer_output_schema
params_schema = infer_params_schema(top_pages)
output_schema = infer_output_schema(top_pages)Tinybird Python SDK Overview
What is it
The tinybird-sdk is a Python package that enables developers to define Tinybird resources in Python. You can author datasources, pipes, connections, and queries in Python, then synchronize them directly to Tinybird.
Requirements
- Python: Version 3.11 or higher
- Server-side only; web browsers are not supported to protect API credentials
Installation
pip install tinybird-sdkProject Initialization
tinybird init
tinybird init --force # Overwrite existing files
tinybird init --skip-login # Skip browser authenticationThis generates:
tinybird.config.json- Configuration filelib/datasources.py- Data source definitionslib/pipes.py- Pipe/endpoint definitionslib/client.py- Tinybird client module
Environment Setup
Create .env.local:
TINYBIRD_TOKEN=p.your_token_hereKey Features
- Define datasources, pipes, and endpoints in Python
- Data ingestion with automatic schema validation
- Query endpoints with typed results
- Mixed formats: combine Python with legacy
.datasource/.pipefiles - Branch safety: dev mode blocks deployment to main branch
- Connections: Kafka, S3, GCS integrations
- Materialized views for real-time aggregations
- Copy pipes and sink pipes for data workflows
Public Tinybird API (Low-Level)
For cases requiring a decoupled API wrapper without the high-level client:
Creating the API Client
from tinybird_sdk import create_tinybird_api
api = create_tinybird_api(
{
"base_url": "https://api.tinybird.co",
"token": "p.your_token",
}
)Querying Endpoints
top_pages = api.query(
"top_pages",
{
"start_date": "2024-01-01",
"end_date": "2024-01-31",
"limit": 5,
},
)
# top_pages["data"] contains the result rowsIngesting Data
# Ingest one row
api.ingest(
"events",
{
"timestamp": "2024-01-15 10:30:00",
"event_name": "page_view",
"pathname": "/home",
},
)
# Batch ingestion
api.ingest(
"events",
[
{"timestamp": "2024-01-15 10:30:00", "event_name": "page_view", "pathname": "/home"},
{"timestamp": "2024-01-15 10:31:00", "event_name": "click", "pathname": "/home"},
],
)Retry Behavior
Retries are disabled by default. Enable with max_retries:
api.ingest(
"events",
{"timestamp": "2024-01-15 10:31:00", "event_name": "button_click", "pathname": "/pricing"},
{"max_retries": 3},
)- 429 retries use
Retry-After/X-RateLimit-Resetheaders - 503 retries use SDK default exponential backoff
Datasource Operations
Append from URL
api.append_datasource(
"events",
{"url": "https://example.com/events.csv"},
)Delete Rows
api.delete_datasource(
"events",
{"delete_condition": "event_name = 'test'"},
)
# Dry run
api.delete_datasource(
"events",
{"delete_condition": "event_name = 'test'", "dry_run": True},
)Truncate
api.truncate_datasource("events")Executing Raw SQL
sql_result = api.sql("SELECT count() AS total FROM events")
# sql_result["data"][0]["total"]Per-Request Token Override
workspace_response = api.request_json(
"/v1/workspace",
token="p.branch_or_jwt_token",
)When to Use Low-Level API
- Existing projects not using Python definitions
- Dynamic endpoint names or parameters
- Direct SQL execution needs
- Gradual migration from other HTTP clients
- Multi-tenant scenarios with different tokens
Materialized Views
Materialized views automatically aggregate data as it arrives, enabling real-time analytics.
Basic Materialized View
A materialized view consists of: 1. A target datasource with aggregate columns 2. A materialized view definition that populates it
from tinybird_sdk import define_datasource, define_materialized_view, engine, node, t
# Target datasource with aggregate columns
daily_stats = define_datasource(
"daily_stats",
{
"schema": {
"date": t.date(),
"pathname": t.string(),
"views": t.simple_aggregate_function("sum", t.uint64()),
"unique_sessions": t.aggregate_function("uniq", t.string()),
},
"engine": engine.aggregating_merge_tree({"sorting_key": ["date", "pathname"]}),
},
)
# Materialized view that populates it
daily_stats_mv = define_materialized_view(
"daily_stats_mv",
{
"datasource": daily_stats,
"nodes": [
node(
{
"name": "aggregate",
"sql": """
SELECT
toDate(timestamp) AS date,
pathname,
count() AS views,
uniqState(session_id) AS unique_sessions
FROM page_views
GROUP BY date, pathname
""",
}
)
],
},
)Aggregate Types
SimpleAggregateFunction
For simple aggregations (sum, min, max, any):
"views": t.simple_aggregate_function("sum", t.uint64())
"min_value": t.simple_aggregate_function("min", t.float64())
"max_value": t.simple_aggregate_function("max", t.float64())AggregateFunction
For complex aggregations (uniq, quantile, etc.):
"unique_users": t.aggregate_function("uniq", t.string())
"p95_latency": t.aggregate_function("quantile(0.95)", t.float64())SQL State Functions
In materialized view SQL, use state functions to prepare aggregates:
| Final Function | State Function |
|---|---|
count() | count() (no state needed for SimpleAggregateFunction) |
sum(col) | sum(col) (no state needed) |
uniq(col) | uniqState(col) |
quantile(0.95)(col) | quantileState(0.95)(col) |
avg(col) | avgState(col) |
Querying Materialized Views
When querying, use merge functions for AggregateFunction columns:
endpoint = define_endpoint(
"daily_stats_query",
{
"nodes": [
node(
{
"name": "query",
"sql": """
SELECT
date,
pathname,
sum(views) AS total_views,
uniqMerge(unique_sessions) AS unique_sessions
FROM daily_stats
GROUP BY date, pathname
""",
}
)
],
"output": {
"date": t.date(),
"pathname": t.string(),
"total_views": t.uint64(),
"unique_sessions": t.uint64(),
},
},
)Engine Selection
Always use aggregating_merge_tree for materialized view targets:
engine.aggregating_merge_tree(
{
"sorting_key": ["date", "dimension1", "dimension2"],
}
)Tokens
Static Tokens
Define named tokens and attach them to datasources and endpoints:
from tinybird_sdk import define_datasource, define_endpoint, define_token, node, t
# Define tokens
app_token = define_token("app_read")
ingest_token = define_token("ingest_token")
# Attach to datasource
events = define_datasource(
"events",
{
"schema": {
"timestamp": t.date_time(),
"event_name": t.string(),
},
"tokens": [
{"token": app_token, "scope": "READ"},
{"token": ingest_token, "scope": "APPEND"},
],
},
)
# Attach to endpoint
top_events = define_endpoint(
"top_events",
{
"nodes": [node({"name": "endpoint", "sql": "SELECT * FROM events LIMIT 10"})],
"output": {"timestamp": t.date_time(), "event_name": t.string()},
"tokens": [{"token": app_token, "scope": "READ"}],
},
)Token Scopes
| Scope | Description |
|---|---|
READ | Read access |
APPEND | Append/ingest access |
JWT Token Creation
Create short-lived JWT tokens for secure scoped access:
from datetime import datetime, timedelta, timezone
from tinybird_sdk import create_client
client = create_client(
{
"base_url": "https://api.tinybird.co",
"token": "p.your_admin_token",
}
)
result = client.tokens.create_jwt(
{
"name": "user_123_session",
"expires_at": datetime.now(tz=timezone.utc) + timedelta(hours=1),
"scopes": [
{
"type": "PIPES:READ",
"resource": "user_dashboard",
"fixed_params": {"user_id": 123},
}
],
"limits": {"rps": 10},
}
)
jwt_token = result["token"]JWT Scope Types
| Scope | Description |
|---|---|
PIPES:READ | Read access to a specific pipe endpoint |
DATASOURCES:READ | Read access to a datasource |
DATASOURCES:APPEND | Append access to a datasource |
JWT Scope Options
| Option | Description |
|---|---|
resource | Name of the pipe or datasource |
fixed_params | Parameters embedded in token (cannot be overridden) |
filter | SQL WHERE clause for datasource filtering |
Example: Multi-Tenant Access
# Create token for specific organization
org_token = client.tokens.create_jwt(
{
"name": "org_acme_access",
"expires_at": datetime.now(tz=timezone.utc) + timedelta(days=1),
"scopes": [
{
"type": "DATASOURCES:READ",
"resource": "events",
"filter": "org_id = 'acme'",
},
{
"type": "PIPES:READ",
"resource": "analytics_dashboard",
"fixed_params": {"org_id": "acme"},
},
],
"limits": {"rps": 100},
}
)JWT Limits
| Option | Description |
|---|---|
rps | Requests per second limit |
Related skills
How it compares
Use tinybird-python-sdk-guidelines for Python-defined Tinybird resources; use ClickHouse-native skills when the analytics layer is not on Tinybird.
FAQ
What does tinybird-python-sdk-guidelines cover?
tinybird-python-sdk-guidelines guides tinybird-sdk installation, Python definitions for datasources and pipes, client usage, CLI dev/build/deploy flows, and migrations from legacy .datasource/.pipe files.
How many reference rule files ship with the skill?
tinybird-python-sdk-guidelines lists eleven rule files—including getting-started, configuration, datasources, endpoints, connections, materialized views, copy-sink pipes, and tokens.
Can Tinybird tokens be used in frontend code per this skill?
tinybird-python-sdk-guidelines states Tinybird integration is server-side only and tokens must never be exposed in browser clients.