N8n Workflow Patterns

n8n Workflow Patterns

Proven architectural patterns for building n8n workflows.

---

The 6 Core Patterns

Based on analysis of real workflow usage:

1. [Webhook Processing](webhook_processing.md) (Most Common)

• Receive HTTP requests → Process → Output
• Pattern: Webhook → Validate → Transform → Respond/Notify

2. [HTTP API Integration](http_api_integration.md)

• Fetch from REST APIs → Transform → Store/Use
• Pattern: Trigger → HTTP Request → Transform → Action → Error Handler

3. [Database Operations](database_operations.md)

• Read/Write/Sync database data
• Pattern: Schedule → Query → Transform → Write → Verify

4. [AI Agent Workflow](ai_agent_workflow.md)

• AI agents with tools and memory
• Pattern: Trigger → AI Agent (Model + Tools + Memory) → Output

5. [Scheduled Tasks](scheduled_tasks.md)

• Recurring automation workflows
• Pattern: Schedule → Fetch → Process → Deliver → Log

6. Batch Processing (below)

• Process large datasets in chunks with API rate limits
• Pattern: Prepare → SplitInBatches → Process per batch → Accumulate → Aggregate

---

Pattern Selection Guide

When to use each pattern:

Webhook Processing - Use when:

• Receiving data from external systems
• Building integrations (Slack commands, form submissions, GitHub webhooks)
• Need instant response to events
• Example: "Receive Stripe payment webhook → Update database → Send confirmation"

HTTP API Integration - Use when:

• Fetching data from external APIs
• Synchronizing with third-party services
• Building data pipelines
• Example: "Fetch GitHub issues → Transform → Create Jira tickets"

Database Operations - Use when:

• Syncing between databases
• Running database queries on schedule
• ETL workflows
• Example: "Read Postgres records → Transform → Write to MySQL"

AI Agent Workflow - Use when:

• Building conversational AI
• Need AI with tool access
• Multi-step reasoning tasks
• Example: "Chat with AI that can search docs, query database, send emails"

Scheduled Tasks - Use when:

• Recurring reports or summaries
• Periodic data fetching
• Maintenance tasks
• Example: "Daily: Fetch analytics → Generate report → Email team"

Batch Processing - Use when:

• Processing large datasets that exceed API batch limits
• Need to accumulate results across multiple API calls
• Nested loops (e.g., multiple categories × paginated API calls per category)
• Example: "Fetch products for 4 markets × 1000 per API call → Aggregate all results"

---

Common Workflow Components

All patterns share these building blocks:

1. Triggers

• Webhook - HTTP endpoint (instant)
• Schedule - Cron-based timing (periodic)
• Manual - Click to execute (testing)
• Polling - Check for changes (intervals)

2. Data Sources

• HTTP Request - REST APIs
• Database nodes - Postgres, MySQL, MongoDB
• Service nodes - Slack, Google Sheets, etc.
• Code - Custom JavaScript/Python

3. Transformation

• Set - Map/transform fields
• Code - Complex logic
• IF/Switch - Conditional routing
• Merge - Combine data streams

4. Outputs

• HTTP Request - Call APIs
• Database - Write data
• Communication - Email, Slack, Discord
• Storage - Files, cloud storage

5. Error Handling

• Error Trigger - Catch workflow errors
• IF - Check for error conditions
• Stop and Error - Explicit failure
• Continue On Fail - Per-node setting

---

Workflow Creation Checklist

When building ANY workflow, follow this checklist:

Planning Phase

• [ ] Identify the pattern (webhook, API, database, AI, scheduled)
• [ ] List required nodes (use search_nodes)
• [ ] Understand data flow (input → transform → output)
• [ ] Plan error handling strategy

Implementation Phase

• [ ] Create workflow with appropriate trigger
• [ ] Add data source nodes
• [ ] Configure authentication/credentials
• [ ] Add transformation nodes (Set, Code, IF)
• [ ] Add output/action nodes
• [ ] Configure error handling

Validation Phase

• [ ] Validate each node configuration (validate_node)
• [ ] Validate complete workflow (validate_workflow)
• [ ] Test with sample data
• [ ] Handle edge cases (empty data, errors)

Deployment Phase

• [ ] Review workflow settings (execution order, timeout, error handling)
• [ ] Activate workflow using activateWorkflow operation
• [ ] Monitor first executions
• [ ] Document workflow purpose and data flow

---

Workflow lifecycle: validate, verify, test before activating

Building the nodes is the start, not the finish. Before a workflow goes live, run it through four gates — and remember the headline rule: validation passing is necessary, not sufficient. A workflow can validate clean and still drop items, pick the wrong Merge input, or post Slack messages as plain text. Clean validation means the shapes are right, not that the logic is.

1. Validate. Run validate_workflow on the full JSON during build, or n8n_validate_workflow({ id }) once the workflow exists on the instance. Fix every error and re-validate. This catches schema, node-config, expression, and reference errors — the structural layer. 2. Verify the connections. Pull the workflow with n8n_get_workflow({ id }) and read the connections object directly. Validation confirms connections aren't broken; it doesn't confirm they're correct. This is where you catch the valid-but-wrong wiring: a Merge whose useDataOfInput doesn't line up with the connection slot, a Switch fallback that connects to nothing, a fan-out branch that was never wired onward, an error output that goes nowhere. (See the n8n Node Configuration skill's NODE_FAMILY_GOTCHAS.md for the silent ones.) 3. Test. Run n8n_test_workflow and inspect the output via n8n_executions. Confirm the output shape matches what consumers expect, fan-outs all produced data, and (for webhook APIs) the status/body/headers are right. Real side effects fire during a test — writes commit, messages send, external APIs are called. If any node has a user-visible side effect, confirm with the user before running, or test against safe data first. 4. Activate only after the first three pass — using n8n_update_partial_workflow with the activateWorkflow operation. Don't activate straight off a clean validation; an active workflow that drops data or double-sends is worse than one that never started.

Skipping any gate trades a few minutes now for debugging a live, possibly stateful, possibly traffic-bearing workflow later. The trade is never worth it.

---

Data Flow Patterns

Linear Flow

Trigger → Transform → Action → End

Use when: Simple workflows with single path

Branching Flow

Trigger → IF → [True Path]
             └→ [False Path]

Use when: Different actions based on conditions

Parallel Processing

Trigger → [Branch 1] → Merge
       └→ [Branch 2] ↗

Use when: Independent operations that can run simultaneously

Loop Pattern

Trigger → Split in Batches → Process → Loop (until done)

Use when: Processing large datasets in chunks

Error Handler Pattern

Main Flow → [Success Path]
         └→ [Error Trigger → Error Handler]

Use when: Need separate error handling workflow

---

Batch Processing Pattern

SplitInBatches Loop

The SplitInBatches node splits a large dataset into smaller chunks for processing. Understanding its outputs is critical:

• main[0] = done — fires ONCE after all batches complete
• main[1] = each batch — fires per batch (this is the loop body)

Prepare Items → SplitInBatches → [main[1]: Process Batch] → (loops back)
                                  [main[0]: Done] → Limit 1 → Aggregate

Always add a Limit 1 node after the done output.

Choosing batchSize (the cost lever)

A SplitInBatches loop re-runs its whole body once per iteration — ~0.8 ms/iteration of engine overhead plus the body's own cost — so total ≈ ⌈items / batchSize⌉ × (overhead + body). batchSize is a direct speed dial:

• Pick the largest batch your real constraint allows (API page size, rate limit, memory). Bigger batches = fewer iterations = less overhead; the body still sees every item.
• batchSize: 1 is the expensive extreme — one full engine pass per item. Use it only when you must act on a single item at a time (nested-loop control, or an API that takes exactly one id).
• If you're looping only to "go over the items" with no external constraint, you usually don't need the loop — a single All Items Code node processes the whole set far cheaper.

Cross-Iteration Data

After the loop, $('Node Inside Loop').all() returns ONLY the last batch's items. To accumulate across all iterations, use $getWorkflowStaticData('global') in a Code node inside the loop. See the n8n Code JavaScript skill for the full pattern.

Nested Loops

When processing N categories × M items per category (where an API has a batch limit):

Define Categories (N items)
  → Outer Loop (SplitInBatches, batchSize=1)
    → Prepare category data
    → Inner Loop (SplitInBatches, batchSize=1000)
      → API Call → Verify → (loops back to Inner Loop via main[1])
    → Inner done[0] → Rate Limit Delay → back to Outer Loop
  → Outer done[0] → Limit 1 → Final Aggregate

Wiring gotcha: The inner done[0] must connect back to the OUTER loop input, not to the aggregate. The outer done[0] feeds the final aggregate.

API Pagination

For APIs without multi-ID filtering, use id_from + date windowing for efficient pagination:

Schedule → Set Date Window → Fetch Page → Process
  → IF has more? → [true] Update id_from → Fetch Page (loop)
                  → [false] → Aggregate → Output

Dry-Run / Verification Tolerance

When testing with API write nodes disabled (for dry runs), downstream verification nodes receive the request body instead of the response. Make verification tolerant:

// In verification Code node
const body = $input.first().json;
const looksLikeRequest = body.method && body.parameters && !body.status;
if (looksLikeRequest) {
  return [{ json: { status: 'SKIPPED', message: 'Upstream disabled for testing' }}];
}
// Normal response verification below...

---

Performance on the hot path

When a workflow processes thousands of items with little I/O, its speed is set by how many times n8n crosses a per-item / per-iteration boundary — each crossing sets up an execution context and copies the items. Four architecture choices dominate:

1. Prefer fewer, fatter All-Items nodes over long transform chains. Every node→node hop re-copies all items (~0.05 ms/item per hop), so six chained Code/Set nodes cost ~7× one All-Items Code node doing the same steps. Consolidate the hot path. 2. Use Code "Run Once for All Items," not "Each Item" — ~0.02 ms/item vs ~0.6 ms/item (≈25–30×). A chain of Each-Item Code nodes is the worst case; the per-item tax multiplies by node count. 3. Maximize batchSize in SplitInBatches loops (see the Batch Processing pattern above) — iterations are the cost. 4. Don't micro-optimize expressions — complexity is free; node and iteration count are what you pay for.

But profile first. Most production workflows are I/O-bound — sequential HTTP / DB / Sheets calls (hundreds of ms each) dwarf all of the above. These rules matter when transform work is the floor, or when an anti-pattern (Each-Item Code, batchSize 1, long per-item chains) turns a cheap operation into a slow one. Below a few hundred items, none of it matters. The n8n Code JavaScript skill has the full measured model.

---

Integration-Specific Gotchas

Google Sheets

• NEVER use `append` on sheets with formula columns — it breaks formulas. Use Google Sheets API values.update (PUT) via HTTP Request node with a googleApi credential
• Write numbers, not strings for formula-dependent columns — string "4.98" breaks ADD() formulas. Use parseFloat() in a Code node
• Per-item execution trap: Google Sheets nodes execute once per input item. If you need a single bulk write, aggregate items into one in a Code node first
• UNFORMATTED_VALUE returns numbers, not text like "N/A" — filter explicitly in Code nodes

Google Drive

• `convertToGoogleDocument: true` creates a Google Doc (text), NOT a Google Sheet — to upload a CSV for download, omit this option entirely
• CSV download link format: https://drive.google.com/uc?id={fileId}&export=download — use instead of /view links

Bidirectional Threshold Checking

When comparing values (prices, quantities, metrics), always check both directions:

// ❌ Only catches increases
if (diff > threshold) { flag(); }

// ✅ Catches both spikes AND crashes — both are data-quality signals
if (Math.abs(diff) > threshold) { flag(); }

---

Common Gotchas

1. Webhook Data Structure

Problem: Can't access webhook payload data

Solution: Data is nested under $json.body

❌ {{$json.email}}
✅ {{$json.body.email}}

See: n8n Expression Syntax skill

2. Multiple Input Items

Problem: Node processes all input items, but I only want one

Solution: Use "Execute Once" mode or process first item only

{{$json[0].field}}  // First item only

3. Authentication Issues

Problem: API calls failing with 401/403

Solution:

• Configure credentials properly
• Use the "Credentials" section, not parameters
• Test credentials before workflow activation

4. Node Execution Order

Problem: Nodes executing in unexpected order

Solution: Check workflow settings → Execution Order

• v0: Top-to-bottom (legacy)
• v1: Connection-based (recommended)

5. Expression Errors

Problem: Expressions showing as literal text

Solution: Use {{}} around expressions

• See n8n Expression Syntax skill for details

---

Integration with Other Skills

These skills work together with Workflow Patterns:

n8n MCP Tools Expert - Use to:

• Find nodes for your pattern (search_nodes)
• Understand node operations (get_node)
• Create workflows (n8n_create_workflow)
• Deploy templates (n8n_deploy_template)
• Use tools_documentation({topic: "ai_agents_guide", depth: "full"}) for AI pattern guidance
• Manage data tables with n8n_manage_datatable

n8n Expression Syntax - Use to:

• Write expressions in transformation nodes
• Access webhook data correctly ({{$json.body.field}})
• Reference previous nodes ({{$node["Node Name"].json.field}})

n8n Node Configuration - Use to:

• Configure specific operations for pattern nodes
• Understand node-specific requirements

n8n Validation Expert - Use to:

• Validate workflow structure
• Fix validation errors
• Ensure workflow correctness before deployment

---

Pattern Statistics

Common workflow patterns:

Most Common Triggers: 1. Webhook - 35% 2. Schedule (periodic tasks) - 28% 3. Manual (testing/admin) - 22% 4. Service triggers (Slack, email, etc.) - 15%

Most Common Transformations: 1. Set (field mapping) - 68% 2. Code (custom logic) - 42% 3. IF (conditional routing) - 38% 4. Switch (multi-condition) - 18%

Most Common Outputs: 1. HTTP Request (APIs) - 45% 2. Slack - 32% 3. Database writes - 28% 4. Email - 24%

Average Workflow Complexity:

• Simple (3-5 nodes): 42%
• Medium (6-10 nodes): 38%
• Complex (11+ nodes): 20%

---

Quick Start Examples

Example 1: Simple Webhook → Slack

1. Webhook (path: "form-submit", POST)
2. Set (map form fields)
3. Slack (post message to #notifications)

Example 2: Scheduled Report

1. Schedule (daily at 9 AM)
2. HTTP Request (fetch analytics)
3. Code (aggregate data)
4. Email (send formatted report)
5. Error Trigger → Slack (notify on failure)

Example 3: Database Sync

1. Schedule (every 15 minutes)
2. Postgres (query new records)
3. IF (check if records exist)
4. MySQL (insert records)
5. Postgres (update sync timestamp)

Example 4: AI Assistant

1. Webhook (receive chat message)
2. AI Agent
   ├─ OpenAI Chat Model (ai_languageModel)
   ├─ HTTP Request Tool (ai_tool)
   ├─ Database Tool (ai_tool)
   └─ Window Buffer Memory (ai_memory)
3. Webhook Response (send AI reply)

Example 5: API Integration

1. Manual Trigger (for testing)
2. HTTP Request (GET /api/users)
3. Split In Batches (process 100 at a time)
4. Set (transform user data)
5. Postgres (upsert users)
6. Loop (back to step 3 until done)

---

Detailed Pattern Files

For comprehensive guidance on each pattern:

• [webhook_processing.md](webhook_processing.md) - Webhook patterns, data structure, response handling
• [http_api_integration.md](http_api_integration.md) - REST APIs, authentication, pagination, retries
• [database_operations.md](database_operations.md) - Queries, sync, transactions, batch processing
• [ai_agent_workflow.md](ai_agent_workflow.md) - AI agents, tools, memory, langchain nodes
• [scheduled_tasks.md](scheduled_tasks.md) - Cron schedules, reports, maintenance tasks

---

Real Template Examples

From n8n template library:

Template #2947: Weather to Slack

• Pattern: Scheduled Task
• Nodes: Schedule → HTTP Request (weather API) → Set → Slack
• Complexity: Simple (4 nodes)

Webhook Processing: Most common pattern

• Most common: Form submissions, payment webhooks, chat integrations

HTTP API: Common pattern

• Most common: Data fetching, third-party integrations

Database Operations: Common pattern

• Most common: ETL, data sync, backup workflows

AI Agents: Growing in usage

• Most common: Chatbots, content generation, data analysis

Use search_templates and get_template from n8n-mcp tools to find examples!

---

Best Practices

✅ Do

• Start with the simplest pattern that solves your problem
• Plan your workflow structure before building
• Use error handling on all workflows
• Test with sample data before activation
• Follow the workflow creation checklist
• Use descriptive node names
• Document complex workflows (notes field)
• Monitor workflow executions after deployment

❌ Don't

• Build workflows in one shot (iterate! avg 56s between edits)
• Skip validation before activation
• Ignore error scenarios
• Use complex patterns when simple ones suffice
• Hardcode credentials in parameters
• Forget to handle empty data cases
• Mix multiple patterns without clear boundaries
• Deploy without testing

---

Summary

Key Points: 1. 6 core patterns cover 90%+ of workflow use cases 2. Webhook processing is the most common pattern 3. Use the workflow creation checklist for every workflow 4. Plan pattern → Select nodes → Build → Validate → Deploy 5. Integrate with other skills for complete workflow development

Next Steps: 1. Identify your use case pattern 2. Read the detailed pattern file 3. Use n8n MCP Tools Expert to find nodes 4. Follow the workflow creation checklist 5. Use n8n Validation Expert to validate

Related Skills:

• n8n MCP Tools Expert - Find and configure nodes
• n8n Expression Syntax - Write expressions correctly
• n8n Validation Expert - Validate and fix errors
• n8n Node Configuration - Configure specific operations

AI Agent Workflow Pattern

Use Case: An AI agent with tool access, memory, and reasoning sits inside a larger workflow — trigger feeds it, it decides and acts, output flows on.

For agent design depth, use the `n8n-agents` skill. This file covers where an AI agent sits in a workflow's architecture (trigger → agent → output, the ai_* sub-node connection types). The n8n-agents skill owns the design rules: tool selection and $fromAI parameters, the system-prompt vs tool-description split, structured output with autoFix, memory and sessionId, human-in-the-loop review, RAG, and chat shell/core/sub-agent topologies. Start there when building or debugging an agent.

---

Pattern Structure

Trigger → AI Agent (Model + Tools + Memory + optional Output Parser) → [Process Response] → Output

Key Characteristic: AI-powered decision making with tool use. From the workflow angle, an agent is one node with a main input/output plus ai_* sub-node slots — it slots into the same trigger → process → deliver spine as every other pattern.

---

Core AI Connection Types

Agent workflows wire sub-nodes into the agent with dedicated ai_* connection types — not the regular main connection. This is the single most important architectural fact: a tool wired to main is invisible to the agent (and validate_workflow flags it as disconnected).

Wiring direction: a sub-node connects FROM itself TO the agent, and the connection lives on the sub-node keyed by its ai_* type. With n8n_update_partial_workflow you add each with an addConnection op using sourceOutput: "ai_tool" (or "ai_languageModel", etc.). Multiple tools all stack on the same ai_tool index 0.

---

Core Components

The agent has a main input (the user message / prompt) and up to four sub-node slots:

1. Trigger — Chat Trigger (chat UI/streaming), Webhook (API), Manual (testing), or Schedule (periodic). Feeds the agent's main input. 2. Language Model (ai_languageModel, required) — the reasoning engine. One chat-model sub-node; a second can be wired as a fallback. 3. Tools (ai_tool, optional but the whole point) — ANY node can be a tool. HTTP Request, a database node, a sub-workflow, Code, or a pre-built tool node connects via the ai_tool port and the agent calls it by name. 4. Memory (ai_memory, optional) — maintains conversation context across turns, keyed by a sessionKey. 5. Output Parser (ai_outputParser, optional) — forces structured JSON instead of free text.

Critical output fact: the AI Agent node puts its final answer in `$json.output` — not $json.text or $json.response. Downstream nodes reference {{ $json.output }}.

Fan-out tip: when several agents run in parallel (e.g. multiple research agents feeding one report), avoid funneling them into a Merge node — Merge combineAll does a cross-product and mishandles inputs arriving at different times (often yielding 0 output). Either have each agent deliver its own output directly, or collect same-shaped items with an Aggregate node followed by a Code node for formatting.

For the deep slot mechanics — tool types, $fromAI parameters, memory configuration, parser schemas — see n8n-agents.

---

Common Use Cases

Short architecture sketches. Each is a trigger → agent → output spine; the agent's sub-nodes are listed under it.

1. Conversational Chatbot

Webhook (chat message) → AI Agent → Webhook Response
                          ├─ Chat Model (ai_languageModel)
                          ├─ HTTP Request Tool — search knowledge base (ai_tool)
                          ├─ Database node — query orders (ai_tool)
                          └─ Window Buffer Memory, keyed on session_id (ai_memory)

2. Document Q&A (RAG)

Setup (run once):  Read Files → Text Splitter → Embeddings → Vector Store
Query (recurring): Webhook → AI Agent → Webhook Response
                              ├─ Chat Model (ai_languageModel)
                              ├─ Vector Store Tool — search docs (ai_tool)
                              └─ Buffer Memory (ai_memory)

3. Data Analysis Assistant

Webhook (data question) → AI Agent → Code (chart data) → Webhook Response
                          ├─ Chat Model (ai_languageModel)
                          ├─ Postgres node, read-only user (ai_tool)
                          └─ Code Tool — analysis (ai_tool)

4. Workflow Automation Agent

Slack (slash command) → AI Agent → Slack (status)
                        ├─ Chat Model (ai_languageModel)
                        ├─ HTTP Request Tool — GitHub API (ai_tool)
                        ├─ HTTP Request Tool — Deploy API (ai_tool)
                        └─ Postgres node — deployment logs (ai_tool)

5. Email Processing Agent

Email Trigger → AI Agent → Email (auto-response) → Slack (notify team)
                ├─ Chat Model (ai_languageModel)
                ├─ Vector Store Tool — similar tickets (ai_tool)
                └─ HTTP Request Tool — create Jira ticket (ai_tool)

For the content of these (tool descriptions, system prompts, schema design), see n8n-agents EXAMPLES.md.

---

What the deep design lives in n8n-agents

This file is the workflow-architecture view. The design depth below is owned by n8n-agents — go there, don't duplicate it here:

• Tool configuration (the four tool types, native vs .toolWorkflow vs HTTP Request Tool vs MCP Client, $fromAI() anatomy, tool names/descriptions as prompt) → n8n-agents TOOLS.md, and SUBWORKFLOW_AS_TOOL.md for wiring a sub-workflow as a tool.
• Memory configuration (buffer/window/postgres/redis, contextWindowLength, sessionId handling per trigger) → n8n-agents MEMORY.md.
• Agent vs chain vs classifier choice, prompt engineering, system-prompt vs tool-description split → n8n-agents SYSTEM_PROMPT.md (and the SKILL.md "Pick the right node" table).
• RAG chains, structured output, streaming, fallback models → n8n-agents RAG.md and STRUCTURED_OUTPUT.md.
• Human review / gating destructive tools → n8n-agents HUMAN_REVIEW.md.
• Error handling (tool failures, LLM API errors, retries, error workflows) → n8n-error-handling, plus the agent-specific notes in n8n-agents.
• Performance, security, testing, common gotchas → n8n-agents (anti-patterns table and quick-reference checklist) for the agent-specific ones; the workflow lifecycle (test → validate → activate) is in this skill's SKILL.md "Workflow lifecycle" section.

One workflow-architecture safety note worth restating here: any tool that fetches third-party content (HTTP Request, web search, MCP Client, scrapers) can return attacker-controlled text that reaches the agent's context — indirect prompt injection. If the agent can both read the internet AND take an action the user can't undo, put a guardrail (human review, read-only scopes) between them. The detail lives in n8n-agents HUMAN_REVIEW.md and the n8n-agents anti-patterns.

---

Checklist for AI Agent Workflows

Architecture-level checks (the design-level checklist lives in n8n-agents):

• [ ] Trigger feeds the agent's main input
• [ ] Language model wired via `ai_languageModel` (required)
• [ ] Tools wired via `ai_tool` ports — NOT main (a tool on main is disconnected from the agent)
• [ ] Memory wired via `ai_memory`, keyed on a stable sessionKey from the trigger — when conversation context is needed
• [ ] Output parser wired via `ai_outputParser` — when downstream needs strict JSON
• [ ] Downstream nodes read the response from `{{ $json.output }}`
• [ ] Parallel agents collected with Aggregate, not Merge combineAll
• [ ] Validated with validate_workflow (confirms sub-nodes sit on ai_*, not main)
• [ ] Tested and activated per the lifecycle (see SKILL.md "Workflow lifecycle" section)

---

Summary

Key Points: 1. An agent is one node with a main input/output plus ai_* sub-node slots — it fits the standard trigger → process → deliver spine. 2. 8 AI connection types — wire the model with ai_languageModel, tools with ai_tool, memory with ai_memory, parsers with ai_outputParser. Never main. 3. ANY node can be a tool — connect it via the ai_tool port. 4. The response is in `$json.output`. 5. For all design depth — tools, memory, prompts, structured output, RAG, human review, chat topologies — go to n8n-agents.

Pattern: Trigger → AI Agent (Model + Tools + Memory + optional Parser) → Output

Related:

• n8n-agents — the deep agent design guide (tools, memory, prompts, structured output, RAG, human review, chat topologies)
• webhook_processing.md — receiving chat messages
• http_api_integration.md — tools that call APIs
• database_operations.md — database tools for agents
• SKILL.md "Workflow lifecycle" section — test, validate, and activate the workflow
• n8n-error-handling — tool-failure and LLM-error handling

Database Operations Pattern

Use Case: Read, write, sync, and manage database data in workflows.

---

Pattern Structure

Trigger → [Query/Read] → [Transform] → [Write/Update] → [Verify/Log]

Key Characteristic: Data persistence and synchronization

---

Core Components

1. Trigger

Options:

• Schedule - Periodic sync/maintenance (most common)
• Webhook - Event-driven writes
• Manual - One-time operations

2. Database Read Nodes

Supported databases:

• Postgres
• MySQL
• MongoDB
• Microsoft SQL
• SQLite
• Redis
• And more via community nodes

3. Transform

Purpose: Map between different database schemas or formats

Typical nodes:

• Set - Field mapping
• Code - Complex transformations
• Merge - Combine data from multiple sources

4. Database Write Nodes

Operations:

• INSERT - Create new records
• UPDATE - Modify existing records
• UPSERT - Insert or update
• DELETE - Remove records

5. Verification

Purpose: Confirm operations succeeded

Methods:

• Query to verify records
• Count rows affected
• Log results

---

Common Use Cases

1. Data Synchronization

Flow: Schedule → Read Source DB → Transform → Write Target DB → Log

Example (Postgres to MySQL sync):

1. Schedule (every 15 minutes)
2. Postgres (SELECT * FROM users WHERE updated_at > {{$json.last_sync}})
3. IF (check if records exist)
4. Set (map Postgres schema to MySQL schema)
5. MySQL (INSERT or UPDATE users)
6. Postgres (UPDATE sync_log SET last_sync = NOW())
7. Slack (notify: "Synced X users")

Incremental sync query:

SELECT *
FROM users
WHERE updated_at > $1
ORDER BY updated_at ASC
LIMIT 1000

Parameters:

{
  "parameters": [
    "={{$node['Get Last Sync'].json.last_sync}}"
  ]
}

2. ETL (Extract, Transform, Load)

Flow: Extract from multiple sources → Transform → Load into warehouse

Example (Consolidate data):

1. Schedule (daily at 2 AM)
2. [Parallel branches]
   ├─ Postgres (SELECT orders)
   ├─ MySQL (SELECT customers)
   └─ MongoDB (SELECT products)
3. Merge (combine all data)
4. Code (transform to warehouse schema)
5. Postgres (warehouse - INSERT into fact_sales)
6. Email (send summary report)

3. Data Validation & Cleanup

Flow: Schedule → Query → Validate → Update/Delete invalid records

Example (Clean orphaned records):

1. Schedule (weekly)
2. Postgres (SELECT users WHERE email IS NULL OR email = '')
3. IF (invalid records exist)
4. Postgres (UPDATE users SET status='inactive' WHERE email IS NULL)
5. Postgres (DELETE FROM users WHERE created_at < NOW() - INTERVAL '1 year' AND status='inactive')
6. Slack (alert: "Cleaned X invalid records")

4. Backup & Archive

Flow: Schedule → Query → Export → Store

Example (Archive old records):

1. Schedule (monthly)
2. Postgres (SELECT * FROM orders WHERE created_at < NOW() - INTERVAL '2 years')
3. Code (convert to JSON)
4. Write File (save to archive.json)
5. Google Drive (upload archive)
6. Postgres (DELETE FROM orders WHERE created_at < NOW() - INTERVAL '2 years')

5. Real-time Data Updates

Flow: Webhook → Parse → Update Database

Example (Update user status):

1. Webhook (receive status update)
2. Postgres (UPDATE users SET status = {{$json.body.status}} WHERE id = {{$json.body.user_id}})
3. IF (rows affected > 0)
4. Redis (SET user:{{$json.body.user_id}}:status {{$json.body.status}})
5. Webhook Response ({"success": true})

---

Database Node Configuration

Postgres

SELECT Query

{
  operation: "executeQuery",
  query: "SELECT id, name, email FROM users WHERE created_at > $1 LIMIT $2",
  parameters: [
    "={{$json.since_date}}",
    "100"
  ]
}

INSERT

{
  operation: "insert",
  table: "users",
  columns: "id, name, email, created_at",
  values: [
    {
      id: "={{$json.id}}",
      name: "={{$json.name}}",
      email: "={{$json.email}}",
      created_at: "={{$now}}"
    }
  ]
}

UPDATE

{
  operation: "update",
  table: "users",
  updateKey: "id",
  columns: "name, email, updated_at",
  values: {
    id: "={{$json.id}}",
    name: "={{$json.name}}",
    email: "={{$json.email}}",
    updated_at: "={{$now}}"
  }
}

UPSERT (INSERT ... ON CONFLICT)

{
  operation: "executeQuery",
  query: `
    INSERT INTO users (id, name, email)
    VALUES ($1, $2, $3)
    ON CONFLICT (id)
    DO UPDATE SET name = $2, email = $3, updated_at = NOW()
  `,
  parameters: [
    "={{$json.id}}",
    "={{$json.name}}",
    "={{$json.email}}"
  ]
}

MySQL

SELECT with JOIN

{
  operation: "executeQuery",
  query: `
    SELECT u.id, u.name, o.order_id, o.total
    FROM users u
    LEFT JOIN orders o ON u.id = o.user_id
    WHERE u.created_at > ?
  `,
  parameters: [
    "={{$json.since_date}}"
  ]
}

Bulk INSERT

{
  operation: "insert",
  table: "orders",
  columns: "user_id, total, status",
  values: $json.orders  // Array of objects
}

MongoDB

Find Documents

{
  operation: "find",
  collection: "users",
  query: JSON.stringify({
    created_at: { $gt: new Date($json.since_date) },
    status: "active"
  }),
  limit: 100
}

Insert Document

{
  operation: "insert",
  collection: "users",
  document: JSON.stringify({
    name: $json.name,
    email: $json.email,
    created_at: new Date()
  })
}

Update Document

{
  operation: "update",
  collection: "users",
  query: JSON.stringify({ _id: $json.user_id }),
  update: JSON.stringify({
    $set: {
      status: $json.status,
      updated_at: new Date()
    }
  })
}

---

Batch Processing

Pattern 1: Split In Batches

Use when: Processing large datasets to avoid memory issues

Postgres (SELECT 10000 records)
  → Split In Batches (100 items per batch)
  → Transform
  → MySQL (write batch)
  → Loop (until all processed)

Pattern 2: Paginated Queries

Use when: Database has millions of records

Set (initialize: offset=0, limit=1000)
  → Loop Start
  → Postgres (SELECT * FROM large_table LIMIT {{$json.limit}} OFFSET {{$json.offset}})
  → IF (records returned)
    ├─ Process records
    ├─ Set (increment offset by 1000)
    └─ Loop back
  └─ [No records] → End

Query:

SELECT * FROM large_table
ORDER BY id
LIMIT $1 OFFSET $2

Pattern 3: Cursor-Based Pagination

Better performance for large datasets:

Set (initialize: last_id=0)
  → Loop Start
  → Postgres (SELECT * FROM table WHERE id > {{$json.last_id}} ORDER BY id LIMIT 1000)
  → IF (records returned)
    ├─ Process records
    ├─ Code (get max id from batch)
    └─ Loop back
  └─ [No records] → End

Query:

SELECT * FROM table
WHERE id > $1
ORDER BY id ASC
LIMIT 1000

---

Transaction Handling

Pattern 1: BEGIN/COMMIT/ROLLBACK

For databases that support transactions:

// Node 1: Begin Transaction
{
  operation: "executeQuery",
  query: "BEGIN"
}

// Node 2-N: Your operations
{
  operation: "executeQuery",
  query: "INSERT INTO ...",
  continueOnFail: true
}

// Node N+1: Commit or Rollback
{
  operation: "executeQuery",
  query: "={{$node['Operation'].json.error ? 'ROLLBACK' : 'COMMIT'}}"
}

Pattern 2: Atomic Operations

Use database features for atomicity:

-- Upsert example (atomic)
INSERT INTO inventory (product_id, quantity)
VALUES ($1, $2)
ON CONFLICT (product_id)
DO UPDATE SET quantity = inventory.quantity + $2

Pattern 3: Error Rollback

Manual rollback on error:

Try Operations:
  Postgres (INSERT orders)
  MySQL (INSERT order_items)

Error Trigger:
  Postgres (DELETE FROM orders WHERE id = {{$json.order_id}})
  MySQL (DELETE FROM order_items WHERE order_id = {{$json.order_id}})

---

Data Transformation

Schema Mapping

// Code node - map schemas
const sourceData = $input.all();

return sourceData.map(item => ({
  json: {
    // Source → Target mapping
    user_id: item.json.id,
    full_name: `${item.json.first_name} ${item.json.last_name}`,
    email_address: item.json.email,
    registration_date: new Date(item.json.created_at).toISOString(),
    // Computed fields
    is_premium: item.json.plan_type === 'pro',
    // Default values
    status: item.json.status || 'active'
  }
}));

Data Type Conversions

// Code node - convert data types
return $input.all().map(item => ({
  json: {
    // String to number
    user_id: parseInt(item.json.user_id),
    // String to date
    created_at: new Date(item.json.created_at),
    // Number to boolean
    is_active: item.json.active === 1,
    // JSON string to object
    metadata: JSON.parse(item.json.metadata || '{}'),
    // Null handling
    email: item.json.email || null
  }
}));

Aggregation

// Code node - aggregate data
const items = $input.all();

const summary = items.reduce((acc, item) => {
  const date = item.json.created_at.split('T')[0];
  if (!acc[date]) {
    acc[date] = { count: 0, total: 0 };
  }
  acc[date].count++;
  acc[date].total += item.json.amount;
  return acc;
}, {});

return Object.entries(summary).map(([date, data]) => ({
  json: {
    date,
    count: data.count,
    total: data.total,
    average: data.total / data.count
  }
}));

---

Performance Optimization

1. Use Indexes

Ensure database has proper indexes:

-- Add index for sync queries
CREATE INDEX idx_users_updated_at ON users(updated_at);

-- Add index for lookups
CREATE INDEX idx_orders_user_id ON orders(user_id);

2. Limit Result Sets

Always use LIMIT:

-- ✅ Good
SELECT * FROM large_table
WHERE created_at > $1
LIMIT 1000

-- ❌ Bad (unbounded)
SELECT * FROM large_table
WHERE created_at > $1

3. Use Prepared Statements

Parameterized queries are faster:

// ✅ Good - prepared statement
{
  query: "SELECT * FROM users WHERE id = $1",
  parameters: ["={{$json.id}}"]
}

// ❌ Bad - string concatenation
{
  query: "SELECT * FROM users WHERE id = '={{$json.id}}'"
}

4. Batch Writes

Write multiple records at once:

// ✅ Good - batch insert
{
  operation: "insert",
  table: "orders",
  values: $json.items  // Array of 100 items
}

// ❌ Bad - individual inserts in loop
// 100 separate INSERT statements

5. Connection Pooling

Configure in credentials:

{
  host: "db.example.com",
  database: "mydb",
  user: "user",
  password: "pass",
  // Connection pool settings
  min: 2,
  max: 10,
  idleTimeoutMillis: 30000
}

---

Error Handling

Pattern 1: Check Rows Affected

Database Operation (UPDATE users...)
  → IF ({{$json.rowsAffected === 0}})
    └─ Alert: "No rows updated - record not found"

Pattern 2: Constraint Violations

// Database operation with continueOnFail: true
{
  operation: "insert",
  continueOnFail: true
}

// Next node: Check for errors
IF ({{$json.error !== undefined}})
  → IF ({{$json.error.includes('duplicate key')}})
    └─ Log: "Record already exists - skipping"
  → ELSE
    └─ Alert: "Database error: {{$json.error}}"

Pattern 3: Rollback on Error

Try Operations:
  → Database Write 1
  → Database Write 2
  → Database Write 3

Error Trigger:
  → Rollback Operations
  → Alert Admin

---

Security Best Practices

1. Use Parameterized Queries (Prevent SQL Injection)

// ✅ SAFE - parameterized
{
  query: "SELECT * FROM users WHERE email = $1",
  parameters: ["={{$json.email}}"]
}

// ❌ DANGEROUS - SQL injection risk
{
  query: "SELECT * FROM users WHERE email = '={{$json.email}}'"
}

2. Least Privilege Access

Create dedicated workflow user:

-- ✅ Good - limited permissions
CREATE USER n8n_workflow WITH PASSWORD 'secure_password';
GRANT SELECT, INSERT, UPDATE ON orders TO n8n_workflow;
GRANT SELECT ON users TO n8n_workflow;

-- ❌ Bad - too much access
GRANT ALL PRIVILEGES TO n8n_workflow;

3. Validate Input Data

// Code node - validate before write
const email = $json.email;
const name = $json.name;

// Validation
if (!email || !email.includes('@')) {
  throw new Error('Invalid email address');
}

if (!name || name.length < 2) {
  throw new Error('Invalid name');
}

// Sanitization
return [{
  json: {
    email: email.toLowerCase().trim(),
    name: name.trim()
  }
}];

4. Encrypt Sensitive Data

// Code node - encrypt before storage
const crypto = require('crypto');

const algorithm = 'aes-256-cbc';
const key = Buffer.from($credentials.encryptionKey, 'hex');
const iv = crypto.randomBytes(16);

const cipher = crypto.createCipheriv(algorithm, key, iv);
let encrypted = cipher.update($json.sensitive_data, 'utf8', 'hex');
encrypted += cipher.final('hex');

return [{
  json: {
    encrypted_data: encrypted,
    iv: iv.toString('hex')
  }
}];

---

Common Gotchas

1. ❌ Wrong: Unbounded queries

SELECT * FROM large_table  -- Could return millions

✅ Correct: Use LIMIT

SELECT * FROM large_table
ORDER BY created_at DESC
LIMIT 1000

2. ❌ Wrong: String concatenation in queries

query: "SELECT * FROM users WHERE id = '{{$json.id}}'"

✅ Correct: Parameterized queries

query: "SELECT * FROM users WHERE id = $1",
parameters: ["={{$json.id}}"]

3. ❌ Wrong: No transaction for multi-step operations

INSERT into orders
INSERT into order_items  // Fails → orphaned order record

✅ Correct: Use transaction

BEGIN
INSERT into orders
INSERT into order_items
COMMIT (or ROLLBACK on error)

4. ❌ Wrong: Processing all items at once

SELECT 1000000 records → Process all → OOM error

✅ Correct: Batch processing

SELECT records → Split In Batches (1000) → Process → Loop

5. ❌ Wrong: Expecting output items from write operations

INSERT INTO table ... → Next node never executes (0 output items)

Database write operations (INSERT, UPDATE, DELETE) may return 0 result rows from the database engine — reliably so for raw query execution (e.g. executeQuery with an INSERT), while some database nodes return the affected rows instead. When 0 rows come back, n8n translates this to 0 output items, silently breaking any downstream chain.

✅ Correct: Set alwaysOutputData on write nodes

INSERT INTO table ... (alwaysOutputData: true) → Next node executes with 1 empty item

Set alwaysOutputData: true on any node that executes INSERT, UPDATE, or DELETE. This ensures at least 1 empty item ({json: {}}) flows downstream.

Tip: If downstream nodes need actual data (not the empty passthrough item),

reference the upstream node directly: $('DataSource Node').all()

---

Real Template Examples

From n8n template library (456 database templates):

Data Sync:

Schedule → Postgres (SELECT new records) → Transform → MySQL (INSERT)

ETL Pipeline:

Schedule → [Multiple DB reads] → Merge → Transform → Warehouse (INSERT)

Backup:

Schedule → Postgres (SELECT all) → JSON → Google Drive (upload)

Use search_templates({query: "database"}) to find more!

---

Checklist for Database Workflows

Planning

• [ ] Identify source and target databases
• [ ] Understand schema differences
• [ ] Plan transformation logic
• [ ] Consider batch size for large datasets
• [ ] Design error handling strategy

Implementation

• [ ] Use parameterized queries (never concatenate)
• [ ] Add LIMIT to all SELECT queries
• [ ] Use appropriate operation (INSERT/UPDATE/UPSERT)
• [ ] Configure credentials properly
• [ ] Test with small dataset first

Performance

• [ ] Add database indexes for queries
• [ ] Use batch operations
• [ ] Implement pagination for large datasets
• [ ] Configure connection pooling
• [ ] Monitor query execution times

Security

• [ ] Use parameterized queries (SQL injection prevention)
• [ ] Least privilege database user
• [ ] Validate and sanitize input
• [ ] Encrypt sensitive data
• [ ] Never log sensitive data

Reliability

• [ ] Add transaction handling if needed
• [ ] Check rows affected
• [ ] Handle constraint violations
• [ ] Implement retry logic
• [ ] Add Error Trigger workflow

---

Summary

Key Points: 1. Always use parameterized queries (prevent SQL injection) 2. Batch processing for large datasets 3. Transaction handling for multi-step operations 4. Limit result sets to avoid memory issues 5. Validate input data before writes

Pattern: Trigger → Query → Transform → Write → Verify

Related:

• http_api_integration.md - Fetching data to store in DB
• scheduled_tasks.md - Periodic database maintenance

HTTP API Integration Pattern

Use Case: Fetch data from REST APIs, transform it, and use it in workflows.

---

Pattern Structure

Trigger → HTTP Request → [Transform] → [Action] → [Error Handler]

Key Characteristic: External data fetching with error handling

---

Core Components

1. Trigger

Options:

• Schedule - Periodic fetching (most common)
• Webhook - Triggered by external event
• Manual - On-demand execution

2. HTTP Request Node

Purpose: Call external REST APIs

Configuration:

{
  method: "GET",                    // GET, POST, PUT, DELETE, PATCH
  url: "https://api.example.com/users",
  authentication: "predefinedCredentialType",
  sendQuery: true,
  queryParameters: {
    "page": "={{$json.page}}",
    "limit": "100"
  },
  sendHeaders: true,
  headerParameters: {
    "Accept": "application/json",
    "X-API-Version": "v1"
  }
}

3. Response Processing

Purpose: Extract and transform API response data

Typical flow:

HTTP Request → Code (parse) → Set (map fields) → Action

4. Action

Common actions:

• Store in database
• Send to another API
• Create notifications
• Update spreadsheet

5. Error Handler

Purpose: Handle API failures gracefully

Error Trigger Workflow:

Error Trigger → Log Error → Notify Admin → Retry Logic (optional)

---

Common Use Cases

1. Data Fetching & Storage

Flow: Schedule → HTTP Request → Transform → Database

Example (Fetch GitHub issues):

1. Schedule (every hour)
2. HTTP Request
   - Method: GET
   - URL: https://api.github.com/repos/owner/repo/issues
   - Auth: Bearer Token
   - Query: state=open
3. Code (filter by labels)
4. Set (map to database schema)
5. Postgres (upsert issues)

Response Handling:

// Code node - filter issues
const issues = $input.all();
return issues
  .filter(item => item.json.labels.some(l => l.name === 'bug'))
  .map(item => ({
    json: {
      id: item.json.id,
      title: item.json.title,
      created_at: item.json.created_at
    }
  }));

2. API to API Integration

Flow: Trigger → Fetch from API A → Transform → Send to API B

Example (Jira to Slack):

1. Schedule (every 15 minutes)
2. HTTP Request (GET Jira tickets updated today)
3. IF (check if tickets exist)
4. Set (format for Slack)
5. HTTP Request (POST to Slack webhook)

3. Data Enrichment

Flow: Trigger → Fetch base data → Call enrichment API → Combine → Store

Example (Enrich contacts with company data):

1. Postgres (SELECT new contacts)
2. Code (extract company domains)
3. HTTP Request (call Clearbit API for each domain)
4. Set (combine contact + company data)
5. Postgres (UPDATE contacts with enrichment)

4. Monitoring & Alerting

Flow: Schedule → Check API health → IF unhealthy → Alert

Example (API health check):

1. Schedule (every 5 minutes)
2. HTTP Request (GET /health endpoint)
3. IF (status !== 200 OR response time > 2000ms)
4. Slack (alert #ops-team)
5. PagerDuty (create incident)

5. Batch Processing

Flow: Trigger → Fetch large dataset → Split in Batches → Process → Loop

Example (Process all users):

1. Manual Trigger
2. HTTP Request (GET /api/users?limit=1000)
3. Split In Batches (100 items per batch)
4. HTTP Request (POST /api/process for each batch)
5. Wait (2 seconds between batches - rate limiting)
6. Loop (back to step 4 until all processed)

---

Authentication Methods

1. None (Public APIs)

{
  authentication: "none"
}

2. Bearer Token (Most Common)

Setup: Create credential

{
  authentication: "predefinedCredentialType",
  nodeCredentialType: "httpHeaderAuth",
  headerAuth: {
    name: "Authorization",
    value: "Bearer YOUR_TOKEN"
  }
}

Access in workflow:

{
  authentication: "predefinedCredentialType",
  nodeCredentialType: "httpHeaderAuth"
}

3. API Key (Header or Query)

Header auth:

{
  sendHeaders: true,
  headerParameters: {
    "X-API-Key": "={{$credentials.apiKey}}"
  }
}

Query auth:

{
  sendQuery: true,
  queryParameters: {
    "api_key": "={{$credentials.apiKey}}"
  }
}

4. Basic Auth

Setup: Create "Basic Auth" credential

{
  authentication: "predefinedCredentialType",
  nodeCredentialType: "httpBasicAuth"
}

5. OAuth2

Setup: Create OAuth2 credential with:

• Authorization URL
• Token URL
• Client ID
• Client Secret
• Scopes

{
  authentication: "predefinedCredentialType",
  nodeCredentialType: "oAuth2Api"
}

---

Handling API Responses

Success Response (200-299)

Default: Data flows to next node

Access response:

// Entire response
{{$json}}

// Specific fields
{{$json.data.id}}
{{$json.results[0].name}}

Pagination

Pattern 1: Offset-based

1. Set (initialize: page=1, has_more=true)
2. HTTP Request (GET /api/items?page={{$json.page}})
3. Code (check if more pages)
4. IF (has_more === true)
   └→ Set (increment page) → Loop to step 2

Code node (check pagination):

const items = $input.first().json;
const currentPage = $json.page || 1;

return [{
  json: {
    items: items.results,
    page: currentPage + 1,
    has_more: items.next !== null
  }
}];

Pattern 2: Cursor-based

1. HTTP Request (GET /api/items)
2. Code (extract next_cursor)
3. IF (next_cursor exists)
   └→ Set (cursor={{$json.next_cursor}}) → Loop to step 1

Pattern 3: Link Header

// Code node - parse Link header
const linkHeader = $input.first().json.headers['link'];
const hasNext = linkHeader && linkHeader.includes('rel="next"');

return [{
  json: {
    items: $input.first().json.body,
    has_next: hasNext,
    next_url: hasNext ? parseNextUrl(linkHeader) : null
  }
}];

Error Responses (400-599)

Configure HTTP Request:

{
  continueOnFail: true,  // Don't stop workflow on error
  ignoreResponseCode: true  // Get response even on error
}

Handle errors:

HTTP Request (continueOnFail: true)
  → IF (check error)
    ├─ [Success Path]
    └─ [Error Path] → Log → Retry or Alert

IF condition:

{{$json.error}} is empty
// OR
{{$json.statusCode}} < 400

---

Rate Limiting

Pattern 1: Wait Between Requests

Split In Batches (1 item per batch)
  → HTTP Request
  → Wait (1 second)
  → Loop

Pattern 2: Exponential Backoff

// Code node
const maxRetries = 3;
let retryCount = $json.retryCount || 0;

if ($json.error && retryCount < maxRetries) {
  const delay = Math.pow(2, retryCount) * 1000; // 1s, 2s, 4s

  return [{
    json: {
      ...$json,
      retryCount: retryCount + 1,
      waitTime: delay
    }
  }];
}

Pattern 3: Respect Rate Limit Headers

// Code node - check rate limit
const headers = $input.first().json.headers;
const remaining = parseInt(headers['x-ratelimit-remaining'] || '999');
const resetTime = parseInt(headers['x-ratelimit-reset'] || '0');

if (remaining < 10) {
  const now = Math.floor(Date.now() / 1000);
  const waitSeconds = resetTime - now;

  return [{
    json: {
      shouldWait: true,
      waitSeconds: Math.max(waitSeconds, 0)
    }
  }];
}

return [{ json: { shouldWait: false } }];

---

Request Configuration

GET Request

{
  method: "GET",
  url: "https://api.example.com/users",
  sendQuery: true,
  queryParameters: {
    "page": "1",
    "limit": "100",
    "filter": "active"
  }
}

POST Request (JSON Body)

{
  method: "POST",
  url: "https://api.example.com/users",
  sendBody: true,
  bodyParametersJson: JSON.stringify({
    name: "={{$json.name}}",
    email: "={{$json.email}}",
    role: "user"
  })
}

POST Request (Form Data)

{
  method: "POST",
  url: "https://api.example.com/upload",
  sendBody: true,
  bodyParametersUi: {
    parameter: [
      { name: "file", value: "={{$json.fileData}}" },
      { name: "filename", value: "={{$json.filename}}" }
    ]
  },
  sendHeaders: true,
  headerParameters: {
    "Content-Type": "multipart/form-data"
  }
}

PUT/PATCH Request (Update)

{
  method: "PATCH",
  url: "https://api.example.com/users/={{$json.userId}}",
  sendBody: true,
  bodyParametersJson: JSON.stringify({
    status: "active",
    last_updated: "={{$now}}"
  })
}

DELETE Request

{
  method: "DELETE",
  url: "https://api.example.com/users/={{$json.userId}}"
}

---

Error Handling Patterns

Pattern 1: Retry on Failure

HTTP Request (continueOnFail: true)
  → IF (error occurred)
    └→ Wait (5 seconds)
    └→ HTTP Request (retry)

Pattern 2: Fallback API

HTTP Request (Primary API, continueOnFail: true)
  → IF (failed)
    └→ HTTP Request (Fallback API)

Pattern 3: Error Trigger Workflow

Main Workflow:

HTTP Request → Process Data

Error Workflow:

Error Trigger
  → Set (extract error details)
  → Slack (alert team)
  → Database (log error for analysis)

Pattern 4: Circuit Breaker

// Code node - circuit breaker logic
const failures = $json.recentFailures || 0;
const threshold = 5;

if (failures >= threshold) {
  throw new Error('Circuit breaker open - too many failures');
}

return [{ json: { canProceed: true } }];

---

Response Transformation

Extract Nested Data

// Code node
const response = $input.first().json;

return response.data.items.map(item => ({
  json: {
    id: item.id,
    name: item.attributes.name,
    email: item.attributes.contact.email
  }
}));

Flatten Arrays

// Code node - flatten nested array
const items = $input.all();
const flattened = items.flatMap(item =>
  item.json.results.map(result => ({
    json: {
      parent_id: item.json.id,
      ...result
    }
  }))
);

return flattened;

Combine Multiple API Responses

HTTP Request 1 (users)
  → Set (store users)
  → HTTP Request 2 (orders for each user)
  → Merge (combine users + orders)

---

Testing & Debugging

1. Test with Manual Trigger

Replace Schedule with Manual Trigger for testing

2. Use Postman/Insomnia First

• Test API outside n8n
• Understand response structure
• Verify authentication

3. Log Responses

// Code node - log for debugging
console.log('API Response:', JSON.stringify($input.first().json, null, 2));
return $input.all();

4. Check Execution Data

• View node output in n8n UI
• Check headers, body, status code
• Verify data structure

5. Use Binary Data Properly

For file downloads:

{
  method: "GET",
  url: "https://api.example.com/download/file.pdf",
  responseFormat: "file",  // Important for binary data
  outputPropertyName: "data"
}

---

Performance Optimization

1. Parallel Requests

Use Split In Batches with multiple items:

Set (create array of IDs)
  → Split In Batches (10 items per batch)
  → HTTP Request (processes all 10 in parallel)
  → Loop

2. Caching

IF (check cache exists)
  ├─ [Cache Hit] → Use cached data
  └─ [Cache Miss] → HTTP Request → Store in cache

3. Conditional Fetching

Only fetch if data changed:

HTTP Request (GET with If-Modified-Since header)
  → IF (status === 304)
    └─ Use existing data
  → IF (status === 200)
    └─ Process new data

4. Batch API Calls

If API supports batch operations:

{
  method: "POST",
  url: "https://api.example.com/batch",
  bodyParametersJson: JSON.stringify({
    requests: $json.items.map(item => ({
      method: "GET",
      url: `/users/${item.id}`
    }))
  })
}

---

Common Gotchas

1. ❌ Wrong: Hardcoded URLs

url: "https://api.example.com/prod/users"

✅ Correct: Use environment variables

url: "={{$env.API_BASE_URL}}/users"

2. ❌ Wrong: Credentials in parameters

headerParameters: {
  "Authorization": "Bearer sk-abc123xyz"  // ❌ Exposed!
}

✅ Correct: Use credentials system

authentication: "predefinedCredentialType",
nodeCredentialType: "httpHeaderAuth"

3. ❌ Wrong: No error handling

HTTP Request → Process (fails if API down)

✅ Correct: Handle errors

HTTP Request (continueOnFail: true) → IF (error) → Handle

4. ❌ Wrong: Blocking on large responses

Processing 10,000 items synchronously

✅ Correct: Use batching

Split In Batches (100 items) → Process → Loop

---

Real Template Examples

From n8n template library (892 API integration templates):

GitHub to Notion:

Schedule → HTTP Request (GitHub API) → Transform → HTTP Request (Notion API)

Weather to Slack:

Schedule → HTTP Request (Weather API) → Set (format) → Slack

CRM Sync:

Schedule → HTTP Request (CRM A) → Transform → HTTP Request (CRM B)

Use search_templates({query: "http api"}) to find more!

---

Checklist for API Integration

Planning

• [ ] Test API with Postman/curl first
• [ ] Understand response structure
• [ ] Check rate limits
• [ ] Review authentication method
• [ ] Plan error handling

Implementation

• [ ] Use credentials (never hardcode)
• [ ] Configure proper HTTP method
• [ ] Set correct headers (Content-Type, Accept)
• [ ] Handle pagination if needed
• [ ] Add query parameters properly

Error Handling

• [ ] Set continueOnFail: true if needed
• [ ] Check response status codes
• [ ] Implement retry logic
• [ ] Add Error Trigger workflow
• [ ] Alert on failures

Performance

• [ ] Use batching for large datasets
• [ ] Add rate limiting if needed
• [ ] Consider caching
• [ ] Test with production load

Security

• [ ] Use HTTPS only
• [ ] Store secrets in credentials
• [ ] Validate API responses
• [ ] Use environment variables

---

Summary

Key Points: 1. Authentication via credentials system (never hardcode) 2. Error handling is critical (continueOnFail + IF checks) 3. Pagination for large datasets 4. Rate limiting to respect API limits 5. Transform responses to match your needs

Pattern: Trigger → HTTP Request → Transform → Action → Error Handler

Related:

• webhook_processing.md - Receiving HTTP requests
• database_operations.md - Storing API data

n8n Workflow Patterns

Proven architectural patterns for building n8n workflows.

---

Purpose

Teaches architectural patterns for building n8n workflows. Provides structure, best practices, and proven approaches for common use cases.

Activates On

• build workflow
• workflow pattern
• workflow architecture
• workflow structure
• webhook processing
• http api
• api integration
• database sync
• ai agent
• chatbot
• scheduled task
• automation pattern

File Count

7 files

Priority

HIGH - Addresses 813 webhook searches (most common use case)

Dependencies

n8n-mcp tools:

• search_nodes (find nodes for patterns)
• get_node (understand node operations)
• search_templates (find example workflows)
• tools_documentation with topic "ai_agents_guide" (AI pattern guidance)

Related skills:

• n8n MCP Tools Expert (find and configure nodes)
• n8n Expression Syntax (write expressions in patterns)
• n8n Node Configuration (configure pattern nodes)
• n8n Validation Expert (validate pattern implementations)

Coverage

The 5 Core Patterns

1. Webhook Processing (Most Common - 813 searches)

• Receive HTTP requests → Process → Respond
• Critical gotcha: Data under $json.body
• Authentication, validation, error handling

2. HTTP API Integration (892 templates)

• Fetch from REST APIs → Transform → Store/Use
• Authentication methods, pagination, rate limiting
• Error handling and retries

3. Database Operations (456 templates)

• Read/Write/Sync database data
• Batch processing, transactions, performance
• Security: parameterized queries, read-only access

4. AI Agent Workflow (234 templates, 270 AI nodes)

• AI agents with tool access and memory
• 8 AI connection types
• ANY node can be an AI tool

5. Scheduled Tasks (28% of all workflows)

• Recurring automation workflows
• Cron schedules, timezone handling
• Monitoring and error handling

Cross-Cutting Concerns

• Data flow patterns (linear, branching, parallel, loops)
• Error handling strategies
• Performance optimization
• Security best practices
• Testing approaches
• Monitoring and logging

Evaluations

5 scenarios (100% coverage expected): 1. eval-001: Webhook workflow structure 2. eval-002: HTTP API integration pattern 3. eval-003: Database sync pattern 4. eval-004: AI agent workflow with tools 5. eval-005: Scheduled report generation

Key Features

✅ 5 Proven Patterns: Webhook, HTTP API, Database, AI Agent, Scheduled tasks ✅ Complete Examples: Working workflow configurations for each pattern ✅ Best Practices: Proven approaches from real-world n8n usage ✅ Common Gotchas: Documented mistakes and their fixes ✅ Integration Guide: How patterns work with other skills ✅ Template Examples: Real examples from 2,653+ n8n templates

Files

• SKILL.md - Pattern overview, selection guide, checklist
• webhook_processing.md - Webhook patterns, data structure, auth
• http_api_integration.md - REST APIs, pagination, rate limiting
• database_operations.md - DB operations, batch processing, security
• ai_agent_workflow.md - AI agents, tools, memory, 8 connection types
• scheduled_tasks.md - Cron schedules, timezone, monitoring
• README.md (this file) - Skill metadata

Success Metrics

Expected outcomes:

• Users select appropriate pattern for their use case
• Workflows follow proven structural patterns
• Common gotchas avoided (webhook $json.body, SQL injection, etc.)
• Proper error handling implemented
• Security best practices followed

Pattern Selection Stats

Common workflow composition:

Trigger Distribution:

• Webhook: 35% (most common)
• Schedule: 28%
• Manual: 22%
• Service triggers: 15%

Transformation Nodes:

• Set: 68%
• Code: 42%
• IF: 38%
• Switch: 18%

Output Channels:

• HTTP Request: 45%
• Slack: 32%
• Database: 28%
• Email: 24%

Complexity:

• Simple (3-5 nodes): 42%
• Medium (6-10 nodes): 38%
• Complex (11+ nodes): 20%

Critical Insights

Webhook Processing:

• 813 searches (most common use case!)
• #1 gotcha: Data under $json.body (not $json directly)
• Must choose response mode: onReceived vs lastNode

API Integration:

• Authentication via credentials (never hardcode!)
• Pagination essential for large datasets
• Rate limiting prevents API bans
• continueOnFail: true for error handling

Database Operations:

• Always use parameterized queries (SQL injection prevention)
• Batch processing for large datasets
• Read-only access for AI tools
• Transaction handling for multi-step operations

AI Agents:

• 8 AI connection types (ai_languageModel, ai_tool, ai_memory, etc.)
• ANY node can be an AI tool (connect to ai_tool port)
• Memory essential for conversations (Window Buffer recommended)
• Tool descriptions critical (AI uses them to decide when to call)

Scheduled Tasks:

• Set workflow timezone explicitly (DST handling)
• Prevent overlapping executions (use locks)
• Error Trigger workflow for alerts
• Batch processing for large data

Workflow Creation Checklist

Every pattern follows this checklist:

Planning Phase

• [ ] Identify the pattern (webhook, API, database, AI, scheduled)
• [ ] List required nodes (use search_nodes)
• [ ] Understand data flow (input → transform → output)
• [ ] Plan error handling strategy

Implementation Phase

• [ ] Create workflow with appropriate trigger
• [ ] Add data source nodes
• [ ] Configure authentication/credentials
• [ ] Add transformation nodes (Set, Code, IF)
• [ ] Add output/action nodes
• [ ] Configure error handling

Validation Phase

• [ ] Validate each node configuration
• [ ] Validate complete workflow
• [ ] Test with sample data
• [ ] Handle edge cases

Deployment Phase

• [ ] Review workflow settings
• [ ] Activate workflow
• [ ] Monitor first executions
• [ ] Document workflow

Real Template Examples

Weather to Slack (Template #2947):

Schedule (daily 8 AM) → HTTP Request (weather) → Set → Slack

Webhook Processing: 1,085 templates HTTP API Integration: 892 templates Database Operations: 456 templates AI Workflows: 234 templates

Use search_templates to find examples for your use case!

Integration with Other Skills

Pattern Selection (this skill): 1. Identify use case 2. Select appropriate pattern 3. Follow pattern structure

Node Discovery (n8n MCP Tools Expert): 4. Find nodes for pattern (search_nodes) 5. Understand node operations (get_node)

Implementation (n8n Expression Syntax + Node Configuration): 6. Write expressions ({{$json.body.field}}) 7. Configure nodes properly

Validation (n8n Validation Expert): 8. Validate workflow structure 9. Fix validation errors

Last Updated

2025-10-20

---

Part of: n8n-skills repository Conceived by: Romuald Członkowski - www.aiadvisors.pl/en

Scheduled Tasks Pattern

Use Case: Recurring automation workflows that run automatically on a schedule.

---

Pattern Structure

Schedule Trigger → [Fetch Data] → [Process] → [Deliver] → [Log/Notify]

Key Characteristic: Time-based automated execution

---

Core Components

1. Schedule Trigger

Purpose: Execute workflow at specified times

Modes:

• Interval - Every X minutes/hours/days
• Cron - Specific times (advanced)
• Days & Hours - Simple recurring schedule

2. Data Source

Common sources:

• HTTP Request (APIs)
• Database queries
• File reads
• Service-specific nodes

3. Processing

Typical operations:

• Filter/transform data
• Aggregate statistics
• Generate reports
• Check conditions

4. Delivery

Output channels:

• Email
• Slack/Discord/Teams
• File storage
• Database writes

5. Logging

Purpose: Track execution history

Methods:

• Database log entries
• File append
• Monitoring service

---

Schedule Configuration

Interval Mode

Best for: Simple recurring tasks

Examples:

// Every 15 minutes
{
  mode: "interval",
  interval: 15,
  unit: "minutes"
}

// Every 2 hours
{
  mode: "interval",
  interval: 2,
  unit: "hours"
}

// Every day at midnight
{
  mode: "interval",
  interval: 1,
  unit: "days"
}

Days & Hours Mode

Best for: Specific days and times

Examples:

// Weekdays at 9 AM
{
  mode: "daysAndHours",
  days: ["monday", "tuesday", "wednesday", "thursday", "friday"],
  hour: 9,
  minute: 0
}

// Every Monday at 6 PM
{
  mode: "daysAndHours",
  days: ["monday"],
  hour: 18,
  minute: 0
}

Timezone Gotcha (applies to all modes)

triggerAtHour / hour values use the instance timezone, not UTC. n8n resolves it from the GENERIC_TIMEZONE env var (or the workflow's timezone setting); when neither is set, it falls back to the host system timezone. A trigger set to hour 21 on a server in America/Edmonton fires at 9 PM MST, not 21:00 UTC. Always confirm the instance timezone before scheduling, or set the workflow timezone explicitly.

Cron Mode (Advanced)

Best for: Complex schedules

Examples:

// Every weekday at 9 AM
{
  mode: "cron",
  expression: "0 9 * * 1-5"
}

// First day of every month at midnight
{
  mode: "cron",
  expression: "0 0 1 * *"
}

// Every 15 minutes during business hours (9 AM - 5 PM) on weekdays
{
  mode: "cron",
  expression: "*/15 9-17 * * 1-5"
}

Cron format: minute hour day month weekday

• * = any value
• */15 = every 15 units
• 1-5 = range (Monday-Friday)
• 1,15 = specific values

Cron examples:

0 */6 * * *      Every 6 hours
0 9,17 * * *     At 9 AM and 5 PM daily
0 0 * * 0        Every Sunday at midnight
*/30 * * * *     Every 30 minutes
0 0 1,15 * *     1st and 15th of each month

---

Common Use Cases

1. Daily Reports

Flow: Schedule → Fetch data → Aggregate → Format → Email

Example (Sales report):

1. Schedule (daily at 9 AM)

2. Postgres (query yesterday's sales)
   SELECT date, SUM(amount) as total, COUNT(*) as orders
   FROM orders
   WHERE date = CURRENT_DATE - INTERVAL '1 day'
   GROUP BY date

3. Code (calculate metrics)
   - Total revenue
   - Order count
   - Average order value
   - Comparison to previous day

4. Set (format email body)
   Subject: Daily Sales Report - {{$json.date}}
   Body: Formatted HTML with metrics

5. Email (send to team@company.com)

6. Slack (post summary to #sales)

2. Data Synchronization

Flow: Schedule → Fetch from source → Transform → Write to target

Example (CRM to data warehouse sync):

1. Schedule (every hour)

2. Set (store last sync time)
   SELECT MAX(synced_at) FROM sync_log

3. HTTP Request (fetch new CRM contacts since last sync)
   GET /api/contacts?updated_since={{$json.last_sync}}

4. IF (check if new records exist)

5. Set (transform CRM schema to warehouse schema)

6. Postgres (warehouse - INSERT new contacts)

7. Postgres (UPDATE sync_log SET synced_at = NOW())

8. IF (error occurred)
   └─ Slack (alert #data-team)

3. Monitoring & Health Checks

Flow: Schedule → Check endpoints → Alert if down

Example (Website uptime monitor):

1. Schedule (every 5 minutes)

2. HTTP Request (GET https://example.com/health)
   - timeout: 10 seconds
   - continueOnFail: true

3. IF (status !== 200 OR response_time > 2000ms)

4. Redis (check alert cooldown - don't spam)
   - Key: alert:website_down
   - TTL: 30 minutes

5. IF (no recent alert sent)

6. [Alert Actions]
   ├─ Slack (notify #ops-team)
   ├─ PagerDuty (create incident)
   ├─ Email (alert@company.com)
   └─ Redis (set alert cooldown)

7. Postgres (log uptime check result)

4. Cleanup & Maintenance

Flow: Schedule → Find old data → Archive/Delete → Report

Example (Database cleanup):

1. Schedule (weekly on Sunday at 2 AM)

2. Postgres (find old records)
   SELECT * FROM logs
   WHERE created_at < NOW() - INTERVAL '90 days'
   LIMIT 10000

3. IF (records exist)

4. Code (export to JSON for archive)

5. Google Drive (upload archive file)
   - Filename: logs_archive_{{$now.format('YYYY-MM-DD')}}.json

6. Postgres (DELETE archived records)
   DELETE FROM logs
   WHERE id IN ({{$json.archived_ids}})

7. Slack (report: "Archived X records, deleted Y records")

5. Data Enrichment

Flow: Schedule → Find incomplete records → Enrich → Update

Example (Enrich contacts with company data):

1. Schedule (nightly at 3 AM)

2. Postgres (find contacts without company data)
   SELECT id, email, domain FROM contacts
   WHERE company_name IS NULL
   AND created_at > NOW() - INTERVAL '7 days'
   LIMIT 100

3. Split In Batches (10 contacts per batch)

4. HTTP Request (call Clearbit enrichment API)
   - For each contact domain
   - Rate limit: wait 1 second between batches

5. Set (map API response to database schema)

6. Postgres (UPDATE contacts with company data)

7. Wait (1 second - rate limiting)

8. Loop (back to step 4 until all batches processed)

9. Email (summary: "Enriched X contacts")

6. Backup Automation

Flow: Schedule → Export data → Compress → Store → Verify

Example (Database backup):

1. Schedule (daily at 2 AM)

2. Code (execute pg_dump)
   const { exec } = require('child_process');
   exec('pg_dump -h db.example.com mydb > backup.sql')

3. Code (compress backup)
   const zlib = require('zlib');
   // Compress backup.sql to backup.sql.gz

4. AWS S3 (upload compressed backup)
   - Bucket: backups
   - Key: db/backup-{{$now.format('YYYY-MM-DD')}}.sql.gz

5. AWS S3 (list old backups)
   - Keep last 30 days only

6. AWS S3 (delete old backups)

7. IF (error occurred)
   ├─ PagerDuty (critical alert)
   └─ Email (backup failed!)
   ELSE
   └─ Slack (#devops: "✅ Backup completed")

7. Content Publishing

Flow: Schedule → Fetch content → Format → Publish

Example (Automated social media posts):

1. Schedule (every 3 hours during business hours)
   - Cron: 0 9,12,15,18 * * 1-5

2. Google Sheets (read content queue)
   - Sheet: "Scheduled Posts"
   - Filter: status=pending AND publish_time <= NOW()

3. IF (posts available)

4. HTTP Request (shorten URLs in post)

5. HTTP Request (POST to Twitter API)

6. HTTP Request (POST to LinkedIn API)

7. Google Sheets (update status=published)

8. Slack (notify #marketing: "Posted: {{$json.title}}")

---

Timezone Considerations

Set Workflow Timezone

// In workflow settings
{
  timezone: "America/New_York"  // EST/EDT
}

Common Timezones

America/New_York    - Eastern (US)
America/Chicago     - Central (US)
America/Denver      - Mountain (US)
America/Los_Angeles - Pacific (US)
Europe/London       - GMT/BST
Europe/Paris        - CET/CEST
Asia/Tokyo          - JST
Australia/Sydney    - AEDT
UTC                 - Universal Time

Handle Daylight Saving

Best practice: Use timezone-aware scheduling

// ❌ Bad: UTC schedule for "9 AM local"
// Will be off by 1 hour during DST transitions

// ✅ Good: Set workflow timezone
{
  timezone: "America/New_York",
  schedule: {
    mode: "daysAndHours",
    hour: 9  // Always 9 AM Eastern, regardless of DST
  }
}

---

Error Handling

Pattern 1: Error Trigger Workflow

Main workflow: Normal execution Error workflow: Alerts and recovery

Main:

Schedule → Fetch → Process → Deliver

Error:

Error Trigger (for main workflow)
  → Set (extract error details)
  → Slack (#ops-team: "❌ Scheduled job failed")
  → Email (admin alert)
  → Postgres (log error for analysis)

Pattern 2: Retry with Backoff

Schedule → HTTP Request (continueOnFail: true)
  → IF (error)
    ├─ Wait (5 minutes)
    ├─ HTTP Request (retry 1)
    └─ IF (still error)
      ├─ Wait (15 minutes)
      ├─ HTTP Request (retry 2)
      └─ IF (still error)
        └─ Alert admin

Pattern 3: Partial Failure Handling

Schedule → Split In Batches
  → Process (continueOnFail: true)
  → Code (track successes and failures)
  → Report:
    "✅ Processed: 95/100"
    "❌ Failed: 5/100"

---

Performance Optimization

1. Batch Processing

For large datasets:

Schedule → Query (LIMIT 10000)
  → Split In Batches (100 items)
  → Process batch
  → Loop

2. Parallel Processing

When operations are independent:

Schedule
  ├─ [Branch 1: Update DB]
  ├─ [Branch 2: Send emails]
  └─ [Branch 3: Generate report]
  → Merge (wait for all) → Final notification

3. Skip if Already Running

Prevent overlapping executions:

Schedule → Redis (check lock)
  → IF (lock exists)
    └─ End (skip this execution)
  → ELSE
    ├─ Redis (set lock, TTL 30 min)
    ├─ [Execute workflow]
    └─ Redis (delete lock)

4. Early Exit on No Data

Don't waste time if nothing to process:

Schedule → Query (check if work exists)
  → IF (no results)
    └─ End workflow (exit early)
  → ELSE
    └─ Process data

---

Monitoring & Logging

Pattern 1: Execution Log Table

CREATE TABLE workflow_executions (
  id SERIAL PRIMARY KEY,
  workflow_name VARCHAR(255),
  started_at TIMESTAMP,
  completed_at TIMESTAMP,
  status VARCHAR(50),
  records_processed INT,
  error_message TEXT
);

Log execution:

Schedule
  → Set (record start)
  → [Workflow logic]
  → Postgres (INSERT execution log)

Pattern 2: Metrics Collection

Schedule → [Execute]
  → Code (calculate metrics)
    - Duration
    - Records processed
    - Success rate
  → HTTP Request (send to monitoring system)
    - Datadog, Prometheus, etc.

Pattern 3: Summary Notifications

Daily/weekly execution summaries:

Schedule (daily at 6 PM) → Query execution logs
  → Code (aggregate today's executions)
  → Email (summary report)
    "Today's Workflow Executions:
     - 24/24 successful
     - 0 failures
     - Avg duration: 2.3 min"

---

Testing Scheduled Workflows

1. Use Manual Trigger for Testing

Development pattern:

Manual Trigger (for testing)
  → [Same workflow logic]
  → [Outputs]

// Once tested, replace with Schedule Trigger

2. Test with Different Times

// Code node - simulate different times
const testTime = new Date('2024-01-15T09:00:00Z');
return [{ json: { currentTime: testTime } }];

3. Dry Run Mode

Schedule → Set (dryRun: true)
  → IF (dryRun)
    └─ Log what would happen (don't execute)
  → ELSE
    └─ Execute normally

4. Shorter Interval for Testing

// Testing: every 1 minute
{
  mode: "interval",
  interval: 1,
  unit: "minutes"
}

// Production: every 1 hour
{
  mode: "interval",
  interval: 1,
  unit: "hours"
}

---

Common Gotchas

1. ❌ Wrong: Ignoring timezone

Schedule (9 AM)  // 9 AM in which timezone?

✅ Correct: Set workflow timezone

// Workflow settings
{
  timezone: "America/New_York"
}

2. ❌ Wrong: Overlapping executions

Schedule (every 5 min) → Long-running task (10 min)
// Two executions running simultaneously!

✅ Correct: Add execution lock

Schedule → Redis (check lock)
  → IF (locked) → Skip
  → ELSE → Execute

3. ❌ Wrong: No error handling

Schedule → API call → Process (fails silently)

✅ Correct: Add error workflow

Main: Schedule → Execute
Error: Error Trigger → Alert

4. ❌ Wrong: Processing all data at once

Schedule → SELECT 1000000 records → Process (OOM)

✅ Correct: Batch processing

Schedule → SELECT with pagination → Split In Batches → Process

5. ❌ Wrong: Hardcoded dates

query: "SELECT * FROM orders WHERE date = '2024-01-15'"

✅ Correct: Dynamic dates

query: "SELECT * FROM orders WHERE date = CURRENT_DATE - INTERVAL '1 day'"

---

Real Template Examples

From n8n template library:

Template #2947 (Weather to Slack):

Schedule (daily 8 AM)
  → HTTP Request (weather API)
  → Set (format message)
  → Slack (post to #general)

Daily backup:

Schedule (nightly 2 AM)
  → Postgres (export data)
  → Google Drive (upload)
  → Email (confirmation)

Monitoring:

Schedule (every 5 min)
  → HTTP Request (health check)
  → IF (down) → PagerDuty alert

Use search_templates({query: "schedule"}) to find more!

---

Checklist for Scheduled Workflows

Planning

• [ ] Define schedule frequency (interval, cron, days & hours)
• [ ] Set workflow timezone
• [ ] Estimate execution duration
• [ ] Plan for failures and retries
• [ ] Consider timezone and DST

Implementation

• [ ] Configure Schedule Trigger
• [ ] Set workflow timezone in settings
• [ ] Add early exit for no-op cases
• [ ] Implement batch processing for large data
• [ ] Add execution logging

Error Handling

• [ ] Create Error Trigger workflow
• [ ] Implement retry logic
• [ ] Add alert notifications
• [ ] Log errors for analysis
• [ ] Handle partial failures gracefully

Monitoring

• [ ] Log each execution (start, end, status)
• [ ] Track metrics (duration, records, success rate)
• [ ] Set up daily/weekly summaries
• [ ] Alert on consecutive failures
• [ ] Monitor resource usage

Testing

• [ ] Test with Manual Trigger first
• [ ] Verify timezone behavior
• [ ] Test error scenarios
• [ ] Check for overlapping executions
• [ ] Validate output quality

Deployment

• [ ] Document workflow purpose
• [ ] Set up monitoring
• [ ] Configure alerts
• [ ] Activate workflow in n8n UI ⚠️ Manual activation required (API/MCP cannot activate)
• [ ] Test in production (short interval first)
• [ ] Monitor first few executions

---

Advanced Patterns

Dynamic Scheduling

Change schedule based on conditions:

Schedule (check every hour) → Code (check if it's time to run)
  → IF (business hours AND weekday)
    └─ Execute workflow
  → ELSE
    └─ Skip

Dependent Schedules

Chain workflows:

Workflow A (daily 2 AM): Data sync
  → On completion → Trigger Workflow B

Workflow B: Generate report (depends on fresh data)

Conditional Execution

Skip based on external factors:

Schedule → HTTP Request (check feature flag)
  → IF (feature enabled)
    └─ Execute
  → ELSE
    └─ Skip

---

Summary

Key Points: 1. Set workflow timezone explicitly 2. Batch processing for large datasets 3. Error handling is critical (Error Trigger + retries) 4. Prevent overlaps with execution locks 5. Monitor and log all executions

Pattern: Schedule → Fetch → Process → Deliver → Log

Schedule Modes:

• Interval: Simple recurring (every X minutes/hours)
• Days & Hours: Specific days and times
• Cron: Advanced complex schedules

Related:

• http_api_integration.md - Fetching data on schedule
• database_operations.md - Scheduled database tasks
• webhook_processing.md - Alternative to scheduling

Webhook Processing Pattern

Use Case: Receive HTTP requests from external systems and process them instantly.

---

Pattern Structure

Webhook → [Validate] → [Transform] → [Action] → [Response/Notify]

Key Characteristic: Instant event-driven processing

---

Core Components

1. Webhook Node (Trigger)

Purpose: Create HTTP endpoint to receive data

Configuration:

{
  path: "form-submit",        // URL path: https://n8n.example.com/webhook/form-submit
  httpMethod: "POST",         // GET, POST, PUT, DELETE
  responseMode: "onReceived", // or "lastNode" for custom response
  responseData: "allEntries"  // or "firstEntryJson"
}

Critical Gotcha: Data is nested under $json.body

❌ {{$json.email}}
✅ {{$json.body.email}}

2. Validation (Optional but Recommended)

Purpose: Verify incoming data before processing

Options:

• IF node - Check required fields exist
• Code node - Custom validation logic
• Stop and Error - Fail gracefully with message

Example:

// IF node condition
{{$json.body.email}} is not empty AND
{{$json.body.name}} is not empty

3. Transformation

Purpose: Map webhook data to desired format

Typical nodes:

• Set - Field mapping
• Code - Complex transformations

Example (Set node):

{
  "user_email": "={{$json.body.email}}",
  "user_name": "={{$json.body.name}}",
  "timestamp": "={{$now}}"
}

4. Action

Purpose: Do something with the data

Common actions:

• Store in database (Postgres, MySQL, MongoDB)
• Send notification (Slack, Email, Discord)
• Call another API (HTTP Request)
• Update external system (CRM, support ticket)

5. Response (If responseMode: "lastNode")

Purpose: Send custom HTTP response

Webhook Response Node:

{
  statusCode: 200,
  headers: {
    "Content-Type": "application/json"
  },
  body: {
    "status": "success",
    "message": "Form received"
  }
}

---

Common Use Cases

1. Form Submissions

Flow: Form → Webhook → Validate → Database → Email Confirmation

Example:

1. Webhook (path: "contact-form", POST)
2. IF (check email & message not empty)
3. Postgres (insert into contacts table)
4. Email (send confirmation to user)
5. Slack (notify team in #leads)
6. Webhook Response ({"status": "success"})

Real Data Access:

Name: {{$json.body.name}}
Email: {{$json.body.email}}
Message: {{$json.body.message}}

2. Payment Webhooks (Stripe, PayPal)

Flow: Payment Provider → Webhook → Verify → Update Database → Send Receipt

Security: Verify webhook signatures

// Code node - verify Stripe signature
const crypto = require('crypto');
const signature = $input.item.headers['stripe-signature'];
const secret = $credentials.stripeWebhookSecret;

// Verify signature matches
const expectedSig = crypto
  .createHmac('sha256', secret)
  .update($input.item.body)
  .digest('hex');

if (signature !== expectedSig) {
  throw new Error('Invalid webhook signature');
}

return $input.item.body; // Return validated body

3. Chat Platform Integrations (Slack, Discord, Teams)

Flow: Chat Command → Webhook → Process → Respond

Example (Slack slash command):

1. Webhook (path: "slack-command", POST)
2. Code (parse Slack payload: $json.body.text, $json.body.user_id)
3. HTTP Request (fetch data from API)
4. Set (format Slack message)
5. Webhook Response (immediate Slack response)

Slack Data Access:

Command: {{$json.body.command}}
Text: {{$json.body.text}}
User ID: {{$json.body.user_id}}
Channel ID: {{$json.body.channel_id}}

4. GitHub/GitLab Webhooks

Flow: Git Event → Webhook → Parse → Notify/Deploy

Example (new PR notification):

1. Webhook (path: "github", POST)
2. IF (check $json.body.action equals "opened")
3. Set (extract PR details: title, author, url)
4. Slack (notify #dev-team)
5. Webhook Response (200 OK)

GitHub Data Access:

Event Type: {{$json.headers['x-github-event']}}
Action: {{$json.body.action}}
PR Title: {{$json.body.pull_request.title}}
Author: {{$json.body.pull_request.user.login}}
URL: {{$json.body.pull_request.html_url}}

5. IoT Device Data

Flow: Device → Webhook → Validate → Store → Alert (if threshold)

Example (temperature sensor):

1. Webhook (path: "sensor-data", POST)
2. Set (extract sensor readings)
3. Postgres (insert into sensor_readings)
4. IF (temperature > 80)
5. Email (alert admin)

---

Webhook Data Structure

Standard Structure

{
  "headers": {
    "content-type": "application/json",
    "user-agent": "...",
    "x-custom-header": "..."
  },
  "params": {
    "id": "123"  // From URL: /webhook/form/:id
  },
  "query": {
    "token": "abc"  // From URL: /webhook/form?token=abc
  },
  "body": {
    // ⚠️ YOUR DATA IS HERE!
    "name": "John",
    "email": "john@example.com"
  }
}

Accessing Different Parts

// Headers
{{$json.headers['content-type']}}
{{$json.headers['x-api-key']}}

// URL Parameters
{{$json.params.id}}

// Query Parameters
{{$json.query.token}}
{{$json.query.page}}

// Body (MOST COMMON)
{{$json.body.email}}
{{$json.body.user.name}}
{{$json.body.items[0].price}}

---

Authentication & Security

1. Query Parameter Token

Simple but less secure

// IF node - validate token
{{$json.query.token}} equals "your-secret-token"

2. Header-Based Auth

Better security

// IF node - check header
{{$json.headers['x-api-key']}} equals "your-api-key"

3. Signature Verification

Best security (for webhooks from services like Stripe, GitHub)

// Code node
const crypto = require('crypto');
const signature = $input.item.headers['x-signature'];
const secret = $credentials.webhookSecret;

const calculatedSig = crypto
  .createHmac('sha256', secret)
  .update(JSON.stringify($input.item.body))
  .digest('hex');

if (signature !== `sha256=${calculatedSig}`) {
  throw new Error('Invalid signature');
}

return $input.item.body;

4. IP Whitelist

Restrict access by IP (n8n workflow settings)

• Configure in workflow settings
• Only allow specific IP ranges
• Use for internal systems

---

Response Modes

onReceived (Default)

Behavior: Immediate 200 OK response, workflow continues in background

Use when:

• Long-running workflows
• Response doesn't depend on workflow result
• Fire-and-forget processing

Configuration:

{
  responseMode: "onReceived",
  responseCode: 200
}

lastNode (Custom Response)

Behavior: Wait for workflow completion, send custom response

Use when:

• Need to return data to caller
• Synchronous processing required
• Form submissions with confirmation

Configuration:

{
  responseMode: "lastNode"
}

Then add Webhook Response node:

{
  statusCode: 200,
  headers: {
    "Content-Type": "application/json"
  },
  body: {
    "id": "={{$json.record_id}}",
    "status": "success"
  }
}

---

Error Handling

Pattern 1: Try-Catch with Error Trigger

Main Flow:
  Webhook → [nodes...] → Success Response

Error Flow:
  Error Trigger → Log Error → Slack Alert → Error Response

Error Trigger Configuration:

{
  workflowId: "current-workflow-id"
}

Error Response (if responseMode: "lastNode"):

{
  statusCode: 500,
  body: {
    "status": "error",
    "message": "Processing failed"
  }
}

Pattern 2: Validation Early Exit

Webhook → IF (validate) → [True: Process]
                       └→ [False: Error Response]

False Branch Response:

{
  statusCode: 400,
  body: {
    "status": "error",
    "message": "Invalid data: missing email"
  }
}

Pattern 3: Continue On Fail

Per-node setting: Continue even if node fails

Use case: Non-critical notifications

Webhook → Database (critical) → Slack (continueOnFail: true)

---

Testing Webhooks

1. Use Manual Trigger

Replace Webhook with Manual Trigger for testing:

Manual Trigger → [set test data] → rest of workflow

2. Use curl

curl -X POST https://n8n.example.com/webhook/form-submit \
  -H "Content-Type: application/json" \
  -d '{"email": "test@example.com", "name": "Test User"}'

3. Use Postman/Insomnia

• Create request collection
• Test different payloads
• Verify responses

4. Webhook.site

• Use webhook.site for testing
• Copy webhook.site URL to your service
• View requests and debug

---

Performance Considerations

Large Payloads

• Webhook timeout: 120 seconds (default)
• For large data, consider async processing:

  Webhook → Queue (Redis/DB) → Response (immediate)

  Separate Workflow:
  Schedule → Check Queue → Process

High Volume

• Use "Execute Once" mode if processing all items together
• Consider rate limiting
• Monitor execution times
• Scale n8n instance if needed

Retries

• Webhook calls typically don't retry automatically
• Implement retry logic on caller side
• Or use queue pattern for guaranteed processing

---

Common Gotchas

1. ❌ Wrong: Accessing webhook data

{{$json.email}}  // Empty or undefined

✅ Correct

{{$json.body.email}}  // Data is under .body

2. ❌ Wrong: Response mode confusion

Using Webhook Response node with responseMode: "onReceived" (ignored)

✅ Correct

Set responseMode: "lastNode" to use Webhook Response node

3. ❌ Wrong: No validation

Assuming data is always present and valid

✅ Correct

Validate data early with IF node or Code node

4. ❌ Wrong: Hardcoded paths

Using same path for dev/prod

✅ Correct

Use environment variables: {{$env.WEBHOOK_PATH_PREFIX}}/form-submit

---

Real Template Examples

From n8n template library (1,085 webhook templates):

Simple Form to Slack:

Webhook → Set → Slack

Payment Processing:

Webhook → Verify Signature → Update Database → Send Receipt → Notify Admin

Chat Bot:

Webhook → Parse Command → AI Agent → Format Response → Webhook Response

Use search_templates({query: "webhook"}) to find more!

---

Checklist for Webhook Workflows

Setup

• [ ] Choose descriptive webhook path
• [ ] Configure HTTP method (POST most common)
• [ ] Choose response mode (onReceived vs lastNode)
• [ ] Test webhook URL before connecting services

Security

• [ ] Add authentication (token, signature, IP whitelist)
• [ ] Validate incoming data
• [ ] Sanitize user input (if storing/displaying)
• [ ] Use HTTPS (always)

Data Handling

• [ ] Remember data is under $json.body
• [ ] Handle missing fields gracefully
• [ ] Transform data to desired format
• [ ] Log important data (for debugging)

Error Handling

• [ ] Add Error Trigger workflow
• [ ] Validate required fields
• [ ] Return appropriate error responses
• [ ] Alert team on failures

Testing

• [ ] Test with curl/Postman
• [ ] Test error scenarios
• [ ] Verify response format
• [ ] Monitor first executions

---

Summary

Key Points: 1. Data under $json.body (most common mistake!) 2. Validate early to catch bad data 3. Choose response mode based on use case 4. Secure webhooks with auth 5. Handle errors gracefully

Pattern: Webhook → Validate → Transform → Action → Response

Related:

• n8n Expression Syntax - Accessing webhook data correctly
• http_api_integration.md - Making HTTP requests in response