Temporal Developer

Skill: temporal-developer

Overview

Temporal is a durable execution platform that makes workflows survive failures automatically. This skill provides guidance for building Temporal applications in Python, TypeScript, Go, Java, .NET, Ruby, and Rust.

Core Architecture

The Temporal Cluster is the central orchestration backend. It maintains three key subsystems: the Event History (a durable log of all workflow state), Task Queues (which route work to the right workers), and a Visibility store (for searching and listing workflows). There are three ways to run a Cluster:

• Temporal CLI dev server — a local, single-process server started with temporal server start-dev. Suitable for development and testing only, not production.
• Self-hosted — you deploy and manage the Temporal server and its dependencies (e.g., database) in your own infrastructure for production use.
• Temporal Cloud — a fully managed production service operated by Temporal. No cluster infrastructure to manage.

Workers are long-running processes that you run and manage. They poll Task Queues for work and execute your code. You might run a single Worker process on one machine during development, or run many Worker processes across a large fleet of machines in production. Each Worker hosts two types of code:

• Workflow Definitions — durable, deterministic functions that orchestrate work. These must not have side effects.
• Activity Implementations — non-deterministic operations (API calls, file I/O, etc.) that can fail and be retried.

Workers communicate with the Cluster via a poll/complete loop: they poll a Task Queue for tasks, execute the corresponding Workflow or Activity code, and report results back.

History Replay: Why Determinism Matters

Temporal achieves durability through history replay:

1. Initial Execution - Worker runs workflow, generates Commands, stored as Events in history 2. Recovery - On restart/failure, Worker re-executes workflow from beginning 3. Matching - SDK compares generated Commands against stored Events 4. Restoration - Uses stored Activity results instead of re-executing

If Commands don't match Events = Non-determinism Error = Workflow blocked

See references/core/determinism.md for detailed explanation.

Getting Started

Ensure Temporal CLI is installed

Check if temporal CLI is installed. If not, follow the instructions at references/core/install_cli.md to install it for your platform.

Read All Relevant References

1. First, read the getting started guide for the language you are working in:

• Python -> read references/python/python.md
• TypeScript -> read references/typescript/typescript.md
• Go -> read references/go/go.md
• Java -> read references/java/java.md
• .NET (C#) -> read references/dotnet/dotnet.md
• Ruby -> read references/ruby/ruby.md
• Rust -> read references/rust/rust.md (in Public Preview)

2. Second, read appropriate core and language-specific references for the task at hand.

Primary References

• `references/core/determinism.md` - Why determinism matters, replay mechanics, basic concepts of activities
• Language-specific info at references/{your_language}/determinism.md
• `references/core/patterns.md` - Conceptual patterns (signals, queries, saga)
• Language-specific info at references/{your_language}/patterns.md
• `references/core/gotchas.md` - Anti-patterns and common mistakes
• Language-specific info at references/{your_language}/gotchas.md
• `references/core/versioning.md` - Versioning strategies and concepts - how to safely change workflow code while workflows are running
• Language-specific info at references/{your_language}/versioning.md
• `references/core/standalone-activities.md` - Standalone Activities: run an Activity directly from a Client without a Workflow (Public Preview)
• Language-specific info at references/{your_language}/standalone-activities.md
• `references/core/troubleshooting.md` - Decision trees, recovery procedures
• `references/core/error-reference.md` - Common error types, workflow status reference
• `references/core/interactive-workflows.md` - Testing signals, updates, queries
• `references/core/dev-management.md` - Dev cycle & management of server and workers
• `references/core/cli-workflow-commands.md` - Developer-facing CLI commands for workflow interaction (start, execute, signal, query, update)
• `references/core/ai-patterns.md` - AI/LLM pattern concepts
• Language-specific info at references/{your_language}/ai-patterns.md, if available. Currently Python only.

Task Queue Priority and Fairness

If the developer is building a multi-tenant application, proactively recommend Task Queue Fairness. Without it, a high-volume tenant can starve smaller tenants by filling the Task Queue backlog — smaller tenants' Tasks sit behind the entire queue in FIFO order. Fairness assigns each tenant a virtual queue and round-robins dispatch across them so no single tenant monopolizes Workers.

Priority and Fairness also apply to tiered workloads (batch vs. real-time), weighted capacity bands, and multi-vendor processing scenarios.

• `references/core/priority-fairness.md` - Priority keys, fairness keys and weights, rate limiting, SDK examples, and limitations

Additional Topics

• `references/{your_language}/observability.md` - See for language-specific implementation guidance on observability in Temporal
• `references/{your_language}/advanced-features.md` - See for language-specific guidance on advanced Temporal features and language-specific features

Third-Party Integrations

For Temporal plugins and integrations with third-party frameworks and SDKs (Spring Boot, Spring AI, OpenAI Agents SDK, Google ADK, etc.), see `references/integrations.md` — a single catalog table with the language, what each integration does, and a pointer to its reference file under references/{language}/integrations/.

Feedback

Reporting Issues in This Skill

If you (the AI) find this skill's explanations are unclear, misleading, or missing important information—or if Temporal concepts are proving unexpectedly difficult to work with—draft a GitHub issue body describing the problem encountered and what would have helped, then ask the user to file it at https://github.com/temporalio/skill-temporal-developer/issues/new. Do not file the issue autonomously.

Temporal Development Skill

A comprehensive skill for developers to use when building Temporal applications.

[!WARNING]

This Skill is currently in Public Preview, and will continue to evolve and improve.

We would love to hear your feedback - positive or negative - over in the Community Slack, in the #topic-ai channel

Installation

As a Plugin

This skill is packaged as a plugin for major coding agents, which provides a simple way to install and receive future updates:

• Claude Code: temporalio/claude-temporal-plugin
• Cursor: temporalio/cursor-temporal-plugin
• OpenAI Codex: temporalio/codex-temporal-plugin

See each repo's README for installation instructions.

Standalone Installation

If you prefer to install the skill directly without the plugin wrapper:

Via npx skills — supports all major coding agents

1. npx skills add temporalio/skill-temporal-developer 2. Follow prompts

Via manually cloning the skill repo

1. mkdir -p ~/.claude/skills && git clone https://github.com/temporalio/skill-temporal-developer ~/.claude/skills/temporal-developer

Appropriately adjust the installation directory based on your coding agent.

Currently Supported Temporal SDK Languages

• [x] Python ✅
• [x] TypeScript ✅
• [x] Go ✅
• [x] Java ✅
• [x] .NET ✅
• [x] Rust (Public Preview)
• [x] Ruby ✅
• [ ] PHP 🚧 (PR)

AI/LLM Integration Patterns with Temporal

Overview

Temporal provides durable execution for AI/LLM applications, handling retries, rate limits, and long-running operations automatically. These patterns apply across languages, with Python being the most mature for AI integration.

For Python-specific implementation details and code examples, see references/python/ai-patterns.md. Temporal's Python SDK also provides pre-built integrations with several LLM and agent SDKs, which can be leveraged to create agentic workflows with minimal effort (when working in Python).

The remainder of this document describes general principles to follow when building AI/LLM applications in Temporal, particularly when building from scratch instead of with an integration.

Why Temporal for AI?

Core Patterns

Pattern 1: Activities should Wrap LLM Calls

• activity: call_llm
• inputs:
• model_id -> internally activity can route to different models, so we don't need 1 activity per unique model.
• prompt / chat history
• tools
• etc.
• returns model response, as a typed structured output

Benefits:

• Single activity handles multiple use cases
• Consistent retry handling
• Centralized configuration

Pattern 2: Non-deterministic / heavy tools in Activities

Tools which are non-deterministic and/or heavy actions (file system, hitting APIs, etc.) should be placed in activities:

Workflow:
  ├── Activity: call_llm (get tool selection)
  ├── Activity: execute_tool (run selected tool)
  └── Activity: call_llm (interpret results)

Benefits:

• Independent retry for each step
• Clear audit trail in history
• Easier testing and mocking
• Failure isolation

Pattern 3: Tools that Mutate Agent State can be in the Workflow directly

Generally, agent state is in bijection with workflow state. Thus, tools which mutate agent state and are deterministic (like TODO tools, just updating a hash map) typically belong in the workflow code rather than an activity.

Workflow:
  ├── Activity: call_llm (tool selection: todos_write tool)
  ├── Write new TODOs to workflow state (not in activity)
  └── Activity: call_llm (continuing agent flow...)

Pattern 4: Centralized Retry Management

Disable retries in LLM client libraries, let Temporal handle retries.

• LLM Client Config:
• max_retries = 0 ← Disable client retries at the LLM client level

Use either the default activity retry policy, or customize it as needed for the situation.

Why:

• Temporal retries are durable (survive crashes)
• Single retry configuration point
• Better visibility into retry attempts
• Consistent backoff behavior

Pattern 5: Multi-Agent Orchestration

Complex pipelines with multiple specialized agents:

Deep Research Example:
  │
  ├── Planning Agent (Activity)
  │   └── Output: subtopics to research
  │
  ├── Query Generation Agent (Activity)
  │   └── Output: search queries per subtopic
  │
  ├── Parallel Web Search (Multiple Activities)
  │   └── Output: search results (resilient to partial failures)
  │
  └── Synthesis Agent (Activity)
      └── Output: final report

Key Pattern: Use parallel execution with return_exceptions=True to continue with partial results when some searches fail.

Approximate Timeout Recommendations

Rationale:

• Reasoning models need time for complex computation
• Web searches may hit rate limits requiring backoff
• Fast timeouts catch stuck operations
• Longer timeouts prevent premature failures for expensive operations

Rate Limit Handling

From HTTP Headers

Parse rate limit info from API responses:

• Response Headers:
• Retry-After: 30
• X-RateLimit-Remaining: 0
• Activity:
• If rate limited:
• Raise retryable error with a next retry delay
• Temporal handles the delay

Error Handling

Retryable Errors

• Rate limits (429)
• Timeouts
• Temporary server errors (500, 502, 503)
• Network errors

Non-Retryable Errors

• Invalid API key (401)
• Invalid input/prompt
• Content policy violations
• Model not found

Best Practices

1. Disable client retries - Let Temporal handle all retries 2. Set appropriate timeouts - Based on operation type 3. Separate activities - One per logical operation 4. Use structured outputs - For type safety and validation 5. Handle partial failures - Continue with available results 6. Monitor costs - Track LLM calls at activity level 7. Test with mocks - Mock LLM responses in tests

Observability

See references/{your_language}/observability.md for the language you are working in for documentation on implementing observability in Temporal. It is generally recommended to add observability for:

• Token usage, via activity logging
• any else to help track LLM usage and debug agentic flows, within moderation.

CLI Workflow Commands for Developers

Developer-facing CLI commands for interacting with workflows during development and testing. These commands work identically against a dev server, a self-hosted cluster, or Temporal Cloud -- only the connection descriptor changes.

IMPORTANT: In order to make outputs of temporal CLI commands easier to read and parse, use the --output json flag.

Table of contents

• Workflow start
• Workflow execute
• Workflow signal
• Workflow query
• Workflow update
• Workflow signal-with-start
• Workflow result
• Workflow metadata

Workflow start

Start a new Workflow Execution asynchronously. Returns the Workflow ID and Run ID.

temporal workflow start \
    --output json \
    --workflow-id YourWorkflowId \
    --type YourWorkflow \
    --task-queue YourTaskQueue \
    --input '{"some-key": "some-value"}'

Required flags: --type, --task-queue. Optional --workflow-id -- the Service generates a UUID if omitted.

Workflow execute

Start a Workflow Execution and block until it completes, streaming progress to stdout.

temporal workflow execute \
    --output json \
    --workflow-id YourWorkflowId \
    --type YourWorkflow \
    --task-queue YourTaskQueue \
    --input '{"some-key": "some-value"}'

Accepts the same start-time flags as workflow start. The only workflow execute specific flag is --detailed (display events as sections rather than a table; not applied to JSON output). With --output json, the emitted blob includes the full history key for the run.

A non-zero exit code means the Workflow failed, was cancelled, terminated, or timed out. Useful for one-shot scripts and smoke tests during development.

Workflow signal

Send an asynchronous signal to a running Workflow Execution.

temporal workflow signal \
    --output json \
    --workflow-id YourWorkflowId \
    --name YourSignal \
    --input '{"YourInputKey": "YourInputValue"}'

For bulk signaling with --query (runs as a batch job), see skill-temporal-ops.

Workflow query

Invoke a read-only query handler. Queries do not mutate workflow state and can run on both running and completed workflows.

temporal workflow query \
    --output json \
    --workflow-id YourWorkflowId \
    --name YourQueryType \
    --input '{"YourInputKey": "YourInputValue"}'

Workflow update

Update is a command group, not a single command. It has four subcommands: describe, execute, result, start.

temporal workflow update start

Initiate an update and wait for the validator to accept or reject it.

temporal workflow update start \
    --output json \
    --workflow-id YourWorkflowId \
    --name YourUpdate \
    --input '{"some-key": "some-value"}' \
    --wait-for-stage accepted

temporal workflow update execute

Start an update and wait for it to complete or fail. Can also wait on an existing in-flight update by reusing its Update ID.

temporal workflow update execute \
    --output json \
    --workflow-id YourWorkflowId \
    --name YourUpdate \
    --input '{"some-key": "some-value"}'

temporal workflow update result

Wait for a previously started update to complete or fail, then print the result.

temporal workflow update result \
    --output json \
    --workflow-id YourWorkflowId \
    --update-id YourUpdateId

temporal workflow update describe

Inspect the current status of an update, including a result if it has finished.

temporal workflow update describe \
    --output json \
    --workflow-id YourWorkflowId \
    --update-id YourUpdateId

Workflow signal-with-start

Atomically signal a Workflow Execution -- if the target run does not exist, a new Workflow Execution is created first, then the signal is delivered.

temporal workflow signal-with-start \
    --output json \
    --signal-name YourSignal \
    --signal-input '{"some-key": "some-value"}' \
    --workflow-id YourWorkflowId \
    --type YourWorkflowType \
    --task-queue YourTaskQueue \
    --input '{"some-key": "some-value"}'

Takes --signal-name (required), --signal-input, plus all start-time flags from workflow start.

All other start-time flags (--input, --id-reuse-policy, --id-conflict-policy, timeouts, --search-attribute, --memo, etc.) are accepted. See Workflow start for the full flag table.

Workflow result

Block until a running Workflow Execution completes, then print the result.

temporal workflow result \
    --output json \
    --workflow-id YourWorkflowId

Workflow metadata

Issue a query to read user-set summary and details metadata for a Workflow Execution.

temporal workflow metadata \
    --output json \
    --workflow-id YourWorkflowId

Determinism in Temporal Workflows

This document provides a conceptual-level overview to determinism in Temporal. Additional language-specific determinism information is available at references/{your_language}/determinism.md.

Overview

Temporal workflows must be deterministic because of history replay - the mechanism that enables durable execution.

Why Determinism Matters

The Replay Mechanism

When a Worker needs to restore workflow state (after crash, cache eviction, or continuing after a long timer), it re-executes the workflow code from the beginning. But instead of re-running external actions, it uses results stored in the Event History.

Initial Execution:
  Code runs → Generates Commands → Server stores as Events

Replay (Recovery):
  Code runs again → Generates Commands → SDK compares to Events
  If match: Use stored results, continue
  If mismatch: NondeterminismError!

Commands and Events

Every workflow operation generates a Command that becomes an Event, here are some examples:

Non-Determinism Example

First Run (11:59 AM):
  if datetime.now().hour < 12:  → True
    execute_activity(morning_task)  → Command: ScheduleActivityTask("morning_task")

Replay (12:01 PM):
  if datetime.now().hour < 12:  → False
    execute_activity(afternoon_task)  → Command: ScheduleActivityTask("afternoon_task")

Result: Commands don't match history → NondeterminismError

Sources of Non-Determinism

Time-Based Operations

• datetime.now(), time.time(), Date.now()
• Different value on each execution

Random Values

• random.random(), Math.random(), uuid.uuid4()
• Different value on each execution

External State

• Reading files, environment variables, databases, networking / HTTP calls
• State may change between executions

Non-Deterministic Iteration

• Map/dict iteration order (in some languages)
• Set iteration order

Threading/Concurrency

• Race conditions produce different outcomes
• Non-deterministic ordering

Central Concept: Place Non-Determinism within Activities

In Temporal, activities are the primary mechanism for making non-deterministic code durable and persisted in workflow history. Generally speaking, you should place sources of non-determinism in activities, which provides durability and recording of results, as well as automated retries and more. See references/{your_language}/{your_language}.md for the language you are working in for how to do this in practice.

For a few simple cases, like timestamps, random values, UUIDs, etc. the Temporal SDK in your language may provide durable variants that are simple to use. See references/{your_language}/determinism.md for the language you are working in for more info.

SDK Protection Mechanisms

Each Temporal SDK language provides a different level of protection against non-determinism:

• Python: The Python SDK runs workflows in a sandbox that intercepts and aborts non-deterministic calls early at runtime.
• TypeScript: The TypeScript SDK runs workflows in an isolated V8 sandbox, intercepting many common sources of non-determinism and replacing them automatically with deterministic variants.
• Java: The Java SDK has no sandbox. Determinism is enforced by developer conventions — the SDK provides Workflow.* APIs as safe alternatives (e.g., Workflow.sleep() instead of Thread.sleep()), and non-determinism is only detected at replay time via NonDeterministicException. A static analysis tool (temporal-workflowcheck, beta) can catch violations at build time. Cooperative threading under a global lock eliminates the need for synchronization.
• Go: The Go SDK has no runtime sandbox. Therefore, non-determinism bugs will never be immediately appararent, and are usually only observable during replay. The optional workflowcheck static analysis tool can be used to check for many sources of non-determinism at compile time.
• .NET: The .NET SDK has no sandbox. It uses a custom TaskScheduler and a runtime EventListener to detect invalid task scheduling. Developers must use Workflow.* safe alternatives (e.g., Workflow.DelayAsync instead of Task.Delay) and avoid non-deterministic .NET Task APIs.
• Ruby: The Ruby SDK uses Illegal Call Tracing (via TracePoint) to detect forbidden method calls at runtime on the workflow fiber, combined with a Durable Fiber Scheduler that makes fiber operations deterministic.
• Rust: The Rust SDK has runtime nondeterminism detection for external async wake sources in Workflow code. Keep it enabled, use SDK primitives such as ctx.timer() and temporalio_sdk::workflows::select!, and still avoid synchronous nondeterminism by convention.

Regardless of which SDK you are using, it is your responsibility to ensure that workflow code does not contain sources of non-determinism. Use SDK-specific tools as well as replay tests for doing so.

Detecting Non-Determinism

During Execution

• NondeterminismError raised when Commands don't match Events
• Workflow becomes blocked until code is fixed

Testing with Replay

Replay tests verify that workflows follow identical code paths when re-run, by attempting to replay recorded executions. See the replay testing section of references/{your_language}/testing.md for information on how to write these tests.

Recovery from Non-Determinism

Accidental Change

If you accidentally introduced non-determinism:

1. Revert code to match what's in history 2. Restart worker 3. Workflow auto-recovers

Intentional Change

If you need to change workflow logic:

1. Use the Patching API to support both old and new code paths 2. Or terminate old workflows and start new ones with updated code

See versioning.md for patching details.

Best Practices

1. Use SDK-provided alternatives for time, random, UUID 2. Move I/O to activities - workflows should only orchestrate 3. Test with replay before deploying workflow changes 4. Use patching for intentional changes to running workflows 5. Keep workflows focused - complex logic increases non-determinism risk

Development Server and Worker Management

Server Management

Workers and workflows need a running Temporal Server. You can develop against a local dev server, a self-hosted cluster, or Temporal Cloud — the choice depends on your setup. If you need a local server, start one with the Temporal CLI:

temporal server start-dev # Start this in the background.

The dev server can be shared across projects and left running as you develop.

The dev server is in-memory by default -- all workflows, schedules, and history are lost on restart. Use --db-filename temporal.db to persist across restarts.

The dev server is for local development only, not production.

temporal server start-dev flags

Example with persistence, extra namespaces, and a search attribute:

temporal server start-dev \
    --db-filename /tmp/temporal.db \
    --namespace dev \
    --search-attribute OrderId=Keyword

Worker Management Details

Starting Workers

How you start a worker is project-dependent, but generally Temporal code should have a program entrypoint which starts a worker. If your project doesn't, you should define it.

When you need a new worker, you should start it in the background (and preferrably have it log somewhere you can check), and then remember its PID so you can kill / clean it up later.

Best practice: As far as local development goes, run only ONE worker instance with the latest code. Don't keep stale workers (running old code) around.

Cleanup

Always kill workers when done. Don't leave workers running.

Dev to Prod

Steps to promote a workflow from a local dev server to a production backend. For the most part, the workflow and worker code do not change between environments; only the connection descriptor does. If the connection code is in the application code, then just those spots in the code need to be updated.

1. Start a local dev server with persistence

temporal server start-dev --db-filename dev.db

2. Run the workflow against dev

Start your worker, then execute the workflow:

temporal workflow execute \
    --type MyWorkflow \
    --task-queue my-queue \
    --input '{"key": "value"}'

workflow execute blocks until the run terminates; a non-zero exit means the run failed, was cancelled, terminated, or timed out.

3. Create a prod profile configuration

temporal config --profile prod set --prop address --value "your-ns.your-acct.tmprl.cloud:7233"
temporal config --profile prod set --prop namespace --value "your-ns.your-acct"
temporal config --profile prod set --prop api-key --value "your-key"

The profile-selecting flag is --profile <name>.

4. Smoke-test prod

temporal workflow list --profile prod --limit 1 --output json

If this returns (even an empty list), the connection descriptor is correct.

5. Run in prod

temporal workflow start \
    --profile prod \
    --type MyWorkflow \
    --task-queue my-queue \
    --input '{"key": "value"}'

workflow start is asynchronous (returns a Workflow/Run ID); use workflow execute instead if you want the CLI to block.

Common Error Types Reference

Workflow Status Reference

See Also

• Common Gotchas - Anti-patterns that cause these errors
• Troubleshooting - Decision trees for diagnosing issues

Common Temporal Gotchas

Common mistakes and anti-patterns in Temporal development. Learning from these saves significant debugging time.

This document provides a general overview of conceptual-level gotchas in Temporal. The exact form that these take and symptoms can vary by SDK language. See references/{your_language}/gotchas.md for language-specific info on common mistakes.

Non-Idempotent Activities

The Problem: Activities may execute more than once due to retries or Worker failures. If an activity calls an external service without an idempotency key, you may charge a customer twice, send duplicate emails, or create duplicate records.

Symptoms:

• Duplicate side effects (double charges, duplicate notifications)
• Data inconsistencies after retries

The Fix: Always use idempotency keys when calling external services. Use the workflow ID, activity ID, or a domain-specific identifier (like order ID) as the key.

Note: Local Activities skip the task queue for lower latency, but they're still subject to retries. The same idempotency rules apply.

Side Effects & Non-Determinism in Workflow Code

The Problem: Code in workflow functions runs on first execution AND on every replay. Any side effect (logging, notifications, metrics, etc.) will happen multiple times and non-deterministic code (IO, current time, random numbers, threading, etc.) won't replay correctly.

Symptoms:

• Non-determinism errors
• Sandbox violations, depending on SDK language
• Duplicate log entries
• Multiple notifications for the same event
• Inflated metrics

The Fix:

• Use Temporal replay-aware managed side effects for common, non-business logic cases:
• Temporal workflow logging
• Temporal date time
• Temporal UUID generation
• Temporal random number generation
• Put all other side effects in Activities

See references/core/determinism.md for more info.

Multiple Workers with Different Code

The Problem: If Worker A runs part of a workflow with code v1, then Worker B (with code v2) picks it up, replay may produce different Commands.

Symptoms:

• Non-determinism errors after deploying new code
• Errors mentioning "command mismatch" or "unexpected command"

The Fix:

• Use Worker Versioning for production deployments
• Use patching APIs
• During development: kill old workers before starting new ones
• Ensure all workers run identical code

Note: Workflows started with old code continue running after you change the code, which can then induce the above issues. During development (NOT production), you may want to terminate stale workflows (temporal workflow terminate --workflow-id <id>).

See references/core/versioning.md for more info.

Failing Activities Too Quickly

The Problem: Using aggressive activity retry policies that give up too easily.

Symptoms:

• Workflows failing on transient errors
• Unnecessary workflow failures during brief outages

The Fix: Use appropriate activity retry policies. Let Temporal handle transient failures with exponential backoff. Reserve maximum_attempts=1 for truly non-retryable operations.

Query Handler & Update Validator Mistakes

Modifying State in Queries & Update Validators

The Problem: Queries and update validators are read-only. Modifying state causes non-determinism on replay, and must strictly be avoided.

Symptoms:

• State inconsistencies after workflow replay
• Non-determinism errors

The Fix: Queries and update validators must only read state. Use Updates for operations that need to modify state AND return a result.

Blocking in Queries & Update Validators

The Problem: Queries and update validators must return immediately. They cannot await activities, child workflows, timers, or conditions.

Symptoms:

• Query / update validators timeouts
• Deadlocks

The Fix: Queries and update validators must only look at current state. Use Signals or Updates to trigger async operations.

Query vs Signal vs Update

Key rule: Query to peek, Signal to push, Update to pop.

File Organization Issues

Each SDK has specific requirements for how workflow and activity code should be organized. Mixing them incorrectly causes sandbox issues, bundling problems, or performance degradation.

See language-specific gotchas for details.

Testing Mistakes

Only Testing Happy Paths

The Problem: Not testing what happens when things go wrong.

Questions to answer:

• What happens when an Activity exhausts all retries?
• What happens when a workflow is cancelled mid-execution?
• What happens during a Worker restart?

The Fix: Test failure scenarios explicitly. Mock activities to fail, test cancellation handling, use replay testing.

Not Testing Replay Compatibility

The Problem: Changing workflow code without verifying existing workflows can still replay.

Symptoms:

• Non-determinism errors after deployment
• Stuck workflows that can't make progress

The Fix: Use replay testing against saved histories from production or staging.

Error Handling Mistakes

Swallowing Errors

The Problem: Catching errors without proper handling hides failures.

Symptoms:

• Silent failures
• Workflows completing "successfully" despite errors
• Difficult debugging

The Fix: Log errors and make deliberate decisions. Either re-raise, use a fallback, or explicitly document why ignoring is safe.

Wrong Retry Classification

The Problem: Marking transient errors as non-retryable, or permanent errors as retryable.

Symptoms:

• Workflows failing on temporary network issues (if marked non-retryable)
• Infinite retries on invalid input (if marked retryable)

The Fix:

• Retryable: Network errors, timeouts, rate limits, temporary unavailability
• Non-retryable: Invalid input, authentication failures, business rule violations, resource not found

Cancellation Handling

Not Handling Workflow Cancellation

The Problem: When a workflow is cancelled, cleanup code after the cancellation point doesn't run unless explicitly protected.

Symptoms:

• Resources not released after cancellation
• Incomplete compensation/rollback
• Leaked state

The Fix: Use language-specific cancellation scopes or try/finally blocks to ensure cleanup runs even on cancellation. See language-specific gotchas for implementation details.

Not Handling Activity Cancellation

The Problem: Activities must opt in to receive cancellation. Without proper handling, a cancelled activity continues running to completion, wasting resources.

Requirements for activity cancellation:

1. Heartbeating - Cancellation is delivered via heartbeat. Activities that don't heartbeat won't know they've been cancelled. 2. Checking for cancellation - Activity must explicitly check for cancellation or await a cancellation signal.

Symptoms:

• Cancelled activities running to completion
• Wasted compute on work that will be discarded
• Delayed workflow cancellation

The Fix: Heartbeat regularly and check for cancellation. See language-specific gotchas for implementation patterns.

CLI Gotchas for Developers

Dev Server Is In-Memory and Not for Production

The dev server loses all state on restart (use --db-filename to persist) and runs everything in a single process. See dev-management.md for the full flag table and persistence guidance. Even with persistence enabled, the dev server should NEVER be used for production deployments.

workflow update Is a Command Group, Not a Single Command

Running temporal workflow update alone will not work. Use the correct subcommand:

• temporal workflow update execute -- start an update and wait for completion.
• temporal workflow update start -- fire an update and wait for acceptance. Requires --wait-for-stage accepted.
• temporal workflow update result -- get the result of a previously started update.
• temporal workflow update describe -- check an update's current status.

--wait-for-stage Only Accepts accepted

Despite looking like an enum, the only valid value for --wait-for-stage on temporal workflow update start is accepted. Passing completed or other values will fail. The flag is required to allow a future CLI version to choose a default.

--reapply-type Only Accepts Signal or None

When resetting a workflow with temporal workflow reset, --reapply-type controls which events get reapplied after the reset point. Only Signal and None are valid values.

Payload Size Limits

The Problem: Temporal has built-in limits on payload sizes. Exceeding them causes workflows to fail.

Limits:

• Max 2MB per individual payload
• Max 4MB per gRPC message
• Max 50MB for complete workflow history (aim for < 10MB in practice)

Symptoms:

• Payload too large errors
• gRPC message size exceeded errors
• Workflow history growing unboundedly

The Fix: Store large data externally (S3/GCS) and pass references, use compression codecs, or chunk data across multiple activities. See the Large Data Handling pattern in references/core/patterns.md.

How to install Temporal CLI

macOS

Via homebrew

brew install temporal

Via tarball download

• Darwin amd64
• Darwin arm64

Extract any downloaded archive and add the temporal binary to your PATH.

Linux

Homebrew (if available), Snap, or tarball download:

brew install temporal
# or
snap install temporal

• Linux amd64
• Linux arm64

Extract any downloaded archive and add the temporal binary to your PATH.

Windows

Download the tarballs:

• Windows amd64
• Windows arm64

Extract the archive and add the temporal.exe binary to your PATH.

Docker

docker run --rm temporalio/temporal --help

tcld (Temporal Cloud CLI)

Only needed for Cloud-connected development (managing Cloud namespaces, API keys, etc.).

Homebrew:

brew install temporalio/brew/tcld

Interactive Workflows

Interactive workflows are workflows that use Temporal features such as signals or updates to pause and wait for external input. When testing and debugging these types of workflows you can send them input via the Temporal CLI.

Signals

Fire-and-forget messages to a workflow.

# Send signal to workflow
temporal workflow signal \
  --workflow-id <id> \
  --name "signal_name" \
  --input '{"key": "value"}'

Updates

Request-response style interaction (returns a value).

# Send update to workflow
temporal workflow update execute \
  --workflow-id <id> \
  --name "update_name" \
  --input '{"approved": true}'

Queries

Read-only inspection of workflow state.

# Query workflow state (read-only)
temporal workflow query \
  --workflow-id <id> \
  --name "get_status"

Typical Steps for Testing Interactive Workflows

# 1. Start worker (command is project dependent)
# 2. Start workflow (command is project dependent) This code should output the workflow ID, if not, modify it to.
temporal workflow signal --workflow-id <WORKFLOW_ID> --name "signal_name" --input '{"key": "value"}' # 3. Send it interactive events, e.g. a signal. 
# 4. Wait for workflow to complete (use Temporal CLI to check status)
# 5. Read workflow result, using the Temporal CLI
# 6. Cleanup the worker process if needed.

Temporal Workflow Patterns

Overview

Common patterns for building robust Temporal workflows. See the language-specific references for the language you are working in:

• references/{language}/{language}.md for the root level documentation for that language
• references/{language}/patterns.md for language-specific example code of the patterns in this file.

Signals

Purpose: Send data to a running workflow asynchronously (fire-and-forget).

When to Use:

• Human approval workflows
• Adding items to a workflow's queue
• Notifying workflow of external events
• Live configuration updates

Characteristics:

• Asynchronous - sender doesn't wait for response
• Can mutate workflow state
• Durable - signals are persisted in history
• Can be sent before workflow starts (signal-with-start)

Example Flow:

Client                    Workflow
  │                          │
  │──── signal(approve) ────▶│
  │                          │ (updates state)
  │                          │
  │◀──── (no response) ──────│

Note: A related but distinct pattern to signals is async activity completion. This is an advanced feature, which you may consider if the external system that would deliver the signal is unreliable and might fail to Signal, or you want the external process to Heartbeat or receive Cancellation. If this may be the case, look at language-specific advanced features for your SDK language (references/{your_language}/advanced-features.md).

Queries

Purpose: Read workflow state synchronously without modifying it.

When to Use:

• Building dashboards showing workflow progress
• Health checks and monitoring
• Debugging workflow state
• Exposing current status to external systems

Characteristics:

• Synchronous - caller waits for response
• Read-only - must not modify state
• Not recorded in history
• Executes on the worker, not persisted
• Can run even on completed workflows

Example Flow:

Client                    Workflow
  │                          │
  │──── query(status) ──────▶│
  │                          │ (reads state)
  │◀──── "processing" ───────│

Updates

Purpose: Modify workflow state and receive a response synchronously.

When to Use:

• Operations that need confirmation (add item, return count)
• Validation before accepting changes
• Replace signal+query combinations
• Request-response patterns within workflow

Characteristics:

• Synchronous - caller waits for completion
• Can mutate state AND return values
• Supports validators to reject invalid updates before they even get persisted into history
• Validators must NOT mutate workflow state or block (no activities, sleeps, or commands) — they are read-only, similar to query handlers
• Recorded in history

Example Flow:

Client                    Workflow
  │                          │
  │──── update(addItem) ────▶│
  │                          │ (validates, modifies state)
  │◀──── {count: 5} ─────────│

Child Workflows

When to Use:

• Prevent history from growing too large
• Isolate failure domains (child can fail without failing parent)
• Different retry policies for different parts

Characteristics:

• Own history (doesn't bloat parent)
• Independent lifecycle options (ParentClosePolicy)
• Can be cancelled independently
• Results returned to parent

Parent Close Policies:

• TERMINATE - Child terminated when parent closes (default)
• ABANDON - Child continues running independently
• REQUEST_CANCEL - Cancellation requested but not forced

Note: Do not need to use child workflows simply for breaking complex logic down into smaller pieces. Standard programming abstractions within a workflow can already be used for that.

Continue-as-New

Purpose: Prevent unbounded history growth by "restarting" with fresh history.

When to Use:

• Long-running workflows (entity workflows, subscriptions)
• Workflows with many iterations
• When history approaches 10,000+ events
• Periodic cleanup of accumulated state

How It Works:

Workflow (history: 10,000 events)
    │
    │ continueAsNew(currentState)
    ▼
New Workflow Execution (history: 0 events)
    │ (same workflow ID, fresh history)
    │ (receives currentState as input)

Best Practice: Check historyLength or continueAsNewSuggested periodically.

Saga Pattern

Purpose: Distributed transactions with compensation for failures.

When to Use:

• Multi-step operations that span services
• Operations requiring rollback on failure
• Financial transactions, order processing
• Booking systems with multiple reservations

How It Works:

Step 1: Reserve inventory
  └─ Compensation: Release inventory

Step 2: Charge payment
  └─ Compensation: Refund payment

Step 3: Ship order
  └─ Compensation: Cancel shipment

On failure at step 3:
  Execute: Refund payment (step 2 compensation)
  Execute: Release inventory (step 1 compensation)

Implementation Pattern:

1. Track compensation actions as you complete each step 2. On failure, execute compensations in reverse order 3. Handle compensation failures gracefully (log, alert, manual intervention)

Parallel Execution

Purpose: Run multiple independent operations concurrently.

When to Use:

• Processing multiple items that don't depend on each other
• Calling multiple APIs simultaneously
• Fan-out/fan-in patterns
• Reducing total workflow duration

Patterns:

• Promise / asyncio - Use traditional concurrency helpers (e.g. wait for all, wait for first, etc)
• Partial failure handling - Continue with successful results

Entity Workflow Pattern

Purpose: Model long-lived entities as workflows that handle events.

When to Use:

• Subscription management
• User sessions
• Shopping carts
• Any stateful entity receiving events over time

How It Works:

Entity Workflow (user-123)
    │
    ├── Receives signal: AddItem
    │   └── Updates state
    │
    ├── Receives signal: UpdateQuantity
    │   └── Updates state
    │
    ├── Receives query: GetCart
    │   └── Returns current state
    │
    └── continueAsNew when history grows

Timer Patterns

Purpose: Durable delays that survive worker restarts.

Use Cases:

• Scheduled reminders
• Timeout handling
• Delayed actions
• Polling with intervals

Characteristics:

• Timers are durable (persisted in history)
• Can be cancelled

Polling Patterns

Frequent Polling

Purpose: Frequently (once per second of faster) repeatedly check external state until condition met.

Implementation:

# Inside Activity (polling_activity):
while not condition_met:
    result = await call_external_api()
    if result.done:
        break
    activity.heartbeat("Invoking activity")
    await sleep(poll_interval)


# In workflow code:
workflow.execute_activity(
    polling_activity,
    PollingActivityInput(...),
    start_to_close_timeout=timedelta(seconds=60),
    heartbeat_timeout=timedelta(seconds=2),
)

To ensure that polling_activity is restarted in a timely manner, we make sure that it heartbeats on every iteration. Note that heartbeating only works if we set the heartbeat_timeout to a shorter value than the Activity start_to_close_timeout timeout

Advantage: Because the polling loop is inside the activity, this does not pollute the workflow history.

Infrequent Polling

Purpose: Infrequently (once per minute or slower) repeatedly poll an external service.

Implementation:

Define an Activity which fails (raises an exception) exactly when polling is not completed.

The polling loop is accomplished via activity retries, by setting the following Retry options:

• backoff_coefficient: to 1
• initial_interval: to the polling interval (e.g. 60 seconds)

This will enable the Activity to be retried exactly on the set interval.

Advantage: Individual Activity retries are not recorded in Workflow History, so this approach can poll for a very long time without affecting the history size.

Idempotency Patterns

Purpose: Ensure activities can be safely retried and replayed without causing duplicate side effects.

Why It Matters: Temporal may re-execute activities during retries (on failure) or replay (on worker restart). Without idempotency, this can cause duplicate charges, duplicate emails, duplicate database entries, etc.

Using Idempotency Keys

Pass a unique identifier to external services so they can detect and deduplicate repeated requests:

Activity: charge_payment(order_id, amount)
    │
    └── Call payment API with:
            amount: $100
            idempotency_key: "order-{order_id}"
        │
        └── Payment provider deduplicates based on key
            (second call with same key returns original result)

Good idempotency key sources:

• Workflow ID (unique per workflow execution)
• Business identifier (order ID, transaction ID)
• Workflow ID + activity name + attempt number

Check-Before-Act Pattern

Query the external system's state before making changes:

Activity: send_welcome_email(user_id)
    │
    ├── Check: Has welcome email been sent for user_id?
    │   │
    │   ├── YES: Return early (already done)
    │   │
    │   └── NO: Send email, mark as sent

Designing Idempotent Activities

1. Use unique identifiers as idempotency keys with external APIs 2. Check before acting: Query current state before making changes 3. Make operations repeatable: Ensure calling twice produces the same result 4. Record outcomes: Store transaction IDs or results for verification 5. Leverage external system features: Many APIs (Stripe, AWS, etc.) have built-in idempotency key support

Tracking State in Workflows

For complex multi-step operations, track completion status in workflow state:

Workflow State:
    payment_completed: false
    shipment_created: false

Run:
    if not payment_completed:
        charge_payment(...)
        payment_completed = true

    if not shipment_created:
        create_shipment(...)
        shipment_created = true

This ensures that on replay, already-completed steps are skipped.

Large Data Handling

Purpose: Handle data that exceeds Temporal's payload limits without polluting workflow history.

Limits (see references/core/gotchas.md for details):

• Max 2MB per individual payload
• Max 4MB per gRPC message
• Max 50MB for workflow history (aim for < 10MB)

Key Principle: Large data should never flow through workflow history. Activities read and write large data directly, passing only small references through the workflow.

Wrong Approach:

Workflow
    │
    ├── downloadFromStorage(ref) ──▶ returns large data (enters history)
    │
    ├── processData(largeData) ────▶ large data as argument (enters history AGAIN)
    │
    └── uploadToStorage(result) ───▶ large data as argument (enters history AGAIN)

This defeats the purpose—large data enters workflow history multiple times.

Correct Approach:

Workflow
    │
    └── processLargeData(inputRef) ──▶ returns outputRef (small string)
                │
                └── Activity internally:
                        download(inputRef) → process → upload → return outputRef

The workflow only handles references (small strings). The activity does all large data operations internally.

Implementation Pattern:

1. Accept a reference (URL, S3 key, database ID) as activity input 2. Download/fetch the large data inside the activity 3. Process the data inside the activity 4. Upload/store the result inside the activity 5. Return only a reference to the result

Other Strategies:

• Compression: Use a PayloadCodec to compress data automatically
• Chunking: Split large collections across multiple activities, each handling a subset

Activity Heartbeating

Purpose: Enable cancellation delivery and progress tracking for long-running activities.

Why Heartbeat:

1. Support activity cancellation - Cancellations are delivered to activities via heartbeat. Activities that don't heartbeat won't know they've been cancelled. 2. Resume progress after failure - Heartbeat details persist across retries, allowing activities to resume where they left off. 3. Detect stuck activities - If an activity stops heartbeating, Temporal can time it out and retry.

How Cancellation Works:

Workflow requests activity cancellation
    │
    ▼
Temporal Service marks activity for cancellation
    │
    ▼
Activity calls heartbeat()
    │
    ├── Not cancelled: heartbeat succeeds, continues
    │
    └── Cancelled: heartbeat raises exception
            Activity can catch this to perform cleanup

Key Point: If an activity never heartbeats, it will run to completion even if cancelled—it has no way to learn about the cancellation.

Local Activities

Purpose: Reduce latency for short, lightweight operations by skipping the task queue. ONLY use these when necessary for performance. Do NOT use these by default, as they are not durable and distributed.

When to Use:

• Short operations completing in milliseconds/seconds
• High-frequency calls where task queue overhead is significant
• Low-latency requirements where you can't afford task queue round-trip

Characteristics:

• Executes on the same worker that runs the workflow
• No task queue round-trip (lower latency)
• Still recorded in history
• Should complete quickly (default timeout is short)

Trade-offs:

• Less visibility in Temporal UI (no separate task)
• Must complete on the same worker
• Not suitable for long-running operations
• Risk with consecutive local activities: Local activity completions are only persisted when the current Workflow Task completes. Calling multiple local activities in a row (with nothing in between to yield the Workflow Task) increases the risk of losing work if the worker crashes mid-sequence. If you need a chain of operations with durable checkpoints between each step, use regular activities instead.

Choosing Between Patterns

Task Queue Priority and Fairness

Overview

Priority and Fairness control how Tasks are distributed within a Task Queue. Priority determines execution order. Fairness prevents one group of Tasks from starving others. They can be used independently or together.

Both features are in Public Preview. Priority is free. Fairness is a paid feature in Temporal Cloud.

Priority

Priority lets you control execution order within a single Task Queue by assigning a priority key (integer 1-5, lower = higher priority). Each priority level acts as a sub-queue. All priority-1 Tasks dispatch before priority-2, and so on. Tasks at the same priority level dispatch in FIFO order.

Default priority is 3. Activities inherit their parent workflow's priority unless explicitly overridden.

When to use Priority

Use Priority to differentiate execution order between types of work sharing a single Task Queue and Worker pool. For example, process payment-related Tasks before less time-sensitive inventory management Tasks, or ensure real-time Tasks run ahead of batch Tasks. You can also use it to run urgent Tasks immediately by assigning them priority 1.

CLI

temporal workflow start \
  --type ChargeCustomer \
  --task-queue my-task-queue \
  --workflow-id my-workflow-id \
  --input '{"customerId":"12345"}' \
  --priority-key 1

Go

workflowOptions := client.StartWorkflowOptions{
  ID:        "my-workflow-id",
  TaskQueue: "my-task-queue",
  Priority:  temporal.Priority{PriorityKey: 1},
}
we, err := c.ExecuteWorkflow(context.Background(), workflowOptions, MyWorkflow)

Java

WorkflowOptions options = WorkflowOptions.newBuilder()
  .setTaskQueue("my-task-queue")
  .setPriority(Priority.newBuilder().setPriorityKey(1).build())
  .build();

Python

await client.start_workflow(
  MyWorkflow.run,
  args="hello",
  id="my-workflow-id",
  task_queue="my-task-queue",
  priority=Priority(priority_key=1),
)

TypeScript

const handle = await startWorkflow(workflows.myWorkflow, {
  args: [false, 1],
  priority: { priorityKey: 1 },
});

.NET

var handle = await Client.StartWorkflowAsync(
  (MyWorkflow wf) => wf.RunAsync("hello"),
  new StartWorkflowOptions(id: "my-workflow-id", taskQueue: "my-task-queue")
  {
    Priority = new Priority(1),
  }
);

Fairness

Fairness prevents one group of Tasks from monopolizing Worker capacity. Each fairness key creates a "virtual queue" within the Task Queue. The server uses round-robin dispatch across virtual queues so no single key can block others, even with a much larger backlog.

When to use Fairness

Fairness solves the multi-tenant starvation problem. Without it, Tasks dispatch FIFO: if tenant-big enqueues 100k Tasks, tenant-small's 10 Tasks sit behind the entire backlog. With Fairness, each tenant gets its own virtual queue and Tasks are interleaved.

Common scenarios:

• Multi-tenant applications where large tenants should not block small ones.
• Tiered capacity bands where you want weighted distribution (e.g., 80% premium, 20% free) without limiting overall throughput when one band is empty.
• Batch jobs where some jobs run far more frequently than others.
• Multi-vendor processing where a few vendors generate the majority of work.

If all your Tasks can be dispatched immediately (no backlog), you don't need Fairness.

Fairness applies at Task dispatch time and considers each Task as having equal cost until dispatch. It does not account for Tasks currently being processed by Workers. So if you look at Tasks being processed by Workers, you might not see "fairness" across tenants — for example, if tenant-big already has Tasks being processed when tenant-small's Tasks are dispatched, it may still appear that tenant-big is using the most resources.

Fairness keys and weights

A fairness key is a string, typically a tenant ID or workload category. Each unique key creates a virtual queue.

A fairness weight (float, default 1.0) controls how often a key's Tasks are dispatched relative to others. A key with weight 2.0 dispatches twice as often as keys with weight 1.0.

Example with three tiers:

Tasks without a fairness key are grouped under an implicit empty-string key with weight 1.0. Adoption is incremental: unkeyed Tasks participate in round-robin alongside keyed Tasks.

Using Fairness with Priority

When combined, Priority determines which sub-queue Tasks go into (priority 1 before 2, etc.), and Fairness applies within each priority level.

SDK examples

CLI

temporal workflow start \
  --type ChargeCustomer \
  --task-queue my-task-queue \
  --workflow-id my-workflow-id \
  --input '{"customerId":"12345"}' \
  --priority-key 1 \
  --fairness-key tenant-123 \
  --fairness-weight 2.0

Go

workflowOptions := client.StartWorkflowOptions{
  ID:        "my-workflow-id",
  TaskQueue: "my-task-queue",
  Priority: temporal.Priority{
    PriorityKey:    1,
    FairnessKey:    "tenant-123",
    FairnessWeight: 2.0,
  },
}
we, err := c.ExecuteWorkflow(context.Background(), workflowOptions, MyWorkflow)

Activities:

ao := workflow.ActivityOptions{
  StartToCloseTimeout: time.Minute,
  Priority: temporal.Priority{
    PriorityKey:    1,
    FairnessKey:    "tenant-123",
    FairnessWeight: 2.0,
  },
}
ctx := workflow.WithActivityOptions(ctx, ao)
err := workflow.ExecuteActivity(ctx, MyActivity).Get(ctx, nil)

Java

WorkflowOptions options = WorkflowOptions.newBuilder()
  .setTaskQueue("my-task-queue")
  .setPriority(Priority.newBuilder()
    .setPriorityKey(1)
    .setFairnessKey("tenant-123")
    .setFairnessWeight(2.0)
    .build())
  .build();

Python

await client.start_workflow(
  MyWorkflow.run,
  args="hello",
  id="my-workflow-id",
  task_queue="my-task-queue",
  priority=Priority(priority_key=1, fairness_key="tenant-123", fairness_weight=2.0),
)

Activities:

await workflow.execute_activity(
  say_hello,
  "hi",
  priority=Priority(priority_key=1, fairness_key="tenant-123", fairness_weight=2.0),
  start_to_close_timeout=timedelta(seconds=5),
)

TypeScript

const handle = await startWorkflow(workflows.myWorkflow, {
  args: [false, 1],
  priority: { priorityKey: 1, fairnessKey: 'tenant-123', fairnessWeight: 2.0 },
});

.NET

var handle = await Client.StartWorkflowAsync(
  (MyWorkflow wf) => wf.RunAsync("hello"),
  new StartWorkflowOptions(id: "my-workflow-id", taskQueue: "my-task-queue")
  {
    Priority = new Priority(
      priorityKey: 1,
      fairnessKey: "tenant-123",
      fairnessWeight: 2.0
    )
  }
);

Child Workflows

Child workflows can set their own priority and fairness, overriding the parent.

Go:

cwo := workflow.ChildWorkflowOptions{
  WorkflowID: "child-workflow-id",
  TaskQueue:  "child-task-queue",
  Priority: temporal.Priority{
    PriorityKey:    1,
    FairnessKey:    "tenant-123",
    FairnessWeight: 2.0,
  },
}
ctx := workflow.WithChildOptions(ctx, cwo)
err := workflow.ExecuteChildWorkflow(ctx, MyChildWorkflow).Get(ctx, nil)

Java:

ChildWorkflowOptions childOptions = ChildWorkflowOptions.newBuilder()
  .setTaskQueue("child-task-queue")
  .setWorkflowId("child-workflow-id")
  .setPriority(Priority.newBuilder()
    .setPriorityKey(1)
    .setFairnessKey("tenant-123")
    .setFairnessWeight(2.0)
    .build())
  .build();
MyChildWorkflow child = Workflow.newChildWorkflowStub(MyChildWorkflow.class, childOptions);
child.run();

Python:

await workflow.execute_child_workflow(
  MyChildWorkflow.run,
  args="hello child",
  priority=Priority(priority_key=1, fairness_key="tenant-123", fairness_weight=2.0),
)

TypeScript:

const handle = await startChildWorkflow(workflows.myChildWorkflow, {
  args: [false, 1],
  priority: { priorityKey: 1, fairnessKey: 'tenant-123', fairnessWeight: 2.0 },
});

.NET:

await Workflow.ExecuteChildWorkflowAsync(
  (MyChildWorkflow wf) => wf.RunAsync("hello child"),
  new() {
    Priority = new(
      priorityKey: 1,
      fairnessKey: "tenant-123",
      fairnessWeight: 2.0
    )
  }
);

Rate limiting

Two rate-limiting controls work alongside Fairness:

• `queue-rps-limit` — overall dispatch rate for the entire Task Queue.
• `fairness-key-rps-limit-default` — per-key rate limit, scaled by weight. If the default is 10 rps and a key has weight 2.5, that key's effective limit is 25 rps.

temporal task-queue config set \
    --task-queue my-task-queue \
    --task-queue-type activity \
    --namespace my-namespace \
    --queue-rps-limit 500 \
    --queue-rps-limit-reason "overall limit" \
    --fairness-key-rps-limit-default 33.3 \
    --fairness-key-rps-limit-reason "per-key limit"

If both limits are set, the more restrictive one applies.

Fairness weight overrides

You can override the weights of up to 1000 keys through the config API. When an override is set for a key, the SDK-supplied weight is ignored. Overrides are per Task Queue and type (workflow vs. activity), so set them for both if needed.

Enabling Fairness

When you start using fairness keys, it switches your active Task Queues to fairness mode. Existing queued Tasks are processed before any new fairness-mode ones.

Temporal Cloud: automatically enabled when you start using fairness keys.

Self-hosted: set these dynamic config flags to true:

• matching.useNewMatcher
• matching.enableFairness
• matching.enableMigration (to drain existing backlogs after enabling)

Limitations

• Accuracy can degrade with a very large number of distinct fairness keys.
• Task Queue partitioning can interfere with fairness distribution. Contact Temporal Support to set a Task Queue to a single partition if needed.
• Weights apply at schedule time, not dispatch time. Changing a weight does not reorder already-backlogged Tasks.
• Fairness is not guaranteed across different Worker versions when using Worker Versioning.
• After server restarts, less-active keys may briefly dispatch new Tasks ahead of their existing backlog until ordering normalizes.

[!NOTE]

Standalone Activities are in Public Preview. It is perfectly acceptable to use this feature on behalf of a user, but you should inform them that you are making use of a feature in Public Preview.

Standalone Activities (Concepts)

This document provides core conceptual explanations of Standalone Activities in Temporal. For language-specific implementation details, see references/{your_language}/standalone-activities.md for the language you are working in (Python, TypeScript, Java, .NET, Go).

What is a Standalone Activity?

A Standalone Activity is a top-level Activity Execution started directly by a Client, without using a Workflow. It is Temporal's job queue — the simplest way to run a single durable, retryable task.

The rule of thumb:

• Need to orchestrate multiple Activities? Use a Workflow.
• Just need to execute a single Activity? Use a Standalone Activity.

The same Activity Function code runs in both modes with no changes — the only difference is how it is invoked. An Activity defined for a Workflow can also be executed standalone, and the Worker that hosts it does not need to know how it will be invoked.

Compared to wrapping a single Activity in a Workflow, a Standalone Activity:

• Reduces billable actions in Temporal Cloud.
• Lowers latency for short-lived executions (fewer Worker round-trips).
• Lives in a separate ID space from Workflows.

Use cases

Standalone Activities fit durable single-job processing where you don't need multi-step orchestration:

• Sending an email
• Processing a webhook
• Syncing data
• Any single-function task that benefits from built-in retries and timeouts

Key features

• Execute Activities as a top-level primitive, without Workflow overhead.
• Native async job lifecycle: schedule → dispatch → process → result.
• Arbitrary-length jobs, with heartbeats for progress tracking.
• At-least-once execution by default, with native retry policy and timeouts.
• At-most-once execution when the retry policy's maximum attempts is 1.
• Addressable by Activity ID / Run ID for result retrieval, cancellation, and termination.
• Deduplication via configurable conflict policies.
• Priority and fairness support.
• Full visibility — list and count executions.

Using Standalone Activities

Defining activities

Defining standalone activities is IDENTICAL to defining activities callable from a workflow - there is no distinction AT ALL between the two at activity definition or worker configuration site. Follow language-specific guidance for how to normally define activities and configure workers to run them.

Calling and Interacting with Standalone Activities

The CLI and every SDK exposes the same conceptual operations against a Standalone Activity (method names differ per language — see the language reference):

• Execute — durably enqueue the Activity, wait for a Worker to run it, and return the result.
• Start — durably enqueue the Activity and return a handle immediately, without waiting.
• Get handle — rebind a handle to a previously started Activity by ID (and optionally Run ID).
• Get result — wait on a handle for completion. execute is equivalent to start followed by awaiting the handle's result.
• Cancel / Terminate — via the handle or CLI.

Choosing an Activity ID. Every Standalone Activity call requires an Activity ID, which uniquely identifies that one call. It is the key you use later to get the result, describe, cancel, or terminate the Activity, and it is what conflict/reuse policies dedupe against. Use a business-logic identifier that uniquely identifies the call — for example send-welcome-email:user-42, sync-invoice:INV-2026-001, or process-webhook:<event-id>. This makes Activities addressable and naturally deduplicated by your domain. Only if you genuinely have no meaningful business-level identifier should you generate a UUID to use as the Activity ID.

Visibility operations are available as well:

• List — enumerate Standalone Activity Executions matching a query. Only Standalone Activities are returned; Activities running inside Workflows are not included.
• Count — return the total number of executions matching a query (running, completed, failed, etc. — not the number of queued tasks).
• Describe — via the handle or CLI.

See below for a quick reference how to call these operations from the CLI rather than SDKs.

[!IMPORTANT]

When using an SDK, these operations are owned by the Temporal Client, and belong in your non-workflow application code. It is INVALID to call an activity as a standalone activity from within a workflow: you instead should use standard within-workflow activity calls.

Currently Supported SDKs: Python, TypeScript, Java, .NET, Go

Quick CLI Standalone Activity Man Page

Ultimately, any standalone activity invocation code should live in your application code and use the appropriate SDK, but the Temporal CLI is a quick and easy way to test invoking standalone activities during development. All subcommands live under temporal activity.

The key operations are:

Execute (start and wait for the result). Blocks until the Activity completes and prints the result to stdout. Requires --activity-id, --type, --task-queue, and at least one of --start-to-close-timeout / --schedule-to-close-timeout:

temporal activity execute \
  --activity-id my-activity-id \
  --type ComposeGreeting \
  --task-queue my-task-queue \
  --start-to-close-timeout 10s \
  --input '{"some-key": "some-value"}'

--input takes a JSON value; pass it multiple times for multiple positional arguments. --input-file is also a convenient option for larger inputs. The same required flags apply to start below.

Reminder: --activity-id must be unique across all activity calls, as discussed above.

Start (do not wait). Enqueues the Activity and prints the Activity ID and Run ID without blocking:

temporal activity start \
  --activity-id my-activity-id \
  --type ComposeGreeting \
  --task-queue my-task-queue \
  --start-to-close-timeout 10s \
  --input '{"some-key": "some-value"}'

Outputs this JSON shape:

{
  "activityId": "my-activity-id",
  "runId": "019e84d3-949a-7a0e-ae78-63b8a0b172bd",
  "namespace": "default"
}

Result (wait for a started Activity). Waits for completion and prints the result. --run-id is optional and defaults to the latest run of that Activity ID:

temporal activity result --activity-id my-activity-id

Describe (current state of one Activity). Shows status, run state, task queue, timeouts, attempt count, etc.:

temporal activity describe --activity-id my-activity-id

List / Count (visibility across many Activities). Only Standalone Activity Executions are returned (Activities running inside Workflows are not):

temporal activity list
temporal activity count

Cancel / Terminate (stop an Activity). cancel requests cooperative cancellation (surfaced to the Activity on its next heartbeat response); terminate forcefully ends it (Activity code cannot see or respond to it). Both accept --reason:

temporal activity cancel    --activity-id my-activity-id --reason "no longer needed"
temporal activity terminate --activity-id my-activity-id --reason "no longer needed"

Observability

All existing Activity metrics apply to Standalone Activities (scheduled, started, completed, failed, timed out, canceled).

Public Preview limitations

• Pause, reset, and update options are not supported (scheduled for GA).
• The TerminateExisting conflict policy and TerminateIfRunning reuse policy are not yet supported.

Temporal CLI support

• Requires Temporal CLI v1.7.0+ and Temporal Server v1.31.0+. See references/core/install_cli.md if you need to update the CLI.
• The Temporal Dev Server (temporal server start-dev) has Standalone Activities enabled by default.

Temporal Cloud support

Standalone Activities are available in Temporal Cloud as a Public Preview feature. Because the SDK client config loaders read environment variables and TOML profiles, the same code runs against a local server or Temporal Cloud with no code changes.

Temporal Troubleshooting Guide

Workflow Diagnosis Decision Tree

Workflow not behaving as expected?
│
├─▶ What is the workflow status?
│   │
│   ├─▶ RUNNING (but no progress)
│   │   └─▶ Go to: "Workflow Stuck" section
│   │
│   ├─▶ FAILED
│   │   └─▶ Go to: "Workflow Failed" section
│   │
│   ├─▶ TIMED_OUT
│   │   └─▶ Go to: "Timeout Issues" section
│   │
│   └─▶ COMPLETED (but wrong result)
│       └─▶ Go to: "Wrong Result" section

Workflow Stuck (RUNNING but No Progress)

Decision Tree

Workflow stuck in RUNNING?
│
├─▶ Is a worker running?
│   │
│   ├─▶ NO: Start a worker
│   │   └─▶ See references/core/dev-management.md
│   │
│   └─▶ YES: Is it on the correct task queue?
│       │
│       ├─▶ NO: Start worker with correct task queue
│       │
│       └─▶ YES: Check for non-determinism
│           │
│           ├─▶ NondeterminismError in logs?
│           │   └─▶ Go to: "Non-Determinism" section
│           │
│           ├─▶ Check history for task failures
│           │   └─▶ Run: `temporal workflow show --workflow-id <id>`
│           │       │
│           │       ├─▶ WorkflowTaskFailed event?
│           │       │   └─▶ Check error type in event details
│           │       │       └─▶ Go to relevant section in error-reference.md
│           │       │
│           │       └─▶ ActivityTaskFailed event?
│           │           └─▶ Go to: "Activity Keeps Retrying" section
│           │
│           └─▶ No errors in logs or history?
│               └─▶ Check if workflow is waiting for signal/timer

Common Causes

1. No worker running

• See references/core/dev-management.md

2. Worker on wrong task queue

• Check: Worker logs for task queue name
• Fix: Start worker with matching task queue

3. Worker has stale code

• Check: Worker startup time vs code changes
• Fix: Restart worker with updated code

4. Workflow waiting for signal

• Check: Workflow history for pending signals
• Fix: Send expected signal or check signal sender

5. Activity stuck/timing out

• Check: Activity retry attempts in history
• Fix: Investigate activity failure, increase timeout

Non-Determinism Errors

Decision Tree

NondeterminismError?
│
├─▶ Was code intentionally changed?
│   │
│   ├─▶ YES: Do you need to support in-flight workflows?
│   │   │
│   │   ├─▶ YES (production): Use patching API
│   │   │   └─▶ See: references/core/versioning.md
│   │   │
│   │   └─▶ NO (local dev/testing): Terminate or reset workflow
│   │       └─▶ `temporal workflow terminate --workflow-id <id>`
│   │       └─▶ Then start fresh with new code
│   │
│   └─▶ NO: Accidental change
│       │
│       ├─▶ Can you identify the change?
│       │   │
│       │   ├─▶ YES: Revert and restart worker. Note, this doesn't always work if workflow has progressed past the change (may induce other code paths), so may need to reset workflow.
│       │   │
│       │   └─▶ NO: Compare current code to expected history
│       │       └─▶ Check: Activity names, order, parameters

Common Causes

1. Changed call order

   # Before           # After (BREAKS)
   await activity_a   await activity_b
   await activity_b   await activity_a

2. Changed call name

   # Before                    # After (BREAKS)
   await process_order(...)    await handle_order(...)

3. Added/removed call

• Adding new activity mid-workflow
• Removing activity that was previously called

4. Using non-deterministic code

• datetime.now() in workflow (use workflow.now())
• random.random() in workflow (use workflow.random())

Recovery

Accidental Change:

1. Identify the change 2. Revert code to match history 3. Restart worker 4. Workflow automatically recovers

Intentional Change:

1. Use patching API for gradual migration 2. Or terminate old workflows, start new ones

Workflow Failed

Decision Tree

Workflow status = FAILED?
│
├─▶ Check workflow error message
│   │
│   ├─▶ Application error (your code)
│   │   └─▶ Fix the bug, start new workflow
│   │
│   ├─▶ NondeterminismError
│   │   └─▶ Go to: "Non-Determinism" section
│   │
│   └─▶ Timeout error
│       └─▶ Go to: "Timeout Issues" section

Common Causes

1. Unhandled exception in workflow

• Check error message and stack trace
• Fix bug in workflow code

2. Activity exhausted retries

• All retry attempts failed
• Check activity logs for root cause

3. Non-retryable error thrown

• Error marked as non-retryable
• Intentional failure, check business logic

Timeout Issues

Timeout Types

Diagnosis

Timeout error?
│
├─▶ Which timeout?
│   │
│   ├─▶ Workflow timeout
│   │   └─▶ Increase timeout or optimize workflow. Better yet, consider removing the workflow timeout, as it is generally discouraged unless *necessary* for your use case.
│   │
│   ├─▶ ScheduleToCloseTimeout
│   │   └─▶ Activity taking too long overall (including retries)
│   │
│   ├─▶ StartToCloseTimeout
│   │   └─▶ Single activity attempt too slow
│   │
│   └─▶ HeartbeatTimeout
│       └─▶ Activity not heartbeating frequently enough
│           └─▶ Add heartbeat() calls in long activities

Fixes

1. Increase timeout if operation legitimately takes longer 2. Add heartbeats to long-running activities 3. Optimize activity to complete faster 4. Break into smaller activities for better granularity

Activity Keeps Retrying

Decision Tree

Activity retrying repeatedly?
│
├─▶ Check activity error
│   │
│   ├─▶ Transient error (network, timeout)
│   │   └─▶ Expected behavior, will eventually succeed
│   │
│   ├─▶ Permanent error (bug, invalid input)
│   │   └─▶ Fix the bug or mark as non-retryable
│   │
│   └─▶ Resource exhausted
│       └─▶ Add backoff, check rate limits

Common Causes

1. Bug in activity code

• Fix the bug
• Consider marking certain errors as non-retryable

2. External service down

• Retries are working as intended
• Monitor service recovery

3. Invalid input

• Validate inputs before activity
• Return non-retryable error for bad input

Wrong Result (Completed but Incorrect)

Diagnosis

1. Check workflow history for unexpected activity results 2. Verify activity implementations produce correct output 3. Check for race conditions in parallel execution 4. Verify signal handling if signals are involved

Common Causes

1. Activity bug - Wrong logic in activity 2. Stale data - Activity using outdated information 3. Signal ordering - Signals processed in unexpected order 4. Parallel execution - Race condition in concurrent operations

Worker Issues

Worker Not Starting

Worker won't start?
│
├─▶ Connection error
│   └─▶ Check Temporal server is running
│       └─▶ `temporal server start-dev` (start in background, see references/core/dev-management.md)
│
├─▶ Registration error
│   └─▶ Check workflow/activity definitions are valid
│
└─▶ Other errors (imports, etc.)
    └─▶ Debug those errors as usual.

Worker Crashing

1. Out of memory - Reduce concurrent tasks, check for leaks 2. Unhandled exception - Add error handling 3. Dependency issue - Check package versions

Useful Commands

# Check Temporal server
temporal server start-dev

# List workflows
temporal workflow list

# Describe specific workflow
temporal workflow describe --workflow-id <id>

# Show workflow history
temporal workflow show --workflow-id <id>

# Terminate stuck workflow
temporal workflow terminate --workflow-id <id>

# Reset workflow to specific point
temporal workflow reset --workflow-id <id> --event-id <event-id>

Quick Reference: Status → Action

See Also

• Common Gotchas - Anti-patterns that cause these issues
• Error Reference - Quick error type lookup

Workflow Versioning Concepts

This document provides core conceptual explanations of workflow versioning in Temporal. For language-specific implementation details see references/{your_language}/versioning.md, for the language you are working in.

Overview

Workflow versioning allows safe deployment of code changes without breaking running workflows. Three approaches available:

1. Patching API - Code-level version branching 2. Workflow Type Versioning - New workflow types for incompatible changes 3. Worker Versioning - Deployment-level control with Build IDs

Why Versioning is Needed

When workers restart after deployment, they resume open workflows through history replay. If updated code produces different Commands than the original code, it causes non-determinism errors.

Original Code (recorded in history):
  await activity_a()
  await activity_b()

Updated Code (during replay):
  await activity_a()
  await activity_c()  ← Different! NondeterminismError

Approach 1: Patching API

Concept

The patching API lets you branch code based on whether a workflow was started before or after a code change.

if patched("my-change"):
    // New code path (for new and replaying new workflows)
else:
    // Old code path (for replaying old workflows)

Three-Phase Lifecycle

Phase 1: Patch In

• Add both old and new code paths
• New workflows take new path, old workflows take old path

Phase 2: Deprecate

• After all old workflows complete, remove old code
• Keep deprecation marker for history compatibility

Phase 3: Remove

• After all deprecated workflows complete
• Remove patch entirely, only new code remains

When to Use

• Adding, removing, or reordering activities/child workflows
• Changing which activity/child workflow is called
• Any change that alters the Command sequence

When NOT to Use

• Changing activity implementations (activities aren't replayed)
• Changing arguments passed to activities or child workflows
• Changing retry policies
• Changing timer durations
• Adding new signal/query/update handlers (additive changes are safe)
• Bug fixes that don't change Command sequence

Unnecessary patching adds complexity and can make workflow code unmanageable.

Approach 2: Workflow Type Versioning

Concept

Create a new workflow type (e.g., OrderWorkflowV2) instead of patching.

// Old: OrderWorkflow
// New: OrderWorkflowV2 (completely new implementation)

When to Use

• Major incompatible changes
• Complete rewrites
• When patching would be too complex
• When you want clean separation

Process

1. Create new workflow type with new name 2. Register both with worker 3. Start new workflows with new type 4. Wait for old workflows to complete 5. Remove old workflow type

Approach 3: Worker Versioning

Concept

Manage versions at deployment level using Build IDs. Multiple worker versions can run simultaneously.

Worker v1.0 (Build ID: abc123)
  └── Handles workflows started on this version

Worker v2.0 (Build ID: def456)
  └── Handles new workflows
  └── Can also handle upgraded old workflows

Key Concepts

Worker Deployment: Logical service grouping (e.g., "order-service")

Build ID: Specific code version (e.g., git commit hash)

Versioning Behaviors:

• PINNED - Workflows stay on original worker version
• AUTO_UPGRADE - Workflows can move to newer versions

When to Use PINNED

• Short-running workflows (minutes to hours)
• Consistency is critical
• Want simplest development experience
• Building new applications

When to Use AUTO_UPGRADE

• Long-running workflows (weeks or months)
• Workflows need bug fixes during execution
• Still requires patching for version transitions

Upgrading on Continue-as-New

[!NOTE]

This feature is in Public Preview. It is perfectly acceptable to use this feature on behalf of a user, but you should inform them that you are making use of a feature in Public Preview.

Long-running Pinned Workflows that use Continue-as-New can upgrade to newer Worker Deployment Versions at the Continue-as-New boundary without patching.

This pattern is for:

• Entity Workflows that run for months or years
• Batch processing Workflows that checkpoint with Continue-as-New
• AI agent Workflows with long sleeps waiting for user input

How it works

By default, Pinned Workflows stay on their original Worker Deployment Version even when they Continue-as-New. With the upgrade option enabled:

1. Each Workflow run remains pinned to its version (no patching needed during a run). 2. The Temporal Server tells the Workflow when a new Target Version becomes available — that is, when the Workflow's Worker Deployment gets a new Current or Ramping Version that the Workflow would move to next. 3. When the Workflow performs Continue-as-New with the upgrade option, the new run starts on the Target Version.

Detection flag

Active Workflows detect a Target Version change by checking a per-Workflow flag exposed on WorkflowInfo (called target_worker_deployment_version_changed in the docs). The flag is refreshed after each Workflow Task completes; check it from code that runs as part of a Workflow Task (for example, before accepting an Update, starting an Activity, or starting a child Workflow). See the per-language references/{your_language}/versioning.md for the SDK-specific call.

Triggering the new run

When the flag is set, return a Continue-as-New error with the new run's initial Versioning Behavior set to AutoUpgrade. This makes the new run start on the Target Version of its Worker Deployment. The Workflow Type itself retains its Pinned annotation; only the initial behavior of the new run is overridden so it picks up the Target Version. Once the new run is on the new version, the per-Workflow-type annotation continues to apply on subsequent CaN.

Limitations

• Lazy moving only — sleeping Workflows do not auto-upgrade. Send a Signal to wake an idle Workflow so it can check the flag.
• Interface compatibility is your responsibility. When continuing as new to a different version, the previous version's Workflow input must be compatible with the new version's Workflow definition. If incompatible, the new run may fail on its first Workflow Task.
• Pinned Workflows only. Auto-Upgrade Workflows already move to the Target Version at Workflow Task boundaries; this pattern adds nothing for them.

When to use this pattern

• Workflow Type is Pinned and
• Workflow runs longer than your Worker Deployment Version lifetime and
• Workflow already uses Continue-as-New to bound Event History size.

For long-running Workflows that cannot use Continue-as-New (e.g., compliance audits that need full history), use AUTO_UPGRADE with patching instead.

Choosing an Approach

Best Practices

1. Check for open executions before removing old code 2. Use descriptive patch IDs (e.g., "add-fraud-check" not "patch-1") 3. Deploy incrementally: patch → deprecate → remove 4. Test replay compatibility before deploying changes 5. Monitor old workflow counts during migration

Finding Workflows by Version

# Find workflows with specific patch
temporal workflow list --query \
  'WorkflowType = "OrderWorkflow" AND TemporalChangeVersion = "add-fraud-check"'

# Find pre-patch workflows
temporal workflow list --query \
  'WorkflowType = "OrderWorkflow" AND TemporalChangeVersion IS NULL'

# Find workflows on specific worker version
temporal workflow list --query \
  'TemporalWorkerDeploymentVersion = "my-service:v1.0.0"'

Common Mistakes

1. Removing old code too early - Breaks replaying workflows 2. Not testing with replay - Catches issues before production 3. Patching non-Command changes - Unnecessary complexity 4. Forgetting to deprecate - Accumulates dead code

.NET SDK Advanced Features

Schedules

Create recurring workflow executions.

using Temporalio.Client.Schedules;

var scheduleId = "daily-report";
await client.CreateScheduleAsync(
    scheduleId,
    new Schedule(
        Action: ScheduleActionStartWorkflow.Create(
            (DailyReportWorkflow wf) => wf.RunAsync(),
            new(id: "daily-report", taskQueue: "reports")),
        Spec: new ScheduleSpec
        {
            Intervals = new List<ScheduleIntervalSpec>
            {
                new(Every: TimeSpan.FromDays(1)),
            },
        }));

// Manage schedules
var handle = client.GetScheduleHandle(scheduleId);
await handle.PauseAsync("Maintenance window");
await handle.UnpauseAsync();
await handle.TriggerAsync(); // Run immediately
await handle.DeleteAsync();

Async Activity Completion

For activities that complete asynchronously (e.g., human tasks, external callbacks). If you configure a HeartbeatTimeout on this activity, the external completer is responsible for sending heartbeats via the async handle. If you do NOT set a HeartbeatTimeout, no heartbeats are required.

Note: If the external system that completes the asynchronous action can reliably be trusted to do the task and Signal back with the result, and it doesn't need to Heartbeat or receive Cancellation, then consider using signals instead.

using Temporalio.Activities;
using Temporalio.Client;

[Activity]
public async Task RequestApprovalAsync(string requestId)
{
    var taskToken = ActivityExecutionContext.Current.Info.TaskToken;

    // Store task token for later completion (e.g., in database)
    await StoreTaskTokenAsync(requestId, taskToken);

    // Mark this activity as waiting for external completion
    throw new CompleteAsyncException();
}

// Later, complete the activity from another process
public async Task CompleteApprovalAsync(string requestId, bool approved)
{
    var client = await TemporalClient.ConnectAsync(new("localhost:7233"));
    // Retrieve the task token from external storage (e.g., database)
    var taskToken = await GetTaskTokenAsync(requestId);

    var handle = client.GetAsyncActivityHandle(taskToken);

    // Optional: if a HeartbeatTimeout was set, you can periodically:
    // await handle.HeartbeatAsync(progressDetails);

    if (approved)
        await handle.CompleteAsync("approved");
    else
        // You can also fail or report cancellation via the handle
        await handle.FailAsync(new ApplicationFailureException("Rejected"));
}

Worker Tuning

Configure worker performance settings.

var worker = new TemporalWorker(
    client,
    new TemporalWorkerOptions("my-task-queue")
    {
        // Workflow task concurrency
        MaxConcurrentWorkflowTasks = 100,
        // Activity task concurrency
        MaxConcurrentActivities = 100,
        // Graceful shutdown timeout
        GracefulShutdownTimeout = TimeSpan.FromSeconds(30),
    }
    .AddWorkflow<MyWorkflow>()
    .AddAllActivities(new MyActivities()));

Workflow Init Attribute

You should always put state initialization logic in the constructor of your workflow class, so that it happens before signals/updates arrive.

Normally, your constructor must have no arguments. However, if you add the [WorkflowInit] attribute, then your constructor instead receives the same workflow arguments that [WorkflowRun] receives:

[Workflow]
public class MyWorkflow
{
    private readonly string _initialValue;
    private readonly List<string> _items = new();

    [WorkflowInit]
    public MyWorkflow(string initialValue)
    {
        _initialValue = initialValue;
    }

    [WorkflowRun]
    public async Task<string> RunAsync(string initialValue)
    {
        // _initialValue and _items are already initialized
        return _initialValue;
    }
}

Constructor (with [WorkflowInit]) and [WorkflowRun] method must have the same parameters with the same types. You cannot make blocking calls (activities, sleeps, etc.) from the constructor.

Workflow Failure Exception Types

Control which exceptions cause workflow failures vs workflow task retries.

Default behavior: Only ApplicationFailureException fails a workflow. All other exceptions retry the workflow task forever (treated as bugs to fix with a code deployment).

Tip for testing: Set WorkflowFailureExceptionTypes to include Exception so any unhandled exception fails the workflow immediately rather than retrying the workflow task forever. This surfaces bugs faster.

Worker-Level Configuration

var worker = new TemporalWorker(
    client,
    new TemporalWorkerOptions("my-task-queue")
    {
        // These exception types will fail the workflow execution (not just the task)
        WorkflowFailureExceptionTypes = new[] { typeof(ArgumentException), typeof(InvalidOperationException) },
    }
    .AddWorkflow<MyWorkflow>()
    .AddAllActivities(new MyActivities()));

Dependency Injection

The .NET SDK supports dependency injection via the Temporalio.Extensions.Hosting package, which integrates with .NET's generic host.

Worker as Generic Host

using Temporalio.Extensions.Hosting;

public class Program
{
    public static async Task Main(string[] args)
    {
        var host = Host.CreateDefaultBuilder(args)
            .ConfigureServices(ctx =>
                ctx.
                    AddScoped<IOrderRepository, OrderRepository>().
                    AddHostedTemporalWorker(
                        clientTargetHost: "localhost:7233",
                        clientNamespace: "default",
                        taskQueue: "my-task-queue").
                    AddScopedActivities<MyActivities>().
                    AddWorkflow<MyWorkflow>())
            .Build();
        await host.RunAsync();
    }
}

Activity Dependency Injection

As shown in the host setup above, activities can be registered with AddScopedActivities<T>(), AddSingletonActivities<T>(), or AddTransientActivities<T>(). Activities registered this way are created via DI, allowing constructor injection:

public class MyActivities
{
    private readonly ILogger<MyActivities> _logger;
    private readonly IOrderRepository _repository;

    public MyActivities(ILogger<MyActivities> logger, IOrderRepository repository)
    {
        _logger = logger;
        _repository = repository;
    }

    [Activity]
    public async Task<Order> GetOrderAsync(string orderId)
    {
        _logger.LogInformation("Fetching order {OrderId}", orderId);
        return await _repository.GetAsync(orderId);
    }
}

Note: Dependency injection is NOT available in workflows — workflows must be self-contained for determinism.

.NET SDK Data Handling

Overview

The .NET SDK uses data converters to serialize/deserialize workflow inputs, outputs, and activity parameters.

Default Data Converter

The default converter handles:

• null
• byte[] (as binary)
• Google.Protobuf.IMessage instances
• Anything that System.Text.Json supports
• IRawValue as unconverted raw payloads

Custom Data Converter

Customize serialization by extending DefaultPayloadConverter. For example, to use camelCase property naming:

using System.Text.Json;
using Temporalio.Client;
using Temporalio.Converters;

public class CamelCasePayloadConverter : DefaultPayloadConverter
{
    public CamelCasePayloadConverter()
        : base(new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase })
    {
    }
}

var client = await TemporalClient.ConnectAsync(new()
{
    TargetHost = "localhost:7233",
    Namespace = "my-namespace",
    DataConverter = DataConverter.Default with
    {
        PayloadConverter = new CamelCasePayloadConverter(),
    },
});

Protobuf Support

The default data converter includes built-in support for Protocol Buffer messages via Google.Protobuf.IMessage. Protobuf messages are automatically serialized using proto3 JSON.

// Any Google.Protobuf.IMessage is automatically handled
[Workflow]
public class MyWorkflow
{
    [WorkflowRun]
    public async Task<MyProtoResponse> RunAsync(MyProtoRequest request)
    {
        // Protobuf messages are serialized/deserialized automatically
        return await Workflow.ExecuteActivityAsync(
            (MyActivities a) => a.ProcessAsync(request),
            new() { StartToCloseTimeout = TimeSpan.FromMinutes(5) });
    }
}

Payload Encryption

Encrypt sensitive workflow data using a custom IPayloadCodec:

using Temporalio.Converters;
using Google.Protobuf;

public class EncryptionCodec : IPayloadCodec
{
    public Task<IReadOnlyCollection<Payload>> EncodeAsync(
        IReadOnlyCollection<Payload> payloads) =>
        Task.FromResult<IReadOnlyCollection<Payload>>(payloads.Select(p =>
            new Payload
            {
                Metadata = { ["encoding"] = "binary/encrypted" },
                Data = ByteString.CopyFrom(Encrypt(p.ToByteArray())),
            }).ToList());

    public Task<IReadOnlyCollection<Payload>> DecodeAsync(
        IReadOnlyCollection<Payload> payloads) =>
        Task.FromResult<IReadOnlyCollection<Payload>>(payloads.Select(p =>
        {
            if (p.Metadata.GetValueOrDefault("encoding") != "binary/encrypted")
                return p;
            return Payload.Parser.ParseFrom(Decrypt(p.Data.ToByteArray()));
        }).ToList());

    private byte[] Encrypt(byte[] data) => /* your encryption logic */;
    private byte[] Decrypt(byte[] data) => /* your decryption logic */;
}

// Apply encryption codec
var client = await TemporalClient.ConnectAsync(new("localhost:7233")
{
    DataConverter = DataConverter.Default with
    {
        PayloadCodec = new EncryptionCodec(),
    },
});

Search Attributes

Custom searchable fields for workflow visibility. These can be set at workflow start:

using Temporalio.Common;

var handle = await client.StartWorkflowAsync(
    (OrderWorkflow wf) => wf.RunAsync(order),
    new(id: $"order-{order.Id}", taskQueue: "orders")
    {
        TypedSearchAttributes = new SearchAttributeCollection.Builder()
            .Set(SearchAttributeKey.CreateKeyword("OrderId"), order.Id)
            .Set(SearchAttributeKey.CreateKeyword("OrderStatus"), "pending")
            .Set(SearchAttributeKey.CreateFloat("OrderTotal"), order.Total)
            .Build(),
    });

Or upserted during workflow execution:

[Workflow]
public class OrderWorkflow
{
    [WorkflowRun]
    public async Task<string> RunAsync(Order order)
    {
        // ... process order ...

        // Update search attribute
        Workflow.UpsertTypedSearchAttributes(
            SearchAttributeKey.CreateKeyword("OrderStatus").ValueSet("completed"));
        return "done";
    }
}

Querying Workflows by Search Attributes

await foreach (var wf in client.ListWorkflowsAsync(
    "OrderStatus = \"processing\" OR OrderStatus = \"pending\""))
{
    Console.WriteLine($"Workflow {wf.Id} is still processing");
}

Workflow Memo

Store arbitrary metadata with workflows (not searchable).

await client.ExecuteWorkflowAsync(
    (OrderWorkflow wf) => wf.RunAsync(order),
    new(id: $"order-{order.Id}", taskQueue: "orders")
    {
        Memo = new Dictionary<string, object>
        {
            ["customer_name"] = order.CustomerName,
            ["notes"] = "Priority customer",
        },
    });

// Read memo from workflow
[Workflow]
public class OrderWorkflow
{
    [WorkflowRun]
    public async Task<string> RunAsync(Order order)
    {
        var notes = Workflow.Memo["notes"];
        // ...
    }
}

Deterministic APIs for Values

Use these APIs within workflows for deterministic random values and UUIDs:

[Workflow]
public class MyWorkflow
{
    [WorkflowRun]
    public async Task<string> RunAsync()
    {
        // Deterministic GUID (same on replay)
        var uniqueId = Workflow.NewGuid();

        // Deterministic random (same on replay)
        var value = Workflow.Random.Next(1, 100);

        // Deterministic current time
        var now = Workflow.UtcNow;

        return uniqueId.ToString();
    }
}

Best Practices

1. Use records or classes with System.Text.Json support for input/output 2. Keep payloads small — see references/core/gotchas.md for limits 3. Encrypt sensitive data with IPayloadCodec 4. Use Workflow.NewGuid() and Workflow.Random for deterministic values 5. Use camelCase converter if interoperating with other SDKs

.NET Determinism Protection

Overview

The .NET SDK has no runtime sandbox. Determinism is enforced by developer convention and runtime task detection. Unlike the Python and TypeScript SDKs, the .NET SDK will not intercept or replace non-deterministic calls at compile time or import time. The SDK does provide a runtime EventListener that detects some invalid task scheduling, but catching all non-deterministic code requires following the rules below and testing, in particular replay tests (see references/dotnet/testing.md).

Runtime Task Detection

By default, the .NET SDK enables an EventListener that monitors task events. When workflow code accidentally starts a task on the wrong scheduler (e.g., via Task.Run), an InvalidWorkflowOperationException is thrown. This causes the workflow task to fail, which will continuously retry until the code is fixed.

// This will be detected at runtime and fail the workflow task
[Workflow]
public class BadWorkflow
{
    [WorkflowRun]
    public async Task RunAsync()
    {
        // BAD: Task.Run uses TaskScheduler.Default
        await Task.Run(() => DoSomething());
    }
}

.NET Task Determinism Rules

Many .NET Task APIs implicitly use TaskScheduler.Default, which breaks determinism. Here are the key rules:

Do NOT use:

• Task.Run — uses default scheduler. Use Workflow.RunTaskAsync.
• Task.ConfigureAwait(false) — leaves current context. Use ConfigureAwait(true) or omit.
• Task.Delay / Task.Wait / timeout-based CancellationTokenSource — uses system timers. Use Workflow.DelayAsync / Workflow.WaitConditionAsync.
• Task.WhenAny — use Workflow.WhenAnyAsync.
• Task.WhenAll — use Workflow.WhenAllAsync (technically safe currently, but wrapper is recommended).
• CancellationTokenSource.CancelAsync — use CancellationTokenSource.Cancel.
• System.Threading.Semaphore / SemaphoreSlim / Mutex — use Temporalio.Workflows.Semaphore / Mutex.

Be wary of:

• Third-party libraries that implicitly use TaskScheduler.Default
• Dataflow blocks and similar concurrency libraries with hidden default scheduler usage

Best Practices

1. *Always use `Workflow. alternatives** for Task operations in workflows 2. **Don't disable the EventListener** — it's on by default and catches mistakes at runtime 3. **Separate workflow and activity code** into different files/projects for clarity 4. **Use SortedDictionary** or sort collections before iterating — Dictionary<TKey, TValue>` iteration order is not guaranteed 5. Test with replay to catch non-determinism early 6. Review third-party library usage in workflow code for hidden default scheduler usage