Akka Net Best Practices

Akka.NET Best Practices

When to Use This Skill

Use this skill when:

• Designing actor communication patterns
• Deciding between EventStream and DistributedPubSub
• Implementing error handling in actors
• Understanding supervision strategies
• Choosing between Props patterns and DependencyResolver
• Designing work distribution across nodes
• Creating testable actor systems that can run with or without cluster infrastructure
• Abstracting over Cluster Sharding for local testing scenarios

Reference Files

• work-distribution-patterns.md: Database queues, Akka.Streams throttling, outbox pattern
• cluster-local-abstractions.md: GenericChildPerEntityParent, IPubSubMediator, execution mode wiring
• async-cancellation-patterns.md: Actor-scoped CancellationToken, linked CTS, timeout handling

---

1. EventStream vs DistributedPubSub

Critical: EventStream is LOCAL ONLY

Context.System.EventStream is local to a single ActorSystem process. It does NOT work across cluster nodes.

// BAD: This only works on a single server
// When you add a second server, subscribers on server 2 won't receive events from server 1
Context.System.EventStream.Subscribe(Self, typeof(PostCreated));
Context.System.EventStream.Publish(new PostCreated(postId, authorId));

When EventStream is appropriate:

• Logging and diagnostics within a single process
• Local event bus for truly single-process applications
• Development/testing scenarios

Use DistributedPubSub for Multi-Node

For events that must reach actors across multiple cluster nodes, use Akka.Cluster.Tools.PublishSubscribe:

using Akka.Cluster.Tools.PublishSubscribe;

public class TimelineUpdatePublisher : ReceiveActor
{
    private readonly IActorRef _mediator;

    public TimelineUpdatePublisher()
    {
        // Get the DistributedPubSub mediator
        _mediator = DistributedPubSub.Get(Context.System).Mediator;

        Receive<PublishTimelineUpdate>(msg =>
        {
            // Publish to a topic - reaches all subscribers across all nodes
            _mediator.Tell(new Publish($"timeline:{msg.UserId}", msg.Update));
        });
    }
}

Akka.Hosting Configuration for DistributedPubSub

builder.WithDistributedPubSub(role: null); // Available on all roles, or specify a role

Topic Design Patterns

---

2. Supervision Strategies

Key Clarification: Supervision is for CHILDREN

A supervision strategy defined on an actor dictates how that actor supervises its children, NOT how the actor itself is supervised.

public class ParentActor : ReceiveActor
{
    // This strategy applies to children of ParentActor, NOT to ParentActor itself
    protected override SupervisorStrategy SupervisorStrategy()
    {
        return new OneForOneStrategy(
            maxNrOfRetries: 10,
            withinTimeRange: TimeSpan.FromSeconds(30),
            decider: ex => ex switch
            {
                ArithmeticException => Directive.Resume,
                NullReferenceException => Directive.Restart,
                ArgumentException => Directive.Stop,
                _ => Directive.Escalate
            });
    }
}

Default Supervision Strategy

The default OneForOneStrategy already includes rate limiting:

• 10 restarts within 1 second = actor is permanently stopped
• This prevents infinite restart loops

You rarely need a custom strategy unless you have specific requirements.

When to Define Custom Supervision

Good reasons:

• Actor throws exceptions indicating irrecoverable state corruption -> Restart
• Actor throws exceptions that should NOT cause restart (expected failures) -> Resume
• Child failures should affect siblings -> Use AllForOneStrategy
• Need different retry limits than the default

Bad reasons:

• "Just to be safe" - the default is already safe
• Don't understand what the actor does - understand it first

---

3. Error Handling: Supervision vs Try-Catch

When to Use Try-Catch (Most Cases)

Use try-catch when:

• The failure is expected (network timeout, invalid input, external service down)
• You know exactly why the exception occurred
• You can handle it gracefully (retry, return error response, log and continue)
• Restarting would not help (same error would occur again)

public class RssFeedPollerActor : ReceiveActor
{
    public RssFeedPollerActor()
    {
        ReceiveAsync<PollFeed>(async msg =>
        {
            try
            {
                var feed = await _httpClient.GetStringAsync(msg.FeedUrl);
                var items = ParseFeed(feed);
                // Process items...
            }
            catch (HttpRequestException ex)
            {
                // Expected failure - log and schedule retry
                _log.Warning("Feed {Url} unavailable: {Error}", msg.FeedUrl, ex.Message);
                Context.System.Scheduler.ScheduleTellOnce(
                    TimeSpan.FromMinutes(5), Self, msg, Self);
            }
            catch (XmlException ex)
            {
                // Invalid feed format - log and mark as bad
                _log.Error("Feed {Url} has invalid format: {Error}", msg.FeedUrl, ex.Message);
                Sender.Tell(new FeedPollResult.InvalidFormat(msg.FeedUrl));
            }
        });
    }
}

When to Let Supervision Handle It

Let exceptions propagate (trigger supervision) when:

• You have no idea why the exception occurred
• The actor's state might be corrupt
• A restart would help (fresh state, reconnect resources)
• It's a programming error (NullReferenceException, InvalidOperationException from bad logic)

Anti-Pattern: Swallowing Unknown Exceptions

// BAD: Swallowing exceptions hides problems
catch (Exception ex)
{
    _log.Error(ex, "Error processing work");
    // Actor continues with potentially corrupt state
}

// GOOD: Handle known exceptions, let unknown ones propagate
catch (HttpRequestException ex)
{
    // Known, expected failure - handle gracefully
    _log.Warning("HTTP request failed: {Error}", ex.Message);
    Sender.Tell(new WorkResult.TransientFailure());
}
// Unknown exceptions propagate to supervision

---

4. Props vs DependencyResolver

When to Use Plain Props

Use `Props.Create()` when:

• Actor doesn't need IServiceProvider or IRequiredActor<T>
• All dependencies can be passed via constructor
• Actor is simple and self-contained

// Simple actor with no DI needs
public static Props Props(PostId postId, IPostWriteStore store)
    => Akka.Actor.Props.Create(() => new PostEngagementActor(postId, store));

When to Use DependencyResolver

Use `resolver.Props<T>()` when:

• Actor needs IServiceProvider to create scoped services
• Actor uses IRequiredActor<T> to get references to other actors
• Actor has many dependencies that are already in DI container

// Registration with DI
builder.WithActors((system, registry, resolver) =>
{
    var actor = system.ActorOf(resolver.Props<OrderProcessorActor>(), "order-processor");
    registry.Register<OrderProcessorActor>(actor);
});

Remote Deployment Considerations

You almost never need remote deployment. If you're not doing remote deployment (and you probably aren't):

• Props.Create(() => new Actor(...)) with closures is fine
• The "serialization issue" warning doesn't apply

For most applications, use cluster sharding instead of remote deployment - it handles distribution automatically.

---

5. Work Distribution Patterns

When you have many background jobs (RSS feeds, email sending, etc.), don't process them all at once - this causes thundering herd problems.

Three patterns to solve this: 1. Database-Driven Work Queue - Use FOR UPDATE SKIP LOCKED for natural cross-node distribution 2. Akka.Streams Rate Limiting - Throttle processing within a single node 3. Durable Queue (Outbox Pattern) - Database-backed outbox for reliable processing

See work-distribution-patterns.md for full code samples.

---

6. Common Mistakes Summary

---

7. Quick Reference

Communication Pattern Decision Tree

Need to communicate between actors?
├── Same process only? -> EventStream is fine
├── Across cluster nodes?
│   ├── Point-to-point? -> Use ActorSelection or known IActorRef
│   └── Pub/sub? -> Use DistributedPubSub
└── Fire-and-forget to external system? -> Consider outbox pattern

Error Handling Decision Tree

Exception occurred in actor?
├── Expected failure (HTTP timeout, invalid input)?
│   └── Try-catch, handle gracefully, continue
├── State might be corrupt?
│   └── Let supervision restart
├── Unknown cause?
│   └── Let supervision restart
└── Programming error (null ref, bad logic)?
    └── Let supervision restart, fix the bug

Props Decision Tree

Creating actor Props?
├── Actor needs IServiceProvider?
│   └── Use resolver.Props<T>()
├── Actor needs IRequiredActor<T>?
│   └── Use resolver.Props<T>()
├── Simple actor with constructor params?
│   └── Use Props.Create(() => new Actor(...))
└── Remote deployment needed?
    └── Probably not - use cluster sharding instead

---

8. Cluster/Local Mode Abstractions

For applications that need to run both in clustered production and local/test environments, use abstraction patterns to toggle between implementations:

• `AkkaExecutionMode` enum - Controls which implementations are used (LocalTest vs Clustered)
• `GenericChildPerEntityParent` - Mimics sharding behavior locally using the same IMessageExtractor
• `IPubSubMediator` - Abstracts DistributedPubSub for swappable local/cluster implementations

See cluster-local-abstractions.md for complete implementation code.

---

9. Actor Logging

Use ILoggingAdapter, Not ILogger<T>

In actors, use ILoggingAdapter from Context.GetLogger() instead of DI-injected ILogger<T>:

public class MyActor : ReceiveActor
{
    private readonly ILoggingAdapter _log = Context.GetLogger();

    public MyActor()
    {
        Receive<MyMessage>(msg =>
        {
            _log.Info("Processing message for user {UserId}", msg.UserId);
            _log.Error(ex, "Failed to process {MessageType}", msg.GetType().Name);
        });
    }
}

Why ILoggingAdapter:

• Integrates with Akka's logging pipeline and supervision
• Supports semantic/structured logging as of v1.5.57
• Method names: Info(), Debug(), Warning(), Error() (not Log* variants)
• No DI required - obtained directly from actor context

Don't inject ILogger<T> into actors - it bypasses Akka's logging infrastructure.

Semantic Logging (v1.5.57+)

// Named placeholders for better log aggregation and querying
_log.Info("Order {OrderId} processed for customer {CustomerId}", order.Id, order.CustomerId);

// Prefer named placeholders over positional
// Good: {OrderId}, {CustomerId}
// Avoid: {0}, {1}

---

10. Managing Async Operations with CancellationToken

When actors launch async operations via PipeTo, those operations can outlive the actor if not properly managed. Key practices:

• Actor CTS in PostStop - Always cancel and dispose in PostStop()
• New CTS per operation - Cancel previous before starting new work
• Pass token everywhere - EF Core queries, HTTP calls, etc.
• Linked CTS for timeouts - External calls get short timeouts to prevent hanging
• Graceful handling - Distinguish timeout vs shutdown in catch blocks

See async-cancellation-patterns.md for complete implementation code.

Managing Async Operations with CancellationToken

When actors launch async operations via PipeTo, those operations can outlive the actor if not properly managed. Use CancellationToken tied to the actor lifecycle.

Contents

• Actor-Scoped CancellationTokenSource
• Linked CTS for Per-Operation Timeouts
• Graceful Timeout vs Shutdown Handling
• Key Points
• When to Use

Actor-Scoped CancellationTokenSource

Cancel in-flight async work when the actor stops:

public class DataSyncActor : ReceiveActor
{
    private CancellationTokenSource? _operationCts;

    public DataSyncActor()
    {
        ReceiveAsync<StartSync>(HandleStartSyncAsync);
    }

    protected override void PostStop()
    {
        // Cancel any in-flight async work when actor stops
        _operationCts?.Cancel();
        _operationCts?.Dispose();
        _operationCts = null;
        base.PostStop();
    }

    private Task HandleStartSyncAsync(StartSync cmd)
    {
        // Cancel any previous operation, create new CTS
        _operationCts?.Cancel();
        _operationCts?.Dispose();
        _operationCts = new CancellationTokenSource();
        var ct = _operationCts.Token;

        async Task<SyncResult> PerformSyncAsync()
        {
            try
            {
                ct.ThrowIfCancellationRequested();

                // Pass token to all async operations
                var data = await _repository.GetDataAsync(ct);
                await _service.ProcessAsync(data, ct);

                return new SyncResult(Success: true);
            }
            catch (OperationCanceledException) when (ct.IsCancellationRequested)
            {
                // Actor is stopping - graceful exit
                return new SyncResult(Success: false, "Cancelled");
            }
        }

        PerformSyncAsync().PipeTo(Self);
        return Task.CompletedTask;
    }
}

Linked CTS for Per-Operation Timeouts

For external API calls that might hang, use linked CTS with short timeouts:

private static readonly TimeSpan ApiTimeout = TimeSpan.FromSeconds(30);

async Task<SyncResult> PerformSyncAsync()
{
    // Check actor-level cancellation
    ct.ThrowIfCancellationRequested();

    // Per-operation timeout linked to actor's CTS
    SomeResult result;
    using (var opCts = CancellationTokenSource.CreateLinkedTokenSource(ct))
    {
        opCts.CancelAfter(ApiTimeout);
        result = await _externalApi.FetchDataAsync(opCts.Token);
    }

    // Process result...
}

How linked CTS works:

• Inherits cancellation from parent (actor stop -> cancels immediately)
• Adds its own timeout via CancelAfter (hung API -> cancels after timeout)
• Whichever fires first wins
• Disposed after each operation (short-lived)

Graceful Timeout vs Shutdown Handling

Distinguish between actor shutdown and operation timeout:

try
{
    using var opCts = CancellationTokenSource.CreateLinkedTokenSource(ct);
    opCts.CancelAfter(ApiTimeout);
    await _api.CallAsync(opCts.Token);
}
catch (OperationCanceledException) when (!ct.IsCancellationRequested)
{
    // Timeout (not actor death) - can retry or handle gracefully
    _log.Warning("API call timed out, skipping item");
}
// If ct.IsCancellationRequested is true, let it propagate up

Key Points

When to Use

• Any actor that launches async work via PipeTo
• Long-running operations (sync jobs, batch processing)
• External API calls that might hang
• Database operations in loops

Cluster/Local Mode Abstractions

For applications that need to run both in clustered production environments and local/test environments without cluster infrastructure, use abstraction patterns to toggle between implementations.

Contents

• AkkaExecutionMode Enum
• GenericChildPerEntityParent - Local Sharding Alternative
• IPubSubMediator - Abstracting DistributedPubSub
• LocalPubSubMediator - In-Memory Implementation
• ClusterPubSubMediator - Production Implementation
• Wiring It All Together
• Usage in Application Code
• Benefits and When to Use Each Mode

AkkaExecutionMode Enum

Define an execution mode that controls which implementations are used:

/// <summary>
/// Determines how Akka.NET infrastructure features are configured.
/// </summary>
public enum AkkaExecutionMode
{
    /// <summary>
    /// Local test mode - no cluster infrastructure.
    /// Uses in-memory implementations for pub/sub and local parent actors
    /// instead of cluster sharding.
    /// </summary>
    LocalTest,

    /// <summary>
    /// Full cluster mode with sharding, singletons, and distributed pub/sub.
    /// </summary>
    Clustered
}

GenericChildPerEntityParent - Local Sharding Alternative

When testing locally, you can't use Cluster Sharding. This actor mimics sharding behavior by creating child actors per entity ID using the same IMessageExtractor interface:

/// <summary>
/// A local parent actor that mimics Cluster Sharding behavior.
/// Creates and manages child actors per entity ID using the same IMessageExtractor
/// that would be used with real sharding, enabling seamless switching between modes.
/// </summary>
public sealed class GenericChildPerEntityParent : ReceiveActor
{
    private readonly IMessageExtractor _extractor;
    private readonly Func<string, Props> _propsFactory;
    private readonly Dictionary<string, IActorRef> _children = new();
    private readonly ILoggingAdapter _log = Context.GetLogger();

    public GenericChildPerEntityParent(
        IMessageExtractor extractor,
        Func<string, Props> propsFactory)
    {
        _extractor = extractor;
        _propsFactory = propsFactory;

        ReceiveAny(msg =>
        {
            var entityId = _extractor.EntityId(msg);
            if (string.IsNullOrEmpty(entityId))
            {
                _log.Warning("Could not extract entity ID from message {0}", msg.GetType().Name);
                Unhandled(msg);
                return;
            }

            var child = GetOrCreateChild(entityId);

            // Unwrap the message if it's a ShardingEnvelope
            var unwrapped = _extractor.EntityMessage(msg);
            child.Forward(unwrapped);
        });
    }

    private IActorRef GetOrCreateChild(string entityId)
    {
        if (_children.TryGetValue(entityId, out var existing))
            return existing;

        var props = _propsFactory(entityId);
        var child = Context.ActorOf(props, entityId);
        Context.Watch(child);
        _children[entityId] = child;

        _log.Debug("Created child actor for entity {0}", entityId);
        return child;
    }

    protected override void PreRestart(Exception reason, object message)
    {
        // Don't stop children on restart
    }

    public static Props CreateProps(
        IMessageExtractor extractor,
        Func<string, Props> propsFactory)
    {
        return Props.Create(() => new GenericChildPerEntityParent(extractor, propsFactory));
    }
}

IPubSubMediator - Abstracting DistributedPubSub

Create an interface to abstract over pub/sub so tests can use a local implementation:

/// <summary>
/// Abstraction over pub/sub messaging that allows swapping between
/// DistributedPubSub (clustered) and local implementations (testing).
/// </summary>
public interface IPubSubMediator
{
    /// <summary>
    /// Subscribe an actor to a topic.
    /// </summary>
    void Subscribe(string topic, IActorRef subscriber);

    /// <summary>
    /// Unsubscribe an actor from a topic.
    /// </summary>
    void Unsubscribe(string topic, IActorRef subscriber);

    /// <summary>
    /// Publish a message to all subscribers of a topic.
    /// </summary>
    void Publish(string topic, object message);

    /// <summary>
    /// Send a message to one subscriber of a topic (load balanced).
    /// </summary>
    void Send(string topic, object message);
}

LocalPubSubMediator - In-Memory Implementation

/// <summary>
/// In-memory pub/sub implementation for local testing without cluster.
/// Uses the EventStream internally for simplicity.
/// </summary>
public sealed class LocalPubSubMediator : IPubSubMediator
{
    private readonly ActorSystem _system;
    private readonly ConcurrentDictionary<string, HashSet<IActorRef>> _subscriptions = new();
    private readonly object _lock = new();

    public LocalPubSubMediator(ActorSystem system)
    {
        _system = system;
    }

    public void Subscribe(string topic, IActorRef subscriber)
    {
        lock (_lock)
        {
            var subs = _subscriptions.GetOrAdd(topic, _ => new HashSet<IActorRef>());
            subs.Add(subscriber);
        }

        // Send acknowledgement like real DistributedPubSub does
        subscriber.Tell(new SubscribeAck(new Subscribe(topic, subscriber)));
    }

    public void Unsubscribe(string topic, IActorRef subscriber)
    {
        lock (_lock)
        {
            if (_subscriptions.TryGetValue(topic, out var subs))
            {
                subs.Remove(subscriber);
            }
        }

        subscriber.Tell(new UnsubscribeAck(new Unsubscribe(topic, subscriber)));
    }

    public void Publish(string topic, object message)
    {
        HashSet<IActorRef> subscribers;
        lock (_lock)
        {
            if (!_subscriptions.TryGetValue(topic, out var subs))
                return;
            subscribers = new HashSet<IActorRef>(subs);
        }

        foreach (var subscriber in subscribers)
        {
            subscriber.Tell(message);
        }
    }

    public void Send(string topic, object message)
    {
        IActorRef? target = null;
        lock (_lock)
        {
            if (_subscriptions.TryGetValue(topic, out var subs) && subs.Count > 0)
            {
                // Simple round-robin - pick first available
                target = subs.FirstOrDefault();
            }
        }

        target?.Tell(message);
    }
}

ClusterPubSubMediator - Production Implementation

/// <summary>
/// Production implementation wrapping Akka.Cluster.Tools.PublishSubscribe.
/// </summary>
public sealed class ClusterPubSubMediator : IPubSubMediator
{
    private readonly IActorRef _mediator;

    public ClusterPubSubMediator(ActorSystem system)
    {
        _mediator = DistributedPubSub.Get(system).Mediator;
    }

    public void Subscribe(string topic, IActorRef subscriber)
    {
        _mediator.Tell(new Subscribe(topic, subscriber));
    }

    public void Unsubscribe(string topic, IActorRef subscriber)
    {
        _mediator.Tell(new Unsubscribe(topic, subscriber));
    }

    public void Publish(string topic, object message)
    {
        _mediator.Tell(new Publish(topic, message));
    }

    public void Send(string topic, object message)
    {
        _mediator.Tell(new Send(topic, message, localAffinity: true));
    }
}

Wiring It All Together

Configure your ActorSystem based on execution mode:

public static class AkkaHostingExtensions
{
    public static AkkaConfigurationBuilder ConfigureActorSystem(
        this AkkaConfigurationBuilder builder,
        AkkaExecutionMode mode,
        IServiceCollection services)
    {
        if (mode == AkkaExecutionMode.Clustered)
        {
            builder
                .WithClustering()
                .WithShardRegion<MyEntity>(
                    "my-entity",
                    (system, registry, resolver) => entityId =>
                        resolver.Props<MyEntityActor>(entityId),
                    new MyEntityMessageExtractor(),
                    new ShardOptions())
                .WithDistributedPubSub();

            // Register cluster pub/sub mediator
            services.AddSingleton<IPubSubMediator>(sp =>
            {
                var system = sp.GetRequiredService<ActorSystem>();
                return new ClusterPubSubMediator(system);
            });
        }
        else // LocalTest mode
        {
            // Register local pub/sub mediator
            services.AddSingleton<IPubSubMediator>(sp =>
            {
                var system = sp.GetRequiredService<ActorSystem>();
                return new LocalPubSubMediator(system);
            });

            // Use GenericChildPerEntityParent instead of sharding
            builder.WithActors((system, registry, resolver) =>
            {
                var parent = system.ActorOf(
                    GenericChildPerEntityParent.CreateProps(
                        new MyEntityMessageExtractor(),
                        entityId => resolver.Props<MyEntityActor>(entityId)),
                    "my-entity");

                registry.Register<MyEntityParent>(parent);
            });
        }

        return builder;
    }
}

Usage in Application Code

Application code uses the abstractions and doesn't need to know which mode is active:

public class MyService
{
    private readonly IPubSubMediator _pubSub;
    private readonly IRequiredActor<MyEntityParent> _entityParent;

    public MyService(
        IPubSubMediator pubSub,
        IRequiredActor<MyEntityParent> entityParent)
    {
        _pubSub = pubSub;
        _entityParent = entityParent;
    }

    public async Task ProcessAsync(string entityId, MyCommand command)
    {
        // Works identically in both modes
        var parent = await _entityParent.GetAsync();
        parent.Tell(new ShardingEnvelope(entityId, command));

        // Publish event - works with both local and distributed pub/sub
        _pubSub.Publish($"entity:{entityId}", new EntityUpdated(entityId));
    }
}

Benefits and When to Use Each Mode

Work Distribution Patterns

Contents

• Problem: Thundering Herd
• Pattern 1: Database-Driven Work Queue
• Pattern 2: Akka.Streams for Rate Limiting
• Pattern 3: Durable Queue (Email Outbox Pattern)

Problem: Thundering Herd

When you have many background jobs (RSS feeds, email sending, etc.), don't process them all at once:

// BAD: Polls all feeds simultaneously on startup
public class BadRssCoordinator : ReceiveActor
{
    public BadRssCoordinator(IRssFeedRepository repo)
    {
        ReceiveAsync<StartPolling>(async _ =>
        {
            var feeds = await repo.GetAllFeedsAsync();
            foreach (var feed in feeds) // 2000 feeds = 2000 simultaneous HTTP requests
            {
                Context.ActorOf(RssFeedPollerActor.Props(feed.Url));
            }
        });
    }
}

Pattern 1: Database-Driven Work Queue

Use the database as a work queue with FOR UPDATE SKIP LOCKED:

public class RssPollerWorker : ReceiveActor
{
    public RssPollerWorker(IRssFeedRepository repo)
    {
        ReceiveAsync<PollBatch>(async _ =>
        {
            // Each worker claims a batch - naturally distributes across nodes
            var feeds = await repo.ClaimFeedsForPollingAsync(
                batchSize: 10,
                staleAfter: TimeSpan.FromMinutes(10));

            foreach (var feed in feeds)
            {
                try
                {
                    await PollFeed(feed);
                    await repo.MarkPolledAsync(feed.Id, success: true);
                }
                catch (Exception ex)
                {
                    await repo.MarkPolledAsync(feed.Id, success: false, error: ex.Message);
                }
            }

            // Schedule next batch
            Context.System.Scheduler.ScheduleTellOnce(
                TimeSpan.FromSeconds(5),
                Self,
                PollBatch.Instance,
                Self);
        });
    }
}

-- ClaimFeedsForPollingAsync implementation
UPDATE rss_feeds
SET status = 'processing',
    processing_started_at = NOW()
WHERE id IN (
    SELECT id FROM rss_feeds
    WHERE status = 'pending'
      AND (next_poll_at IS NULL OR next_poll_at <= NOW())
    ORDER BY next_poll_at NULLS FIRST
    LIMIT @batchSize
    FOR UPDATE SKIP LOCKED
)
RETURNING *;

Benefits:

• Naturally distributes work across multiple server nodes
• No coordination needed - database handles locking
• Easy to monitor (query the table)
• Survives server restarts

Pattern 2: Akka.Streams for Rate Limiting

Use Akka.Streams to throttle processing within a single node:

public class ThrottledRssProcessor : ReceiveActor
{
    public ThrottledRssProcessor(IRssFeedRepository repo)
    {
        var materializer = Context.System.Materializer();

        ReceiveAsync<StartProcessing>(async _ =>
        {
            await Source.From(await repo.GetPendingFeedsAsync())
                .Throttle(10, TimeSpan.FromSeconds(1)) // Max 10 per second
                .SelectAsync(4, async feed => // Max 4 concurrent
                {
                    await PollFeed(feed);
                    return feed;
                })
                .RunWith(Sink.Ignore<RssFeed>(), materializer);
        });
    }
}

Pattern 3: Durable Queue (Email Outbox Pattern)

For work that must be reliably processed, use a database-backed outbox:

// Enqueue work transactionally with business operation
public async Task CreatePostAsync(Post post)
{
    await using var transaction = await _db.BeginTransactionAsync();

    await _postStore.CreateAsync(post);

    // Enqueue notification emails in same transaction
    foreach (var follower in await _followStore.GetFollowersAsync(post.AuthorId))
    {
        await _emailOutbox.EnqueueAsync(new EmailJob
        {
            To = follower.Email,
            Template = "new-post",
            Data = JsonSerializer.Serialize(new { PostId = post.Id })
        });
    }

    await transaction.CommitAsync();
}

// Worker processes outbox
public class EmailOutboxWorker : ReceiveActor
{
    public EmailOutboxWorker(IEmailOutboxStore outbox, IEmailSender sender)
    {
        ReceiveAsync<ProcessBatch>(async _ =>
        {
            var batch = await outbox.ClaimBatchAsync(10);
            foreach (var job in batch)
            {
                try
                {
                    await sender.SendAsync(job);
                    await outbox.MarkSentAsync(job.Id);
                }
                catch (Exception ex)
                {
                    await outbox.MarkFailedAsync(job.Id, ex.Message);
                }
            }
        });
    }
}