Redis Connections

Redis Connections

Client-side guidance for talking to Redis efficiently: how to share connections, how to batch commands, which commands not to call in production, when to turn on client-side caching, and how to set timeouts that fail fast without breaking healthy traffic.

When to apply

• Creating or reviewing a Redis client setup (redis-py, Jedis, Lettuce, go-redis, NRedisStack).
• Making many small Redis calls and wondering where the latency is going.
• Iterating large keyspaces, sets, hashes, or lists.
• Enabling client-side caching for hot keys.
• Tuning connect / read / write timeouts.

1. Pool or multiplex — never one connection per request

The single biggest mistake in Redis client code is opening a new TCP connection for every operation. Always either:

• Pool — keep N persistent connections that the application leases per call (redis-py ConnectionPool, Jedis JedisPooled, go-redis client).
• Multiplex — share a single connection across all requests (Lettuce, NRedisStack).

# redis-py — connection pool
pool = redis.ConnectionPool(host="localhost", port=6379, max_connections=50)
r = redis.Redis(connection_pool=pool)

See references/pooling.md for Python + Java + Lettuce examples.

2. Pipeline bulk work

For N commands that don't depend on each other's results, send them as a single batch with pipelining. One round-trip instead of N.

pipe = redis.pipeline()
for user_id in user_ids:
    pipe.get(f"user:{user_id}")
results = pipe.execute()

Use non-transactional pipelining for performance, and pipeline(transaction=True) only when you actually need atomicity (see redis-core's transactions guidance).

See references/pipelining.md.

3. Avoid commands that scan everything

Anything that walks the whole keyspace (or a whole large container) blocks the server. Use incremental variants instead.

cursor = 0
while True:
    cursor, keys = redis.scan(cursor, match="user:*", count=100)
    for key in keys:
        process(key)
    if cursor == 0:
        break

Blocking commands (`BLPOP`, `BRPOP`, `BLMOVE`) are different — they intentionally wait for data and are fine for queue consumers, but always pass a timeout, and don't issue them on a multiplexed connection (Lettuce, NRedisStack).

See references/blocking.md.

4. Client-side caching for hot keys

For data that's read often and written rarely (config, feature flags, sessions on every request), enable RESP3 client-side caching. The client keeps a local copy and the server invalidates it on writes — saving the round trip for hot reads.

client = redis.Redis(
    host="localhost",
    port=6379,
    protocol=3,                                    # RESP3 is required
    cache_config=redis.CacheConfig(max_size=1000),
)

Skip it for write-heavy workloads or data that changes constantly — the invalidation traffic overruns the savings.

See references/client-cache.md.

5. Set explicit timeouts

Defaults vary by client and may be too generous. Pick values that match the application's failure model:

r = redis.Redis(
    host="localhost",
    socket_connect_timeout=2.0,   # fail fast on dead nodes
    socket_timeout=5.0,           # tune to expected operation time
    retry_on_timeout=True,
)

Rule of thumb: connect timeout shorter than read/write timeout. Tight timeouts + retry-on-timeout for latency-sensitive paths; longer timeouts for batch jobs.

See references/timeouts.md.

References

• Redis: Connection Pools and Multiplexing
• Redis: Pipelining
• Redis: SCAN
• Redis: Client-side caching
• Redis: Clients

Avoid Slow Commands in Production

Some Redis commands are slow because they scan large datasets. Use incremental alternatives to avoid blocking the server.

Correct: Use SCAN for iteration.

Python (redis-py):

# Good: Non-blocking iteration
cursor = 0
while True:
    cursor, keys = redis.scan(cursor, match="user:*", count=100)
    for key in keys:
        process(key)
    if cursor == 0:
        break

Java (Jedis):

import redis.clients.jedis.ScanIteration;
import redis.clients.jedis.UnifiedJedis;
import java.util.List;

try (UnifiedJedis jedis = new UnifiedJedis("redis://localhost:6379")) {
    // ScanIteration manages the cursor automatically
    ScanIteration scan = jedis.scanIteration(10, "user:*", "hash");

    while (!scan.isIterationCompleted()) {
        List<String> result = scan.nextBatch().getResult();
        for (String key : result) {
            process(key);
        }
    }
}

Incorrect: Using KEYS in production.

Python (redis-py):

# Bad: Scans all keys, slow on large datasets
keys = redis.keys("user:*")

Java (Jedis):

// Bad: Scans all keys, blocks the server
Set<String> result = jedis.keys("*");

Note: Truly blocking commands (like BLPOP, BRPOP, BLMOVE) that wait indefinitely for data are appropriate for some use cases like job queues, but should be used with timeouts.

# Blocking pop with timeout - appropriate for queue consumers
result = redis.blpop("task_queue", timeout=5)

Reference: Redis SCAN

Use Client-Side Caching for Frequently Read Data

Use a connection with client-side caching enabled for any data that will be read frequently but written only occasionally. Client-side caching avoids contacting the server for repeated access to data that has recently been read, reducing network traffic and improving performance.

Correct: Enable client-side caching with RESP3 protocol for frequently accessed data.

Python (redis-py):

import redis

# Enable client-side caching with RESP3
client = redis.Redis(
    host='localhost',
    port=6379,
    protocol=3,  # RESP3 required for client-side caching
    cache_config=redis.CacheConfig(max_size=1000)
)

# Cached reads avoid server round-trips
value = client.get("frequently:read:key")

Java (Jedis):

import redis.clients.jedis.DefaultJedisClientConfig;
import redis.clients.jedis.UnifiedJedis;
import redis.clients.jedis.HostAndPort;
import redis.clients.jedis.CacheConfig;

HostAndPort endpoint = new HostAndPort("localhost", 6379);

DefaultJedisClientConfig config = DefaultJedisClientConfig
    .builder()
    .password("secretPassword")
    .protocol(RedisProtocol.RESP3)
    .build();

CacheConfig cacheConfig = CacheConfig.builder().maxSize(1000).build();

UnifiedJedis client = new UnifiedJedis(endpoint, config, cacheConfig);

When to use:

• Configuration data read frequently, updated rarely
• User session data accessed on every request
• Feature flags or settings checked repeatedly
• Any read-heavy workload with low write frequency

When NOT needed:

• Data that changes frequently (cache invalidation overhead outweighs benefits)
• Write-heavy workloads
• Simple applications where network latency is not a bottleneck
• When you need guaranteed real-time consistency

Trade-offs:

• Adds memory overhead on the client
• Requires RESP3 protocol
• Cache invalidation adds complexity for frequently changing data

Reference: Client-side caching

Use Pipelining for Bulk Operations

Batch multiple commands into a single round trip to reduce network latency.

Correct: Use pipeline for multiple commands.

Python (redis-py):

# Good: Single round trip for multiple commands
pipe = redis.pipeline()
for user_id in user_ids:
    pipe.get(f"user:{user_id}")
results = pipe.execute()

Java (Jedis):

import redis.clients.jedis.Pipeline;

// Good: Buffer commands and send as single batch
Pipeline pipe = (Pipeline) jedis.pipelined();

pipe.set("person:1:name", "Alex");
pipe.set("person:1:rank", "Captain");
pipe.set("person:1:serial", "AB1234");

pipe.sync();

Incorrect: Sequential commands in a loop.

Python (redis-py):

# Bad: N round trips
results = []
for user_id in user_ids:
    results.append(redis.get(f"user:{user_id}"))

Java (Jedis):

// Bad: 3 separate round trips
jedis.set("person:1:name", "Alex");
jedis.set("person:1:rank", "Captain");
jedis.set("person:1:serial", "AB1234");

Reference: Redis Pipelining

Use Connection Pooling or Multiplexing

Reuse connections via a pool or multiplexing instead of creating new connections per request.

Correct: Use a connection pool.

Python (redis-py):

import redis

# Good: Connection pool - reuses existing connections
pool = redis.ConnectionPool(host='localhost', port=6379, max_connections=50)
r = redis.Redis(connection_pool=pool)

Java (Jedis):

import redis.clients.jedis.JedisPooled;

// JedisPooled manages a connection pool internally
try (JedisPooled jedis = new JedisPooled("redis://localhost:6379")) {
    jedis.set("testKey", "testValue");
}

Correct: Use multiplexing (Lettuce, NRedisStack).

// Lettuce uses multiplexing by default - single connection handles all traffic
RedisClient client = RedisClient.create("redis://localhost:6379");
StatefulRedisConnection<String, String> connection = client.connect();

// All commands share the single connection efficiently
connection.sync().set("key", "value");

Incorrect: Creating new connections per request.

Python (redis-py):

# Bad: New connection every time
def get_user(user_id):
    r = redis.Redis(host='localhost', port=6379)  # Don't do this
    return r.get(f"user:{user_id}")

Java (Jedis):

// Bad: Creating new client per request
public String getUser(String userId) {
    try (UnifiedJedis jedis = new UnifiedJedis("redis://localhost:6379")) {
        return jedis.get("user:" + userId);  // Don't do this
    }
}

Pooling vs Multiplexing:

• Pooling: Multiple connections shared across requests (redis-py, Jedis, go-redis)
• Multiplexing: Single connection handles all traffic (NRedisStack, Lettuce)
• Multiplexing cannot support blocking commands (BLPOP, etc.) as they would stall all callers

Reference: Connection Pools and Multiplexing

Configure Connection Timeouts

Configure appropriate timeout values to improve your application's connection resilience. While most Redis clients set default timeouts, choosing well-tuned values based on your application's usage patterns leads to better failure recovery.

Correct: Set timeouts based on your application needs.

r = redis.Redis(
    host='localhost',
    socket_timeout=5.0,          # Read/write timeout - tune based on expected operation time
    socket_connect_timeout=2.0,  # Connection timeout - shorter for fast failure detection
    retry_on_timeout=True        # Automatic retry on timeout
)

Incorrect: Relying solely on defaults without considering your use case.

# Not ideal: Default timeouts may not match your application's needs
r = redis.Redis(host='localhost')

# For example, if your app needs fast failure detection,
# the default timeouts might be too generous

Considerations:

• Set socket_connect_timeout shorter than socket_timeout for quick connection failure detection
• For latency-sensitive apps, use tighter timeouts with retry logic
• For batch operations, allow longer timeouts to complete large operations
• Consider using health checks alongside timeouts for robust failure handling

Reference: Redis Client Configuration