Golang Samber Hot

Persona: You are a Go engineer who treats caching as a system design decision. You choose eviction algorithms based on measured access patterns, size caches from working-set data, and always plan for expiration, loader failures, and monitoring.

Using samber/hot for In-Memory Caching in Go

Generic, type-safe in-memory caching library for Go 1.22+ with 9 eviction algorithms, TTL, loader chains with singleflight deduplication, sharding, stale-while-revalidate, and Prometheus metrics.

Official Resources:

• pkg.go.dev/github.com/samber/hot
• github.com/samber/hot

This skill is not exhaustive. Please refer to library documentation and code examples for more information. Context7 can help as a discoverability platform.

go get -u github.com/samber/hot

Algorithm Selection

Pick based on your access pattern — the wrong algorithm wastes memory or tanks hit rate.

Decision shortcut: Start with hot.WTinyLFU. Switch only when profiling shows the miss rate is too high for your SLO.

For detailed algorithm comparison, benchmarks, and a decision tree, see Algorithm Guide.

Core Usage

Basic Cache with TTL

import "github.com/samber/hot"

cache := hot.NewHotCache[string, *User](hot.WTinyLFU, 10_000).
    WithTTL(5 * time.Minute).
    WithJanitor().
    Build()
defer cache.StopJanitor()

cache.Set("user:123", user)
cache.SetWithTTL("session:abc", session, 30*time.Minute)

value, found, err := cache.Get("user:123")

Loader Pattern (Read-Through)

Loaders fetch missing keys automatically with singleflight deduplication — concurrent Get() calls for the same missing key share one loader invocation:

cache := hot.NewHotCache[int, *User](hot.WTinyLFU, 10_000).
    WithTTL(5 * time.Minute).
    WithLoaders(func(ids []int) (map[int]*User, error) {
        return db.GetUsersByIDs(ctx, ids) // batch query
    }).
    WithJanitor().
    Build()
defer cache.StopJanitor()

user, found, err := cache.Get(123) // triggers loader on miss

Capacity Sizing

Before setting the cache capacity, estimate how many items fit in the memory budget:

1. Estimate single-item size — estimate size of the struct, add the size of heap-allocated fields (slices, maps, strings). Include the key size. A rough per-entry overhead of ~100 bytes covers internal bookkeeping (pointers, expiry timestamps, algorithm metadata). 2. Ask the developer how much memory is dedicated to this cache in production (e.g., 256 MB, 1 GB). This depends on the service's total memory and what else shares the process. 3. Compute capacity — capacity = memoryBudget / estimatedItemSize. Round down to leave headroom.

Example: *User struct ~500 bytes + string key ~50 bytes + overhead ~100 bytes = ~650 bytes/entry
         256 MB budget → 256_000_000 / 650 ≈ 393,000 items

If the item size is unknown, ask the developer to measure it with a unit test that allocates N items and checks runtime.ReadMemStats. Guessing capacity without measuring leads to OOM or wasted memory.

Common Mistakes

1. Forgetting `WithJanitor()` — without it, expired entries stay in memory until the algorithm evicts them. Always chain .WithJanitor() in the builder and defer cache.StopJanitor(). 2. Calling `SetMissing()` without missing cache config — panics at runtime. Enable WithMissingCache(algorithm, capacity) or WithMissingSharedCache() in the builder first. 3. `WithoutLocking()` + `WithJanitor()` — mutually exclusive, panics. WithoutLocking() is only safe for single-goroutine access without background cleanup. 4. Oversized cache — a cache holding everything is a map with overhead. Size to your working set (typically 10-20% of total data). Monitor hit rate to validate. 5. Ignoring loader errors — Get() returns (zero, false, err) on loader failure. Always check err, not just found.

Best Practices

1. Always set TTL — unbounded caches serve stale data indefinitely because there is no signal to refresh 2. Use WithJitter(lambda, upperBound) to spread expirations — without jitter, items created together expire together, causing thundering herd on the loader 3. Monitor with WithPrometheusMetrics(cacheName) — hit rate below 80% usually means the cache is undersized or the algorithm is wrong for the workload 4. Use WithCopyOnRead(fn) / WithCopyOnWrite(fn) for mutable values — without copies, callers mutate cached objects and corrupt shared state

For advanced patterns (revalidation, sharding, missing cache, monitoring setup), see Production Patterns.

For the complete API surface, see API Reference.

If you encounter a bug or unexpected behavior in samber/hot, open an issue at <https://github.com/samber/hot/issues>.

Cross-References

• → See samber/cc-skills-golang@golang-performance skill for general caching strategy and when to use in-memory cache vs Redis vs CDN
• → See samber/cc-skills-golang@golang-observability skill for Prometheus metrics integration and monitoring
• → See samber/cc-skills-golang@golang-database skill for database query patterns that pair with cache loaders
• → See samber/cc-skills@promql-cli skill for querying Prometheus cache metrics via CLI

Algorithm Selection Guide

Decision Tree

Start here
  |
  v
Do you know your access pattern?
  |-- No --> Use W-TinyLFU (adapts automatically)
  |-- Yes
       |
       v
     Is recency the primary signal? (sessions, recent queries, time-windowed data)
       |-- Yes --> LRU
       |-- No
            |
            v
          Is frequency the primary signal? (popular products, DNS, static config)
            |-- Yes --> Does the popularity ranking shift over time?
            |           |-- Yes --> TinyLFU (frequency with decay)
            |           |-- No --> LFU
            |-- No
                 |
                 v
               Is throughput critical and cache is large? (>100k items, high write rate)
                 |-- Yes --> S3FIFO
                 |-- No
                      |
                      v
                    Do you want self-tuning with no config? (unknown or shifting patterns)
                      |-- Yes --> ARC (higher memory) or W-TinyLFU (lower memory)
                      |-- No
                           |
                           v
                         Is scan resistance needed with simplicity?
                           |-- Yes --> SIEVE
                           |-- No --> W-TinyLFU (safe default)

Algorithm Deep Dives

LRU (Least Recently Used)

Constant: hot.LRU

Evicts the item that hasn't been accessed for the longest time. Simple doubly-linked list + hash map implementation.

• Strengths: Simple mental model, predictable behavior, low overhead per operation
• Weaknesses: Scan pollution — a single sequential scan evicts all hot items. No frequency awareness.
• Ideal workload: Time-windowed data (user sessions, recent search results, short-lived tokens)
• Degrades when: A batch job or sequential scan touches many cold keys, evicting frequently-used items

LFU (Least Frequently Used)

Constant: hot.LFU

Evicts the item with the fewest accesses. Tracks access counts per key.

• Strengths: Keeps genuinely popular items regardless of access timing
• Weaknesses: Stale popular items never evict — an item accessed 10,000 times yesterday blocks new hot items today. No frequency decay.
• Ideal workload: Stable popularity rankings (DNS records, country code lookups, static configuration)
• Degrades when: Popularity shifts over time or new items need to ramp up quickly

TinyLFU

Constant: hot.TinyLFU

Combines frequency estimation with a compact Count-Min Sketch instead of per-key counters. Includes frequency decay — old access counts fade over time.

• Strengths: Low memory overhead for frequency tracking, handles popularity shifts via decay, good admission filtering
• Weaknesses: Admission filter adds overhead on writes. Sketch approximation can cause rare false positives.
• Ideal workload: Read-heavy with moderate frequency bias (API response caching, content delivery metadata)
• Degrades when: Write-heavy workloads where admission overhead exceeds the benefit

W-TinyLFU (Weighted TinyLFU)

Constant: hot.WTinyLFU

Adds a small "window" LRU in front of TinyLFU's admission filter. New items enter the window, and the admission filter decides whether they're promoted to the main cache. Balances recency and frequency automatically.

• Strengths: Best general-purpose hit rate across diverse workloads. Self-adapting. Handles both recency and frequency patterns.
• Weaknesses: Slightly more complex internals (harder to reason about eviction order during debugging)
• Ideal workload: Mixed or unknown access patterns, general-purpose caching
• Degrades when: Rarely — it's the safest default. May underperform specialized algorithms on extreme workloads.

S3FIFO (Segmented Small-Size FIFO)

Constant: hot.S3FIFO

Three-segment FIFO design: small, main, and ghost queues. Items promoted from small to main only if accessed again. Ghost queue tracks recently evicted keys for scan resistance.

• Strengths: Excellent throughput (FIFO operations are cheaper than linked-list manipulations). Good scan resistance. Simple eviction path.
• Weaknesses: Needs enough capacity for the segmented structure to work. Less effective on small caches.
• Ideal workload: High-throughput systems with large caches (>100k items), CDN-like access patterns
• Degrades when: Cache is very small (<1000 items) — the segments don't have enough room to differentiate access patterns

ARC (Adaptive Replacement Cache)

Constant: hot.ARC

Maintains four internal lists: two for recency (recent and recent-ghost) and two for frequency (frequent and frequent-ghost). Dynamically adjusts the split between recency and frequency based on which ghost list sees more hits.

• Strengths: Self-tuning — learns from misses whether to favor recency or frequency. No manual parameter tuning.
• Weaknesses: ~2x memory overhead for ghost lists. More complex implementation.
• Ideal workload: Workloads that shift between recency and frequency patterns (mixed database query caching)
• Degrades when: Memory is constrained — the ghost lists consume significant space

TwoQueue

Constant: hot.TwoQueue

Separates items into "hot" (frequently accessed) and "cold" (recently added) queues with independent eviction. Items graduate from cold to hot on second access.

• Strengths: Good separation of one-hit wonders from genuinely useful items. Scan resistant.
• Weaknesses: Requires understanding the hot/cold split ratio for optimal tuning
• Ideal workload: Workloads with a clear hot/cold distinction (e.g., 20% of keys serve 80% of requests)
• Degrades when: Access patterns are uniform with no clear hot/cold split

SIEVE

Constant: hot.SIEVE

Modern eviction algorithm that uses a single bit per entry (visited/not-visited) with a circular "hand" pointer. Simple scan-resistant alternative to LRU.

• Strengths: Very low per-item overhead (1 bit). Scan-resistant. Simple implementation.
• Weaknesses: Less sophisticated than W-TinyLFU or ARC for complex patterns
• Ideal workload: When you want scan resistance with minimal complexity and overhead
• Degrades when: Access patterns are highly skewed — specialized algorithms capture the skew better

FIFO (First In, First Out)

Constant: hot.FIFO

Evicts the oldest inserted item regardless of access pattern. No recency or frequency tracking.

• Strengths: Simplest possible eviction. Predictable. Zero per-access overhead.
• Weaknesses: No intelligence — ignores how often or recently items are accessed
• Ideal workload: TTL-driven caches where all items have similar lifetimes and eviction order doesn't matter (log buffers, time-series windows)
• Degrades when: Hit rate matters — any other algorithm will outperform FIFO on non-uniform access patterns

Comparison Matrix

Measuring Hit Rate

Enable Prometheus metrics and check hit ratio to validate your algorithm choice:

cache := hot.NewHotCache[string, *User](hot.WTinyLFU, 10_000).
    WithTTL(5 * time.Minute).
    WithPrometheusMetrics("user_cache").
    WithJanitor().
    Build()
defer cache.StopJanitor()

prometheus.MustRegister(cache)

Key PromQL queries:

# Hit ratio (target: >80%)
rate(hot_cache_hit_count{cache="user_cache"}[5m]) /
rate(hot_cache_get_count{cache="user_cache"}[5m])

# Eviction rate (high = cache too small)
rate(hot_cache_eviction_count{cache="user_cache"}[5m])

If hit rate is below your SLO: increase capacity first, then try a different algorithm.

Switching Algorithms

Changing the algorithm is a one-line change — the rest of the builder chain stays identical:

// Before
cache := hot.NewHotCache[string, *User](hot.LRU, 10_000).
    WithTTL(5 * time.Minute).
    WithJanitor().
    Build()

// After — only the first argument changes
cache := hot.NewHotCache[string, *User](hot.WTinyLFU, 10_000).
    WithTTL(5 * time.Minute).
    WithJanitor().
    Build()

API Reference

Constructor

hot.NewHotCache[K comparable, V any](algorithm hot.EvictionAlgorithm, capacity int) *HotCacheBuilder[K, V]

Algorithm constants:

Builder Methods

Call these on the builder returned by NewHotCache(), then finalize with .Build().

Read Operations

Write Operations

Maintenance Operations

Loader Type

type Loader[K comparable, V any] func(keys []K) (found map[K]V, err error)

Chain semantics:

• Loaders execute sequentially in provided order
• Each loader receives only unmapped keys from previous loaders
• Later loader values overwrite earlier values for the same key
• Any loader error stops the chain and returns (nil, err)
• Built-in singleflight deduplication: concurrent Get() calls for the same key share one loader invocation

Hasher Type (for Sharding)

type Hasher[K any] func(key K) uint64

Prometheus Integration

*HotCache implements prometheus.Collector. Register it to expose metrics:

cache := hot.NewHotCache[string, *User](hot.WTinyLFU, 10_000).
    WithPrometheusMetrics("user_cache").
    Build()

prometheus.MustRegister(cache)

Production Patterns

Stale-While-Revalidate

Return stale data immediately while refreshing in the background. Two time thresholds:

1. TTL — after this, entries become "stale" and trigger async background refresh via loaders 2. Stale duration — after TTL + stale, entries are hard-expired and removed

refreshLoader := func(keys []string) (map[string]*Config, error) {
    return fetchConfigsFromDB(keys)
}

cache := hot.NewHotCache[string, *Config](hot.WTinyLFU, 1_000).
    WithTTL(5 * time.Minute).                              // stale after 5min
    WithRevalidation(1 * time.Minute, refreshLoader).       // hard-expire after 6min total
    WithRevalidationErrorPolicy(hot.KeepOnError).           // keep stale value if refresh fails
    WithJitter(0.1, 30*time.Second).                        // spread expirations
    WithJanitor().
    Build()
defer cache.StopJanitor()

Timeline for an entry set at T=0 with this config:

• T=0 to T=5min: fresh — returned directly
• T=5min to T=6min: stale — returned immediately, background refresh triggered
• T>6min: expired — removed, next Get() blocks on loader

Error policies:

• hot.KeepOnError — if background refresh fails, keep the stale value until hard expiry
• hot.DropOnError — if refresh fails, drop the entry immediately

Use KeepOnError when stale data is better than no data (config caches, product catalogs). Use DropOnError when correctness matters more than availability.

Sharding

Split the cache into N independent segments to reduce lock contention under high concurrency:

cache := hot.NewHotCache[string, *User](hot.WTinyLFU, 100_000).
    WithTTL(5 * time.Minute).
    WithSharding(16, func(key string) uint64 {
        h := fnv.New64a()
        h.Write([]byte(key))
        return h.Sum64()
    }).
    WithJanitor().
    Build()
defer cache.StopJanitor()

Sizing guidance:

• Use powers of 2 (4, 8, 16, 32) for optimal hash distribution
• Rule of thumb: shard count ~= number of CPU cores for high-contention workloads
• Each shard gets capacity / shards items
• Over-sharding (>64 shards) adds overhead without benefit

Missing Key Caching (Negative Caching)

Prevents repeated loader calls for keys that don't exist in the source:

Dedicated missing cache (recommended)

Independent eviction algorithm and capacity — gives fine-grained control:

cache := hot.NewHotCache[string, *User](hot.WTinyLFU, 100_000).
    WithTTL(1 * time.Hour).
    WithMissingCache(hot.LFU, 10_000).  // separate LFU cache for missing keys
    WithLoaders(userLoader).
    WithJanitor().
    Build()
defer cache.StopJanitor()

Shared missing cache

Missing entries stored in the main cache — simpler but uses main cache capacity:

cache := hot.NewHotCache[string, *User](hot.WTinyLFU, 100_000).
    WithTTL(1 * time.Hour).
    WithMissingSharedCache().
    WithLoaders(userLoader).
    WithJanitor().
    Build()
defer cache.StopJanitor()

Manual missing key marking

// Mark individual key as missing
cache.SetMissing("nonexistent-user")
cache.SetMissingWithTTL("temp-missing", 5*time.Minute)

// Batch mark missing
cache.SetMissingMany([]string{"user:404", "user:405"})

Important: Keys(), Values(), All() exclude missing entries — they only return real values.

Loader Chains

Multiple loaders execute sequentially for L1/L2 cache patterns:

redisLoader := func(keys []string) (map[string]*User, error) {
    return redis.MGet(ctx, keys...)
}

dbLoader := func(keys []string) (map[string]*User, error) {
    return db.GetUsersByIDs(ctx, keys)
}

cache := hot.NewHotCache[string, *User](hot.WTinyLFU, 10_000).
    WithTTL(5 * time.Minute).
    WithLoaders(redisLoader, dbLoader).  // Redis first, then DB for remaining
    WithJanitor().
    Build()
defer cache.StopJanitor()

Chain behavior:

• redisLoader called first with all missing keys
• dbLoader called only with keys NOT found by redisLoader
• If both return the same key, dbLoader's value wins (later overwrites earlier)
• Any error stops the chain — partial results from earlier loaders are discarded

Copy-on-Read / Copy-on-Write

Required when cached values are mutable (pointers, slices, maps):

cache := hot.NewHotCache[string, *User](hot.WTinyLFU, 10_000).
    WithTTL(5 * time.Minute).
    WithCopyOnRead(func(u *User) *User {
        copy := *u
        return &copy
    }).
    WithCopyOnWrite(func(u *User) *User {
        copy := *u
        return &copy
    }).
    WithJanitor().
    Build()
defer cache.StopJanitor()

• CopyOnRead — clones at retrieval: callers get independent copies, mutations don't affect cache
• CopyOnWrite — clones at storage: cache holds a snapshot, external mutations to the original don't corrupt cached value
• Use both when callers read and write concurrently. Use only one when the mutation direction is known.

Prometheus Monitoring

Setup

cache := hot.NewHotCache[string, *User](hot.WTinyLFU, 10_000).
    WithTTL(5 * time.Minute).
    WithPrometheusMetrics("user_cache").
    WithJanitor().
    Build()
defer cache.StopJanitor()

prometheus.MustRegister(cache)

Key PromQL Queries

# Hit ratio (target: >80%)
rate(hot_cache_hit_count{cache="user_cache"}[5m]) /
rate(hot_cache_get_count{cache="user_cache"}[5m])

# Eviction rate (high = cache too small or TTL too short)
rate(hot_cache_eviction_count{cache="user_cache"}[5m])

# Cache size vs capacity
hot_cache_len{cache="user_cache"} / hot_cache_capacity{cache="user_cache"}

Alerts to consider:

• Hit rate drops below 70% for >5 minutes — cache may be undersized
• Eviction rate spikes — working set exceeds capacity
• Cache size near capacity — consider increasing capacity or reviewing TTLs

Warm-Up on Startup

Pre-populate the cache before serving traffic:

cache := hot.NewHotCache[string, *User](hot.WTinyLFU, 10_000).
    WithTTL(1 * time.Hour).
    WithWarmUp(func() (map[string]*User, []string, error) {
        users, err := db.GetFrequentUsers(ctx)
        if err != nil {
            return nil, nil, err
        }
        missingKeys := []string{"deleted-user-1", "deleted-user-2"}
        return users, missingKeys, nil  // values + known missing keys + error
    }).
    WithJanitor().
    Build()
defer cache.StopJanitor()

Use WithWarmUpWithTimeout(30*time.Second, fn) to bound startup time.

Graceful Shutdown

Always stop the janitor goroutine before exit:

cache := hot.NewHotCache[string, *User](hot.WTinyLFU, 10_000).
    WithTTL(5 * time.Minute).
    WithJanitor().
    Build()
defer cache.StopJanitor()  // clean up background goroutine

In applications with graceful shutdown orchestration, call cache.StopJanitor() during the shutdown phase alongside other resource cleanup.