Opensearch Personalize Caching Strategies

Marketplace-Research OpenSearch + Personalize Caching Best Practices

A reference distillation of caching strategies for two-sided marketplaces running AWS OpenSearch (search) and AWS Personalize (recommendations behind a microservice). Contains 52 rules across 9 categories, ordered by cascade effect — from the upstream decision of whether to cache, through key design, personalisation boundary, strategy selection, TTL design, stampede protection, observability, and the lower-cascade categories of negative caching and tier composition. Each rule explains the WHY (the cost, latency, or correctness mechanism), shows incorrect-vs-correct code (TypeScript/Node for the microservice layer, Python for batch and analytics, OpenSearch JSON for OS-specific queries, YAML for CDN/Kubernetes), and cites the canonical source — AWS Personalize/OpenSearch/ElastiCache documentation, the XFetch paper (Vattani et al. VLDB 2015), RFC 5861 (stale-while-revalidate), and the engineering blogs of cache infrastructure teams (Netflix EVCache, Pinterest Cachelib, Twitter Twemcache, Cloudflare).

This is the complement to `opensearch-function-scoring-algorithms` — that skill answers "what should the ranking compute?", this skill answers "how do you scale it to production traffic without burning down OpenSearch or Personalize?"

When to Apply

Reach for this skill when:

• Adding caching to a search or recommendation surface for the first time — start with decide-cache-roi-calculation and decide-hot-key-distribution
• A homepage or category page renders 5+ recommenders and Personalize bills are growing faster than traffic — decide-amplification-multiplier, pers-recommender-fan-out-coalescing, pers-cohort-precomputation
• Cache hit rate is suspiciously low (under 30-40%) and you don't know why — key-canonicalize-query, key-strip-volatile-params, key-bucket-numerical-ranges, obs-key-cardinality-tracking
• Personalize is throttling (HTTP 429) during traffic spikes — decide-personalize-quota-budget, neg-cache-throttled-personalize, stamp-circuit-breaker-on-origin-error
• p99 spikes at TTL boundaries — stamp-coalesce-concurrent-misses, stamp-probabilistic-early-expiration, stamp-serve-stale-on-rebuild, ttl-soft-and-hard, ttl-jitter-to-prevent-thundering
• Recommendations stay stale after a model retrain — key-version-the-model, ttl-personalize-solution-version, strat-async-warm-up
• A read-after-write surface shows stale data (user favourites, saved searches) — strat-write-through-mutations, ttl-event-driven-invalidation
• Cache decisions need to be defensible to finance — decide-cache-roi-calculation, obs-cost-attribution, obs-cache-simulation-from-logs
• Anonymous and logged-in traffic mix on the same routes — pers-anonymous-vs-logged-split, tier-cdn-for-anonymous
• Cache is at memory pressure and you need to know whether to upsize, shorten TTL, or change strategy — decide-hot-key-distribution, tier-l2-elasticache-redis, obs-cache-simulation-from-logs
• OpenSearch CPU is high on common queries — tier-opensearch-request-cache, tier-opensearch-filter-context, neg-cache-empty-results
• A traffic spike from a viral link or crawler is hammering the origin — neg-bloom-filter-against-misses, neg-cache-empty-results, stamp-circuit-breaker-on-origin-error

The rules apply to any AWS-based marketplace with OpenSearch and Personalize fronted by an application microservice, regardless of vertical — accommodation, food delivery, fashion, services, jobs, secondhand goods, real estate. Triggers include "cache hit rate", "cache miss storm", "Personalize throttling", "Personalize cost", "multi-recommender page", "cohort caching", "single-flight", "stale-while-revalidate", "XFetch", "OpenSearch slow queries", "ElastiCache sizing", "CloudFront search caching", "Bloom filter cache penetration", and "thundering herd".

The Caching Pipeline

Categories are derived from the request-time caching pipeline. Earlier stages cascade: a wrong "should we cache?" decision wastes everything below; un-canonicalised keys cap hit rate at a fraction of the achievable ceiling; without observability you can't tell whether any of the rules helped.

Request → [1] Decide → [2] Key construction → [3] Personalisation boundary
        → [4] Strategy (read/write path) → [5] TTL/freshness → [6] Stampede protection
        → [8] Negative/defensive → [9] Tier composition (L1/L2/CDN/OS-internal) → Response
                                                ↑
                                [7] Observability (meta-layer applied to all stages:
                                    hit rate by key class, latency-with-and-without,
                                    cost-per-1k, cardinality, staleness, log-replay)

Rule Categories by Priority

Quick Reference

1. Decision & Cost Calculus (CRITICAL)

• `decide-cache-roi-calculation` — Compute Cache ROI Before Adding the Cache
• `decide-cardinality-floor` — Skip Caching When Traffic Distribution is Flat
• `decide-personalize-quota-budget` — Model Personalize TPS Budget Before Choosing a Cache Strategy
• `decide-latency-budget` — Cache Only When Origin p99 Exceeds the Latency Budget
• `decide-amplification-multiplier` — Account for Multi-Recommender Page Amplification
• `decide-hot-key-distribution` — Profile Traffic Distribution Before Sizing the Cache
• `decide-search-vs-personalize-asymmetry` — Cache Candidate Sets for Search, Full Payloads for Personalize

2. Cache Key Design (CRITICAL)

• `key-canonicalize-query` — Canonicalise Queries Before Hashing
• `key-segment-not-user` — Key Recommenders by Cohort When Users Outnumber Cohorts
• `key-version-the-model` — Include the Personalize Solution Version in the Cache Key
• `key-locale-currency-explicit` — Make Locale, Currency, and Timezone Explicit in the Key
• `key-strip-volatile-params` — Strip Volatile and Tracking Params Before Hashing
• `key-bucket-numerical-ranges` — Bucket Continuous Filters Before Hashing
• `key-stable-hash-algorithm` — Use SHA-256 over MD5 for High-Cardinality Keys

3. Personalisation Boundary (HIGH)

• `pers-cohort-precomputation` — Precompute Recommendations Per Cohort Offline
• `pers-anonymous-vs-logged-split` — Route Anonymous Traffic to Global Cache, Logged-in to Cohort Cache
• `pers-cold-start-cache-priority` — Serve Cold-Start Users From Popularity Cache, Skip Personalize
• `pers-recommender-fan-out-coalescing` — Coalesce Multi-Recommender Fan-Out Into Batched Calls
• `pers-shared-candidates-private-ranking` — Cache Retrieval Candidates Globally, Re-Rank Per-User From Cache
• `pers-session-vector-write-through` — Maintain Session Vectors in Cache with Write-Through on Every Event

4. Strategies & Write Paths (HIGH)

• `strat-cache-aside-default` — Use Cache-Aside as the Default Strategy for Read-Heavy Paths
• `strat-refresh-ahead-hot-keys` — Use Refresh-Ahead Only for the Top 1% of Hot Keys
• `strat-write-through-mutations` — Use Write-Through When User Mutations Are Immediately Re-Read
• `strat-precompute-batch` — Precompute the Popular Fraction with Batch Jobs
• `strat-tiered-promotion` — Promote to L1 In-Process Cache on L2 Hit
• `strat-async-warm-up` — Async Warm-Up After Deploy, Restart, or Model Retrain

5. TTL & Freshness (HIGH)

• `ttl-by-content-volatility` — Set TTL From Content Volatility, Not Engineering Convenience
• `ttl-soft-and-hard` — Separate Soft TTL (Async Refresh) from Hard TTL (Sync Miss)
• `ttl-jitter-to-prevent-thundering` — Add Random Jitter to TTL to Prevent Synchronized Expiry
• `ttl-personalize-solution-version` — Pin TTL to Personalize Solution Version, Not Wall Clock
• `ttl-event-driven-invalidation` — Pair TTL with Event-Driven Invalidation for Critical Freshness
• `ttl-bound-by-staleness-tolerance` — Bound TTL by Product Staleness Tolerance, Not the Default

6. Stampede Protection (HIGH)

• `stamp-coalesce-concurrent-misses` — Coalesce Concurrent Misses Into a Single Origin Call
• `stamp-probabilistic-early-expiration` — Use XFetch Probabilistic Early Expiration for Hot Keys
• `stamp-serve-stale-on-rebuild` — Serve Stale While Refresh Is In Flight
• `stamp-circuit-breaker-on-origin-error` — Trip the Circuit Breaker on Origin Errors; Fall Back to Stale
• `stamp-distributed-lock-rebuild` — Use a Distributed Lock to Coordinate Cross-Instance Cache Rebuilds

7. Observability & Empirical Measurement (HIGH)

• `obs-hit-rate-by-key-class` — Track Hit Rate by Key Class, Never Aggregate Only
• `obs-latency-histograms-with-without` — Measure Latency Histograms With-Hit and With-Miss Separately
• `obs-cost-attribution` — Attribute Cost Per Thousand Requests With and Without Cache
• `obs-key-cardinality-tracking` — Sample Key Cardinality Daily; Alert on Explosion
• `obs-stale-served-ratio` — Measure Stale-Served Ratio to Validate TTL Choice
• `obs-cache-simulation-from-logs` — Replay Production Logs Through a Cache Simulator Before Changing TTL or Strategy

8. Negative & Defensive Caching (MEDIUM-HIGH)

• `neg-cache-empty-results` — Cache Empty Search Results With a Short TTL
• `neg-cache-throttled-personalize` — Serve Last-Known-Good When Personalize Throttles
• `neg-bloom-filter-against-misses` — Use a Bloom Filter to Block High-Cardinality Miss Storms
• `neg-poison-pill-detection` — Checksum Cache Entries to Detect and Reject Poisoned Writes

9. Tiered & Edge Caching (MEDIUM-HIGH)

• `tier-l1-in-process` — Use In-Process LRU as L1 for Sub-Millisecond Reads
• `tier-l2-elasticache-redis` — Size L2 ElastiCache to the Cross-Instance Working Set
• `tier-cdn-for-anonymous` — Use CDN for Anonymous Traffic; Bypass for Cookies
• `tier-opensearch-request-cache` — Enable OpenSearch Request Cache for Aggregation-Heavy Queries
• `tier-opensearch-filter-context` — Put Reusable Predicates in Filter Context for Segment-Level Caching

How to Use

For a focused question ("should I cache this?", "why is my hit rate low?", "how do I survive Personalize throttling?"), jump directly to the relevant rule — each is self-contained with the WHY, code, and citation.

For a full caching-design review of a new or struggling surface, work the categories top-to-bottom. The cascade is real: a wrong decide-cache-roi-calculation wastes engineering effort on a cache that doesn't pay; a leaky key-canonicalize-query caps the achievable hit rate; a missing pers-cohort-precomputation keeps Personalize bills proportional to MAU. Stampede and observability are mandatory once hit rate exceeds 90% — the 10% miss in a thundering herd kills the origin, and without per-class hit-rate dashboards you can't tell.

For tuning an existing cache empirically, start with obs-cache-simulation-from-logs (replay your logs through what-if configs) and pair with obs-hit-rate-by-key-class, obs-cost-attribution, and obs-stale-served-ratio for the dashboards. The trio answers: is the cache doing its job, what does it cost, and are users seeing stale data?

For the multi-recommender homepage problem specifically (the most common Personalize cost-explosion pattern), the priority order is: decide-amplification-multiplier → pers-cohort-precomputation → pers-recommender-fan-out-coalescing → pers-anonymous-vs-logged-split. These four typically cut Personalize spend by 70-90% on consumer marketplaces.

For the "Personalize is throttling under load" incident, the priority is: stamp-circuit-breaker-on-origin-error → neg-cache-throttled-personalize → decide-personalize-quota-budget. The first two stabilise the user-facing impact; the third right-sizes minProvisionedTPS so it doesn't happen again.

For sibling-skill cross-reference, see `opensearch-function-scoring-algorithms` — that skill covers what to compute in OpenSearch (function_score, kNN, RRF, rank_feature, decay, LTR, MMR, evaluation). This skill covers how to cache it so the cluster survives production traffic.

Read section definitions for the cascade-impact rationale, or the rule template when adding a new rule.

Related Skills

• `opensearch-function-scoring-algorithms` — Research-backed ranking, retrieval, and evaluation rules for OpenSearch. The "what to compute"; this skill is the "how to scale it."

Reference Files

Caching Strategies for AWS OpenSearch and AWS Personalize

Version 0.1.0 Marketplace-Research May 2026

Note:

This document is mainly for agents and LLMs to follow when maintaining,

generating, or refactoring codebases. Humans may also find it useful,

but guidance here is optimized for automation and consistency by AI-assisted workflows.

---

Abstract

Comprehensive caching reference for two-sided marketplaces running AWS OpenSearch (search) and AWS Personalize (recommenders fronted by a microservice). Contains 52 rules across 9 categories ordered by cascade effect in the request-time caching pipeline — from the upstream decision of whether to cache (cost calculus, Personalize TPS / minProvisionedTPS modelling, Zipf-distribution profiling, latency budget, multi-recommender amplification), through cache-key design (canonicalisation, cohort vs user, solution-version pinning, locale/currency, volatile-param stripping, bucketed ranges, stable hashing), personalisation boundary (anonymous/logged split, cold-start fallback, fan-out coalescing, shared-candidates private-ranking, session-vector write-through), strategies (cache-aside default, refresh-ahead for hot keys, write-through for mutations, batch precomputation, tiered L1+L2 promotion, async warm-up), TTL and freshness (volatility-driven TTL, soft/hard TTL, jitter, model-version pinning, event-driven invalidation, staleness-budget caps), stampede protection (single-flight, XFetch probabilistic early expiry, stale-while-revalidate, circuit breaker, distributed lock), observability and empirical measurement (hit-rate by key class, latency-with-and-without histograms, cost-per-1k attribution, key-cardinality drift, stale-served ratio, log-replay simulation), to the lower-cascade negative & defensive caching (empty results, Personalize-throttle fallback, Bloom-filter cache-penetration defense, poison-pill detection) and tier composition (in-process LRU, ElastiCache Redis sizing, CloudFront for anonymous, OpenSearch request cache, OpenSearch filter-context auto-cache). Each rule explains the underlying mechanism, shows incorrect-vs-correct code (TypeScript/Node for microservices, Python for batch/analytics, OpenSearch JSON for OS-specific queries, YAML for CDN/Kubernetes), and cites the canonical source — AWS Personalize/OpenSearch/ElastiCache documentation, the XFetch paper (Vattani, Chierichetti, Lowenstein VLDB 2015), RFC 5861 (stale-while-revalidate), and the engineering blogs of cache infrastructure teams (Netflix EVCache, Pinterest Cachelib, Twitter Twemcache, Cloudflare). Complements the opensearch-function-scoring-algorithms skill — that one covers what the ranking computes; this one covers how to cache it without overwhelming OpenSearch or Personalize.

---

Table of Contents

1. Decision & Cost Calculus — CRITICAL

• 1.1 Account for Multi-Recommender Page Amplification — CRITICAL (5-10x request-rate multiplier hidden by per-recommender metrics)
• 1.2 Cache Candidate Sets for Search, Full Payloads for Personalize — CRITICAL (2-3x storage savings and faster invalidation by caching at the right grain)
• 1.3 Cache Only When Origin p99 Exceeds the Latency Budget — CRITICAL (prevents caches that add latency on hits)
• 1.4 Compute Cache ROI Before Adding the Cache — CRITICAL (prevents shipping caches that cost more than they save)
• 1.5 Model Personalize TPS Budget Before Choosing a Cache Strategy — CRITICAL (prevents minProvisionedTPS bills 3-5x above actual usage)
• 1.6 Profile Traffic Distribution Before Sizing the Cache — CRITICAL (80/20 sizing without measurement wastes 50-200% of cache capacity)
• 1.7 Skip Caching When Traffic Distribution is Flat — CRITICAL (caching flat-distribution traffic produces <15% hit rate)

2. Cache Key Design — CRITICAL

• 2.1 Bucket Continuous Filters Before Hashing — CRITICAL (5-20x hit rate increase on price/distance/date filters)
• 2.2 Canonicalise Queries Before Hashing — CRITICAL (3-10x hit rate increase by collapsing equivalent queries)
• 2.3 Include the Personalize Solution Version in the Cache Key — CRITICAL (prevents serving stale recommendations for hours after retrain)
• 2.4 Key Recommenders by Cohort When Users Outnumber Cohorts — CRITICAL (50-500x hit rate increase by collapsing N users into K cohorts)
• 2.5 Make Locale, Currency, and Timezone Explicit in the Key — CRITICAL (prevents silent cross-locale cache poisoning)
• 2.6 Strip Volatile and Tracking Params Before Hashing — CRITICAL (hit rate collapses to <5% with UTM/request-id leakage)
• 2.7 Use SHA-256 over MD5 for High-Cardinality Keys — CRITICAL (prevents silent key collisions on >100M keyspace)

3. Personalisation Boundary — HIGH

• 3.1 Cache Retrieval Candidates Globally, Re-Rank Per-User From Cache — HIGH (70-90% hit rate on the candidate set with per-user personalisation preserved)
• 3.2 Coalesce Multi-Recommender Fan-Out Into Batched Calls — HIGH (5-10x latency and TPS reduction on multi-recommender pages)
• 3.3 Maintain Session Vectors in Cache with Write-Through on Every Event — HIGH (<1ms session-vector reads for real-time rerank)
• 3.4 Precompute Recommendations Per Cohort Offline — HIGH (80-95% reduction in Personalize TPS, sub-millisecond serve time)
• 3.5 Route Anonymous Traffic to Global Cache, Logged-in to Cohort Cache — HIGH (90%+ hit rate on anonymous traffic, 60-80% on logged-in)
• 3.6 Serve Cold-Start Users From Popularity Cache, Skip Personalize — HIGH (cuts Personalize cost on cold users by 100%, faster latency)

4. Strategies & Write Paths — HIGH

• 4.1 Async Warm-Up After Deploy, Restart, or Model Retrain — HIGH (avoids 5-30 min of degraded latency after cold start)
• 4.2 Precompute the Popular Fraction with Batch Jobs — HIGH (serves 40-60% of traffic from precomputed cache at near-zero per-request cost)
• 4.3 Promote to L1 In-Process Cache on L2 Hit — HIGH (95%+ of L1-eligible reads served in <100µs)
• 4.4 Use Cache-Aside as the Default Strategy for Read-Heavy Paths — HIGH (simplest correctness-preserving strategy, no coupling to writers)
• 4.5 Use Refresh-Ahead Only for the Top 1% of Hot Keys — HIGH (eliminates p99 spikes at TTL expiry on hot keys)
• 4.6 Use Write-Through When User Mutations Are Immediately Re-Read — HIGH (eliminates the "I saved it but I don't see it" UX bug)

5. TTL & Freshness — HIGH

• 5.1 Add Random Jitter to TTL to Prevent Synchronized Expiry — HIGH (smooths origin-load spikes from cohort batch writes)
• 5.2 Bound TTL by Product Staleness Tolerance, Not the Default — HIGH (prevents serving non-compliant data hours after the rule changed)
• 5.3 Pair TTL with Event-Driven Invalidation for Critical Freshness — HIGH (closes the gap between mutation and cache update from TTL-bound to seconds)
• 5.4 Pin TTL to Personalize Solution Version, Not Wall Clock — HIGH (prevents serving previous-model output for hours after retrain)
• 5.5 Separate Soft TTL (Async Refresh) from Hard TTL (Sync Miss) — HIGH (keeps p99 flat across TTL boundaries)
• 5.6 Set TTL From Content Volatility, Not Engineering Convenience — HIGH (matches staleness to product tolerance, not a default)

6. Stampede Protection — HIGH

• 6.1 Coalesce Concurrent Misses Into a Single Origin Call — HIGH (1 origin call per key per miss instead of N concurrent)
• 6.2 Serve Stale While Refresh Is In Flight — HIGH (keeps p99 flat under origin slowdown or failure)
• 6.3 Trip the Circuit Breaker on Origin Errors; Fall Back to Stale — HIGH (prevents cascade failure when OpenSearch or Personalize errors)
• 6.4 Use a Distributed Lock to Coordinate Cross-Instance Cache Rebuilds — HIGH (prevents N-machine duplicate origin calls during a fleet-wide cold start)
• 6.5 Use XFetch Probabilistic Early Expiration for Hot Keys — HIGH (smooths origin load by spreading refresh decisions across the TTL window)

7. Observability & Empirical Measurement — HIGH

• 7.1 Attribute Cost Per Thousand Requests With and Without Cache — HIGH (turns "cache helps" intuition into a finance-grade number)
• 7.2 Measure Latency Histograms With-Hit and With-Miss Separately — HIGH (reveals the true latency saving and the miss-path tail)
• 7.3 Measure Stale-Served Ratio to Validate TTL Choice — HIGH (makes TTL tuning empirical instead of guesswork)
• 7.4 Replay Production Logs Through a Cache Simulator Before Changing TTL or Strategy — HIGH (predicts hit rate and cost impact before shipping the change)
• 7.5 Sample Key Cardinality Daily; Alert on Explosion — HIGH (catches canonicalisation regressions before they collapse hit rate)
• 7.6 Track Hit Rate by Key Class, Never Aggregate Only — HIGH (aggregate hit rate hides 5-50% per-class variance)

8. Negative & Defensive Caching — MEDIUM-HIGH

• 8.1 Cache Empty Search Results With a Short TTL — MEDIUM-HIGH (prevents repeated empty-query CPU on OpenSearch)
• 8.2 Checksum Cache Entries to Detect and Reject Poisoned Writes — MEDIUM-HIGH (prevents one bad write from corrupting hours of traffic)
• 8.3 Serve Last-Known-Good When Personalize Throttles — MEDIUM-HIGH (prevents user-visible failures during Personalize 429s)
• 8.4 Use a Bloom Filter to Block High-Cardinality Miss Storms — MEDIUM-HIGH (rejects invalid-id traffic before it reaches cache or origin)

9. Tiered & Edge Caching — MEDIUM-HIGH

• 9.1 Enable OpenSearch Request Cache for Aggregation-Heavy Queries — MEDIUM-HIGH (10-100x faster repeated aggregations at the shard level)
• 9.2 Put Reusable Predicates in Filter Context for Segment-Level Caching — MEDIUM-HIGH (5-50x speedup on repeated filters via auto-caching at segment level)
• 9.3 Size L2 ElastiCache to the Cross-Instance Working Set — MEDIUM-HIGH (prevents L2 eviction churn while not over-provisioning)
• 9.4 Use CDN for Anonymous Traffic; Bypass for Cookies — MEDIUM-HIGH (serves anonymous search/recs at edge with zero origin RTT)
• 9.5 Use In-Process LRU as L1 for Sub-Millisecond Reads — MEDIUM-HIGH (1000x faster than Redis for hot keys; eliminates network RTT)

---

References

1. https://docs.aws.amazon.com/personalize/latest/dg/API_CreateCampaign.html 2. https://docs.aws.amazon.com/personalize/latest/dg/limits.html 3. https://docs.aws.amazon.com/personalize/latest/dg/campaigns.html 4. https://docs.aws.amazon.com/personalize/latest/dg/getting-recommendations.html 5. https://docs.aws.amazon.com/personalize/latest/dg/recommendations-batch.html 6. https://docs.aws.amazon.com/personalize/latest/dg/recording-item-interaction-events.html 7. https://docs.aws.amazon.com/personalize/latest/dg/native-recipe-new-item-USER_PERSONALIZATION.html 8. https://docs.aws.amazon.com/personalize/latest/dg/eventbridge.html 9. https://docs.aws.amazon.com/personalize/latest/dg/updating-campaign.html 10. https://aws.amazon.com/personalize/pricing/ 11. https://aws.amazon.com/blogs/machine-learning/create-a-batch-recommendation-pipeline-using-amazon-personalize-with-no-code/ 12. https://aws.amazon.com/blogs/machine-learning/amazon-personalize-can-now-create-up-to-50-better-recommendations-for-fast-changing-catalogs-of-new-products-and-fresh-content/ 13. https://docs.opensearch.org/latest/search-plugins/caching/request-cache/ 14. https://docs.opensearch.org/latest/search-plugins/caching/index/ 15. https://docs.opensearch.org/latest/query-dsl/query-filter-context/ 16. https://docs.opensearch.org/latest/query-dsl/term/range/ 17. https://docs.opensearch.org/latest/search-plugins/search-pipelines/index/ 18. https://opensearch.org/blog/understanding-index-request-cache/ 19. https://opster.com/guides/opensearch/opensearch-basics/cache-node-request-shard-data-field-data-cache/ 20. https://bigdataboutique.com/blog/properly-use-elasticsearch-query-cache-to-accelerate-search-performance-9566ad 21. https://opensourceconnections.com/blog/2017/07/10/caching_in_elasticsearch/ 22. https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/Strategies.html 23. https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/BestPractices.html 24. https://docs.aws.amazon.com/whitepapers/latest/database-caching-strategies-using-redis/caching-patterns.html 25. https://aws.amazon.com/elasticache/pricing/ 26. https://aws.amazon.com/blogs/database/work-with-cluster-mode-on-amazon-elasticache-for-redis/ 27. https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/ 28. https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/ 29. https://cseweb.ucsd.edu/~avattani/papers/cache_stampede.pdf 30. https://datatracker.ietf.org/doc/html/rfc5861 31. https://www.rfc-editor.org/rfc/rfc9111 32. https://www.rfc-editor.org/rfc/rfc6234 33. https://en.wikipedia.org/wiki/Cache_stampede 34. https://en.wikipedia.org/wiki/Bloom_filter 35. https://web.dev/articles/stale-while-revalidate 36. https://www.fastly.com/documentation/guides/concepts/edge-state/cache/stale/ 37. https://www.debugbear.com/docs/stale-while-revalidate 38. https://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html 39. https://martinfowler.com/bliki/CircuitBreaker.html 40. https://pkg.go.dev/golang.org/x/sync/singleflight 41. https://github.com/ben-manes/caffeine 42. https://github.com/Netflix/EVCache 43. https://github.com/Netflix/rend 44. https://netflix.github.io/EVCache/features/ 45. https://netflixtechblog.com/announcing-evcache-distributed-in-memory-datastore-for-cloud-c26a698c27f7 46. https://blog.bytebytego.com/p/how-netflix-warms-petabytes-of-cache 47. https://medium.com/pinterest-engineering/feature-caching-for-recommender-systems-w-cachelib-8fb7bacc2762 48. https://medium.com/pinterest-engineering/pinnersage-multi-modal-user-embedding-framework-for-recommendations-at-pinterest-bfd116b49475 49. https://stackshare.io/pinterest/scaling-cache-infrastructure-at-pinterest 50. https://github.com/facebook/CacheLib 51. https://blog.x.com/engineering/en_us/a/2012/caching-with-twemcache 52. https://airbnb.tech/uncategorized/embedding-based-retrieval-for-airbnb-search/ 53. https://medium.com/airbnb-engineering/listing-embeddings-for-similar-listing-recommendations-and-real-time-personalization-in-search-601172f7603e 54. https://blog.cloudflare.com/when-bloom-filters-dont-bloom/ 55. https://developers.cloudflare.com/cache/how-to/cache-keys/ 56. https://developers.cloudflare.com/cache/concepts/cache-behavior/ 57. https://developers.cloudflare.com/cache/how-to/purge-cache/purge-by-tags/ 58. https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/controlling-the-cache-key.html 59. https://www.fastly.com/documentation/guides/concepts/edge-state/purging/ 60. https://cloud.google.com/cdn/docs/using-negative-caching 61. https://pages.cs.wisc.edu/~cao/papers/zipf-implications.html 62. https://www.usenix.org/system/files/conference/nsdi18/nsdi18-beckmann.pdf 63. https://stefanheule.com/papers/edbt2013-hyperloglog.pdf 64. https://redis.io/docs/latest/develop/data-types/probabilistic/hyperloglogs/ 65. https://redis.io/docs/latest/develop/use/patterns/distributed-locks/ 66. https://redis.io/docs/latest/operate/oss_and_stack/management/scaling/ 67. https://redis.io/docs/latest/develop/reference/eviction/ 68. https://www.designgurus.io/course-play/grokking-scalable-systems-for-interviews/doc/what-is-negative-caching-and-when-should-you-cache-404-or-empty-results 69. https://prometheus.io/docs/practices/histograms/ 70. https://www.unicode.org/reports/tr15/ 71. https://www.iana.org/time-zones 72. https://gdpr-info.eu/art-17-gdpr/ 73. https://github.com/graphql/dataloader

---

Source Files

This document was compiled from individual reference files. For detailed editing or extension:

{Same as title}

{1-3 sentences explaining the WHY — what happens when this pattern is missing, what the cascade or cost mechanism is. This is the highest-signal section. The model generalises from understood reasoning, not from dictation: explain the mechanism, not the rule.}

Incorrect ({label describing what's wrong}):

```{language} {Production-realistic code showing the anti-pattern. Avoid strawman examples — the bad code should be the kind a competent engineer would actually write.} {// Comments explain the COST (latency, cost, failure mode) not the syntax}

**Correct ({label describing what's right}):**

{Production-realistic code showing the right pattern. Keep the diff from "Incorrect" minimal — same variable names, same overall structure, the change is visible in a few lines.} {// Comments explain the BENEFIT (saved cost, latency win, prevented failure)}

{Optional sections — include only when they add information:}

**Why {specific design choice}:**
{Detail-level explanation of a tricky bit. Useful for nuances like "why 60s TTL not 30s?" or "why SHA-256 not MD5?".}

**When NOT to use this pattern:**
- {Exception 1 with reason}
- {Exception 2 with reason}

**Alternative ({context}):**
{Alternative approach for a specific context, e.g. "with Memcached instead of Redis," "for write-heavy paths," "when running cluster mode."}

**Companion rules:**
- [`{prefix}-{slug}`](../{prefix}-{slug}.md) — {one-line description of how it relates}

**Validation:**
{How to verify in observability that the rule is being followed — usually a specific metric to watch.}

Reference: [{Source 1 Title}]({URL 1}) · [{Source 2 Title}]({URL 2})

Sections

This file defines all sections, their ordering, impact levels, and descriptions. The section ID (in parentheses) is the filename prefix used to group rules.

Categories appear in impact order (CRITICAL → MEDIUM-HIGH). The request lifecycle for a search-or-recommendation surface runs in a related order: a request arrives → you decide whether to cache it → if yes, a key is constructed → the key is bucketed by the personalisation boundary → a strategy chooses how to read/write → a TTL governs freshness → stampede protection guards the miss → tier composition determines where the lookup happens → and observability tells you whether any of the above worked. Defensive (negative) caching is applied opportunistically across the pipeline. The cascade is real: a wrong "should we cache?" decision wastes everything below it; un-canonicalised keys cap hit-rate at a fraction of the theoretical ceiling; and without measurement infrastructure you cannot tell whether any of the 50 rules helped.

---

1. Decision & Cost Calculus (decide)

Impact: CRITICAL Description: The decision of whether to cache at all, and the math that justifies it. Personalize bills per provisioned TPS and per transaction; OpenSearch cluster CPU is finite; multi-recommender pages amplify backend load by 5-10×. Caching the wrong things (flat-distribution traffic, low-cardinality misses, ultra-fresh data) burns infrastructure for negligible benefit, while not caching the right things (hot-cohort recommendations, repeated searches, anonymous traffic) starves the cluster. The rules here are the gate every other rule depends on.

2. Cache Key Design (key)

Impact: CRITICAL Description: Cache keys define the hit-rate ceiling. Un-canonicalised whitespace, sort-order in filter arrays, leaky tracking params (UTM, request_id, timestamp), full-precision numerical filters, missing locale/currency/model-version — each silently collapses hit rate by an order of magnitude. Two requests for the same logical result must produce the same key, and two requests for different logical results must not collide. Wilson's "the cache works fine, the hit rate is just 4%" almost always traces to this category.

3. Personalisation Boundary (pers)

Impact: HIGH Description: Where shared cache ends and per-user cache begins. The single biggest lever for Personalize cost on a multi-recommender page. Cohort precomputation collapses N users into K cohorts (often N/K > 100), the anonymous/logged split lets anonymous traffic reuse a global cache, recommender fan-out coalescing batches 5+ Personalize calls per page, and session-vector write-through keeps real-time personalisation under 1ms without blowing up TPS. Getting this wrong means paying Personalize per-user per-request when 80% of users belong to one of a small number of cohorts.

4. Strategies & Write Paths (strat)

Impact: HIGH Description: How data enters and exits the cache — cache-aside (lazy loading), read-through, write-through, refresh-ahead, tiered-promotion, async warm-up. The choice is dictated by mutation frequency, staleness tolerance, and the cost ratio between origin and cache, not by taste. Cache-aside is the default for read-heavy paths; write-through is mandatory when the user mutates state they immediately re-read; refresh-ahead is reserved for the top 1% of hot keys where TTL-expiry spikes would be visible.

5. TTL & Freshness (ttl)

Impact: HIGH Description: TTL is the freshness-vs-hit-rate dial — most teams set it once at "5 minutes" and never re-tune. Soft/hard TTL separates async refresh from sync miss, jittered TTL prevents synchronized expiry stampedes, event-driven invalidation closes the gap between mutation and cache update, Personalize-solution-version-pinned TTL invalidates on model retrain, and per-content-class volatility bounds (inventory: minutes, user prefs: days) come from product staleness tolerance, not engineering convenience.

6. Stampede Protection (stamp)

Impact: HIGH Description: Concurrent-miss protection. At >90% hit rate, the 10% miss in a thundering herd kills the origin — N machines simultaneously refresh the same hot key on TTL expiry. Single-flight (Go semantics) collapses concurrent misses into one origin call; XFetch (Vattani et al. VLDB 2015) probabilistically refreshes before expiry to spread load; stale-while-revalidate (RFC 5861) returns stale during refresh; circuit-breaker on origin error keeps stale rather than propagating failure. These are not optional once cached traffic dwarfs origin capacity.

7. Observability & Empirical Measurement (obs)

Impact: HIGH Description: The empirical backbone. Aggregate hit rate hides everything — measure hit rate by key class (search-anon, search-logged, recommender-popular, recommender-personalized). Measure latency histograms with-hit and with-miss separately to know the actual saving. Track cost-per-thousand-requests with cache vs without cache. Sample key cardinality daily to catch a normalization regression. Measure the stale-served ratio to know if TTLs are right. Replay yesterday's logs through a simulator before changing TTL or strategy. Without this category, every other rule is a guess.

8. Negative & Defensive Caching (neg)

Impact: MEDIUM-HIGH Description: Caching the absence of a result, and surviving downstream failures. Empty search results still consume OpenSearch cluster CPU — cache the "0 hits" with a short TTL. Personalize 429s should not propagate to users — serve last-known-good. High-cardinality miss storms (slug-not-found, hash-not-found) overwhelm Redis lookup before the origin — a Bloom filter in front rejects them. Poison-pill detection (checksums on cache entries) prevents one bad write from corrupting hours of traffic.

9. Tiered & Edge Caching (tier)

Impact: MEDIUM-HIGH Description: Composition of cache layers — L1 (in-process, Caffeine/Guava/Ristretto) for hot keys at sub-millisecond latency, L2 (ElastiCache Redis/Memcached) for cross-instance reuse, CDN (CloudFront/Fastly) for anonymous traffic, OpenSearch's own shard-request cache and segment-level filter cache. Each tier has different hit-rate characteristics, latency, eviction semantics, and consistency guarantees. The rules cover when to add a tier, where to invalidate, and how to size each.

Account for Multi-Recommender Page Amplification

A typical home page renders 5-10 recommenders ("recently viewed", "trending in your category", "similar to your wishlist", "popular near you", "complete the set"). Per-recommender dashboards show 100 req/s each — but the user triggers all of them per page load. The actual fan-out from the user-facing service to Personalize is 5-10×, which is invisible if you only look at per-campaign TPS. Auto-scalers, alerts, and minProvisionedTPS settings tuned per-campaign all underestimate cost and overestimate available headroom by the same factor.

Incorrect (size each campaign as if independent):

// Five recommenders, each campaign sized for "100 req/s peak"
// minProvisionedTPS = 100 per campaign = $864/day floor × 5 = $4320/day floor.
//
// A homepage at 100 page views/s triggers 500 Personalize calls/s.
// Each campaign hits its 100 TPS minimum — looks fine on a per-recommender dashboard.
// Aggregate cost is 5× what the team modelled.

async function renderHomepage(userId: string) {
  const [recent, trending, similar, popular, complete] = await Promise.all([
    personalize.getRecommendations({ campaignArn: RECENT, userId }),
    personalize.getRecommendations({ campaignArn: TRENDING, userId }),
    personalize.getRecommendations({ campaignArn: SIMILAR, userId }),
    personalize.getRecommendations({ campaignArn: POPULAR, userId }),
    personalize.getRecommendations({ campaignArn: COMPLETE, userId }),
  ]);
  return { recent, trending, similar, popular, complete };
}

Correct (treat the page as the unit, cache at the page boundary, coalesce upstream calls):

// Step 1: aggregate the dashboard at the PAGE level (page_views_per_sec)
//         not per-campaign. This is the actual demand signal.
//
// Step 2: cache at the recommender output, with a single cache lookup batch.
//
// Step 3: for cold cache, coalesce shared inputs (user features, context) to
//         avoid fetching them 5x per page.

async function renderHomepage(userId: string, ctx: PageContext) {
  const cohortKey = await getCohortKey(userId);  // one lookup, not five
  const cacheKeys = RECOMMENDER_IDS.map(id => `rec:${id}:${cohortKey}:${ctx.locale}`);

  // Batch L2 lookup — one round trip for all 5 recommenders
  const cached = await redis.mget(...cacheKeys);
  const misses = cached
    .map((v, i) => v === null ? RECOMMENDER_IDS[i] : null)
    .filter(Boolean);

  // Only call Personalize for the misses
  const fresh = await Promise.all(
    misses.map(id => personalize.getRecommendations({
      campaignArn: CAMPAIGN_BY_ID[id],
      userId,
    }))
  );

  // Write-back the misses with jittered TTL (see ttl-jitter-to-prevent-thundering)
  await Promise.all(fresh.map((r, idx) =>
    redis.set(`rec:${misses[idx]}:${cohortKey}:${ctx.locale}`,
              JSON.stringify(r), 'EX', 600 + rand(0, 120))
  ));

  return assembleRecommenders(cached, fresh);
}
// At 100 page views/s with 70% per-recommender hit rate:
//   actual Personalize calls/s = 100 * 5 * 0.3 = 150  (down from 500)
//   cost reduction = 70%, infrastructure aligned with PAGE traffic, not campaign traffic.

Dashboard rule: primary metric is personalize_calls_per_page_view, target = n_recommenders * (1 - hit_rate). If actuals exceed target by >20%, your cache key is leaking (key-canonicalize-query) or your cohorts are too granular (pers-cohort-precomputation).

Reference: Pinterest: Feature Caching for Recommender Systems

Compute Cache ROI Before Adding the Cache

A cache is justified only when request_rate × origin_cost × hit_rate > cache_infra_cost + serialization_cost. Most teams add a cache because "Redis is cheap," then discover hit rates below 30%, working sets larger than memory, or serialization overhead exceeding the saved origin call. For AWS Personalize the math is sharper because you pay tiered per-transaction (currently $0.0556 per 1k for the first 72M/month, dropping to $0.0278 then $0.0139 at scale) plus a minProvisionedTPS floor billed per second regardless of traffic — and a 60% hit rate halves the variable portion only if the cache costs less than the savings.

Incorrect (cache first, measure later):

// Ship the cache, hope it helps. No model of expected hit rate or cost.
async function getRecommendations(userId: string): Promise<Item[]> {
  const cached = await redis.get(`recs:${userId}`);
  if (cached) return JSON.parse(cached);
  const result = await personalize.getRecommendations({ userId });
  await redis.set(`recs:${userId}`, JSON.stringify(result), 'EX', 300);
  return result;
}
// Three months later: hit rate is 4%, Redis is at $800/mo, Personalize bill unchanged.

Correct (model the ROI before writing the code):

// ROI worksheet — fill in measured values, then decide.
//
// Inputs (from production logs over a 7-day window):
//   request_rate           = 1000 req/s
//   personalize_cost       = $0.0556 per 1k transactions  -> $0.0000556/req  (tier 1)
//   p99_origin_latency_ms  = 80
//   measured_hit_rate*     = 0.55  (* from a shadow cache running 1 week without serving)
//   working_set_keys       = 2_400_000   (95th percentile of distinct keys/24h)
//   bytes_per_value        = 8_000        (compressed)
//
// Monthly volume = 1000 * 86400 * 30 = 2.59B requests/mo.
// First 72M billed at $0.0556/1k, next 648M at $0.0278/1k, remainder at $0.0139/1k.
//
// Cache cost @ ElastiCache cache.r7g.large: ~$120/mo, ~13GB usable
//   bytes_needed = working_set_keys * bytes_per_value = 19.2 GB  -> need r7g.xlarge ~$240/mo
//
// Origin savings  ≈ avg_unit_cost * 2.59B * hit_rate ≈ $0.0000167/req * 2.59B * 0.55
//                ≈ ~$24k/mo on the variable portion (rough; recompute per actual tier mix)
// Cache infra     = $240/mo
// Net savings     = ~$23.7k/mo at this volume — a clear win.
//
// At smaller scale (e.g. 50 req/s = 130M req/mo) the numbers shrink ~20x; cache wins
// flip negative once the working set forces a multi-node cluster. ALWAYS run the
// worksheet with YOUR measured volume and tier before deciding.

The "shadow cache" technique: run the cache code path without serving from it — only log what would have been a hit. This measures the actual hit rate against your real traffic distribution before committing infra spend. Pre-deployment hit-rate estimates are often off by a meaningful multiple; the shadow cache is the cheapest way to find that out before you've sized infra around the wrong number.

When NOT to cache at all:

• Hit rate below 20% — cache infra usually exceeds savings
• Working set exceeds memory budget by >2× — eviction churn destroys hit rate
• Origin latency under 10ms — serialization/network overhead is comparable
• Per-request cost under $0.00001 (e.g. a local index lookup) — Personalize is not in this class; OpenSearch shard cache might be

Reference: AWS Personalize Pricing · AWS ElastiCache Pricing

Skip Caching When Traffic Distribution is Flat

Web traffic follows a Zipf-like distribution: the top 1% of keys account for ~40-50% of requests, top 10% for ~80%. When this distribution holds, even a small cache delivers high hit rates. When it doesn't — for example, deep-link product pages where each URL is hit by a handful of users, or hyper-personalised recommenders where every request is unique — the working set rivals the entire keyspace and hit rate stays below 15% regardless of cache size. Breslau et al. (1999) showed this empirically across multiple web traces, and the same applies to search queries and recommender outputs.

Incorrect (cache everything regardless of distribution):

// Caching every search query, including the long tail of one-off queries.
async function search(query: string, filters: Filters) {
  const key = hashKey(query, filters);
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);
  const result = await opensearch.search(buildQuery(query, filters));
  await redis.set(key, JSON.stringify(result), 'EX', 600);
  return result;
}
// Result: 12% hit rate because half the queries are typed once and never repeated.
// Redis fills up with single-use entries that evict the hot ones.

Correct (gate caching by query-popularity threshold):

// Profile traffic over 7 days, identify queries seen >= N times, cache only those.
//
// Build the popularity set offline:
//   query_count >= 10 in last 7 days  -> eligible for caching
//   else                              -> bypass cache, go straight to origin

const POPULAR_QUERIES = await loadFromS3('s3://search-analytics/popular-queries-v1.bloom');
//  ^ Bloom filter of normalized-query-hashes seen >= 10 times in last 7 days
//    Refreshed nightly. ~2MB for 1M popular queries at 1% FP rate.

async function search(query: string, filters: Filters) {
  const normalized = canonicalize(query, filters);

  if (!POPULAR_QUERIES.has(normalized.queryHash)) {
    // Cold long-tail query — go to OpenSearch directly. Cache would just churn.
    return opensearch.search(buildQuery(query, filters));
  }

  const cached = await redis.get(normalized.fullKey);
  if (cached) return JSON.parse(cached);
  const result = await opensearch.search(buildQuery(query, filters));
  await redis.set(normalized.fullKey, JSON.stringify(result), 'EX', 600);
  return result;
}
// Result: hit rate on cached requests rises from 12% to 70%+ because Redis
// only holds queries that will repeat. The long tail bypasses cache entirely.

How to measure distribution:

• Plot log(rank) vs log(frequency) over 7 days of traffic — straight line = Zipf, slope = α
• α ≥ 1.0: caching is highly effective (recommender outputs for logged-in users with cohorts often hit α ~ 1.2)
• 0.7 ≤ α < 1.0: caching helps but needs careful sizing
• α < 0.7: cache the hot heads only, route the tail directly to origin

Personalize-specific signal: if unique_users_per_recommender_per_hour / total_calls_per_recommender_per_hour > 0.5, per-user caching will not work — switch to cohort caching (pers-cohort-precomputation).

Reference: Breslau, Cao, Fan, Phillips, Shenker — Web Caching and Zipf-like Distributions (INFOCOM 1999)

Profile Traffic Distribution Before Sizing the Cache

Cache sizing has a closed-form answer given the traffic distribution: the cache hits the desired hit-rate when it holds the working set up to that percentile. For Zipf-like traffic with α=1, a cache holding the top 10% of keys delivers ~78% hit rate; doubling cache size to 20% adds ~9 percentage points. Without measuring α and the long-tail shape, teams either under-provision (constant eviction churn) or over-provision (paying for cache that holds keys never accessed twice). Both are visible in production only after the cache is in the request path.

Incorrect (size by guess, then iterate in production):

# "Let's start with 16GB Redis, that feels right."
# Six months later:
#   Working set is 80GB. Hit rate plateaued at 31%.
#   Or:
#   Working set is 2GB. We're paying for 16GB of unused capacity.

Correct (profile, fit a curve, then size):

# scripts/profile_traffic.py — run against 7 days of cache-key access logs
import math
from collections import Counter

# Log line per request: {"timestamp": ..., "cache_key": "...", "bytes": ...}
key_counts = Counter()
key_bytes  = {}

for log_line in read_logs("s3://search-logs/cache-keys/7d/"):
    rec = json.loads(log_line)
    key_counts[rec["cache_key"]] += 1
    key_bytes[rec["cache_key"]] = rec["bytes"]

# Sort by frequency descending
ranked = sorted(key_counts.items(), key=lambda kv: -kv[1])

# Compute cumulative hit-rate curve as a function of cache size in bytes
total_requests = sum(key_counts.values())
cumulative_hits = 0
cumulative_bytes = 0

print("cache_size_gb,hit_rate")
for key, count in ranked:
    cumulative_hits  += count
    cumulative_bytes += key_bytes[key]
    if cumulative_bytes % (1024**3) < key_bytes[key]:  # crossed a GB boundary
        gb = cumulative_bytes / 1024**3
        hit_rate = cumulative_hits / total_requests
        print(f"{gb:.1f},{hit_rate:.3f}")

# Output (real workload, marketplace search):
# cache_size_gb,hit_rate
# 1.0,0.51
# 2.0,0.63
# 4.0,0.72       <- knee of the curve
# 8.0,0.79
# 16.0,0.84
# 32.0,0.87      <- diminishing returns, +3pp for 2x capacity
# 64.0,0.89
#
# Decision: provision 8-16 GB. Beyond that, marginal hit rate < marginal cost.

Fit α to confirm:

# Zipf fit: log(frequency) = -α * log(rank) + c
import numpy as np
ranks = np.arange(1, len(ranked) + 1)
freqs = np.array([c for _, c in ranked])
log_ranks = np.log(ranks)
log_freqs = np.log(freqs)
alpha, _  = np.polyfit(log_ranks, log_freqs, 1)
alpha = -alpha
# Marketplace search queries: α ~ 1.0-1.2 (caching very effective)
# Recommender outputs per-user: α ~ 0.4-0.6 (caching less effective without cohorts)
# Catalog item-by-ID:           α ~ 0.7-1.1 (depends on traffic source mix)

Recompute quarterly: distribution shifts with seasonality, new features, and ad-driven traffic. A cache sized for last summer's distribution under-provisions during a holiday spike. Set a quarterly cron to re-run this analysis and flag if recommended size differs by >25% from current.

Reference: Breslau et al. — Web Caching and Zipf-like Distributions (INFOCOM 1999) · Beckmann et al. — LHD: Improving Cache Hit Rate by Maximizing Hit Density (NSDI 2018)

Cache Only When Origin p99 Exceeds the Latency Budget

A cache lookup is not free: ElastiCache RTT is typically 0.5-1.5ms in the same AZ, plus 0.1-0.5ms for JSON deserialization on a 10KB payload. If your origin call takes 3ms (e.g. an OpenSearch term query against a hot shard with filter cache hit), adding Redis in front turns a 3ms call into a 4-5ms call on a hit, and 5-6ms on a miss. The cache only pays off when origin p99 exceeds (cache_rtt + deserialize) by a comfortable margin and the hit rate is high enough that the saved p99 dominates the added p50.

Incorrect (cache a fast origin, increasing average latency):

// OpenSearch term query on a small index — already 4ms p99
async function getListingById(id: string): Promise<Listing> {
  const cached = await redis.get(`listing:${id}`);  // 1.2ms RTT
  if (cached) return JSON.parse(cached);            // 0.4ms deserialize
  // miss path:
  const result = await opensearch.get({ index: 'listings', id });  // 4ms
  await redis.set(`listing:${id}`, JSON.stringify(result), 'EX', 60);
  return result;
}
// Measured: p50 went from 2ms (direct) to 1.8ms (cache hit) — break-even.
//          p99 went from 4ms to 5.5ms because misses pay BOTH costs.
// The cache made things slower on average.

Correct (compute the budget, then decide):

// Latency-budget worksheet:
//
//   origin_p99            = 280ms   (Personalize GetRecommendations under load)
//   origin_p50            = 80ms
//   cache_lookup_rtt      = 1.2ms   (ElastiCache same-AZ)
//   deserialize_overhead  = 0.6ms   (10KB JSON)
//   serialize_overhead    = 0.5ms   (only on miss)
//   expected_hit_rate     = 0.70
//
//   effective_p50 = hit_rate * (cache_rtt + deserialize) + (1-hit_rate) * (cache_rtt + origin_p50 + serialize)
//                 = 0.7 * 1.8 + 0.3 * 82.5 = 26ms
//                 vs origin-only p50 = 80ms                  -> saves 54ms
//
//   effective_p99 ≈ max(origin_p99 + cache_rtt + serialize) on the miss tail
//                ≈ 282ms vs origin-only 280ms                -> p99 ~flat
//
// Decision: cache is worth it for the p50 win on a slow origin. Skip for fast origins.

async function getRecommendations(userId: string): Promise<Item[]> {
  // ...standard cache-aside (see strat-cache-aside-default)
}

Heuristic for "fast origin": if origin p99 < 20ms and you're considering a remote cache (ElastiCache, Memcached), don't bother — use an in-process L1 (tier-l1-in-process) instead. In-process LRU is 50-200ns per lookup, well below origin latency, so the cache is "free" on hits.

Personalize specifics: GetRecommendations p99 is regularly 100-500ms under normal load and spikes to 1-2s during auto-scale events. Cache is always worth it for the latency win, separate from cost.

OpenSearch specifics: complex queries with function_score / kNN / rescore (see the sibling skill opensearch-function-scoring-algorithms) are 50-500ms — cache is worth it. Simple term queries with filter-context caching are 2-10ms — application cache often adds latency.

Reference: ElastiCache latency benchmarks · Amazon Personalize getting recommendations

Model Personalize TPS Budget Before Choosing a Cache Strategy

AWS Personalize bills the higher of (a) minProvisionedTPS you set per campaign, or (b) actual TPS during the hour. Each transaction is billed tiered (currently $0.0556 per 1k for the first 72M/month). A campaign with minProvisionedTPS=10 costs 10 × 86400 × $0.0000556 ≈ $48/day floor even at zero traffic. Personalize auto-scales up but never below the floor. Caching reduces actual TPS but does not reduce minProvisionedTPS — so if your minimum is set too high relative to traffic, caching saves nothing on the campaign bill. Conversely, if minProvisionedTPS is set too low, traffic spikes throttle (HTTP 429) and the cache must absorb the failure. The decision of cache vs minProvisionedTPS-sizing is one decision, not two.

Incorrect (over-provision and cache, paying twice):

# Personalize campaign config — set when nobody knew the traffic pattern
campaign_config = {
    "name": "homepage-recommender",
    "solutionVersionArn": "arn:aws:personalize:...:solutionVersion/abc",
    "minProvisionedTPS": 50,  # picked by guessing
}
# Bill: 50 TPS * $0.0000556/req * 86400 s/day = ~$240/day minimum, even at 5 TPS actual.
# Then a Redis cache is added that absorbs 70% of traffic, dropping actual TPS to 1.5.
# Bill is unchanged because minProvisionedTPS still bills 50.

Correct (right-size minProvisionedTPS to the post-cache traffic):

# Step 1: model expected hit rate before deploying (from shadow cache or simulation)
expected_hit_rate = 0.70
peak_request_rate = 50  # requests/sec at peak hour
post_cache_origin_tps = peak_request_rate * (1 - expected_hit_rate)  # = 15

# Step 2: minProvisionedTPS should sit slightly above post-cache peak,
#         not above raw traffic
SAFETY_MARGIN = 1.3
min_tps = math.ceil(post_cache_origin_tps * SAFETY_MARGIN)  # = 20

# Step 3: configure with the right floor
campaign_config = {
    "name": "homepage-recommender",
    "solutionVersionArn": "arn:aws:personalize:...:solutionVersion/abc",
    "minProvisionedTPS": min_tps,  # 20, not 50
}
# Bill: 20 TPS * $0.0000556/req * 86400 = ~$96/day floor.
# Cache infra: $8/day on ElastiCache.
# Net savings vs over-provisioned: ~$144/day per campaign.

The amplification trap: a page with 5 recommenders calls Personalize 5 times per page view. If each campaign has minProvisionedTPS=10 because "10 felt safe," you're paying ~5×$48 = $240/day floor across 5 campaigns even at zero traffic. Reduce the number of campaigns (merge recommenders to a single multi-output model where possible) or share cache across them.

Throttling guidance: Personalize auto-scales up but there is a short delay during which transactions can be lost. If your cache hit rate drops during a deploy or cache flush, you can briefly exceed minProvisionedTPS and see 429s — design the cache to fail open (neg-cache-throttled-personalize).

Reference: Amazon Personalize CreateCampaign · Personalize endpoints and quotas · Personalize pricing

Cache Candidate Sets for Search, Full Payloads for Personalize

Search and Personalize have asymmetric cache shapes. Search results are often re-rankable: the candidate set (top-200 listings from BM25 + kNN retrieval) is stable for a given query, but the final order depends on user signals, A/B treatment, and rerank model version. Cache the candidate set (just IDs + retrieval scores), re-rank on every request. Personalize, in contrast, returns an opaque ranked list — the model has already incorporated the user; you cannot re-rank without calling it again. Cache the full payload, accept that the cache invalidates on every model retrain. Treating these the same wastes 2-3× cache storage on one side or destroys reusability on the other.

Incorrect (cache final-rendered search results, cache only IDs from Personalize):

// Search: cache the fully-rendered, fully-personalized result — bad
async function search(q: string, userId: string) {
  const key = `search:${q}:${userId}`;     // <- user in the key, low hit rate
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);   // ^ full hydrated response, 80KB
  const result = await opensearch.search(buildPersonalisedQuery(q, userId));
  await redis.set(key, JSON.stringify(result), 'EX', 300);
  return result;
}

// Personalize: cache only the ID list — bad
async function getRecs(userId: string) {
  const key = `recs:${userId}`;
  const ids = await redis.get(key);
  if (ids) {
    // Hydrate from OpenSearch on every hit — adds N extra calls per request
    return opensearch.mget({ index: 'listings', body: { ids: JSON.parse(ids) } });
  }
  const fresh = await personalize.getRecommendations({ userId });
  await redis.set(key, JSON.stringify(fresh.itemList.map(i => i.itemId)), 'EX', 300);
  return opensearch.mget({ index: 'listings', body: { ids: fresh.itemList.map(i => i.itemId) } });
}

Correct (cache the right grain for each backend):

// Search: cache the CANDIDATE SET only, re-rank per request
async function search(q: string, userId: string) {
  const candidateKey = `search-candidates:${canonicalize(q)}`;  // <- user OUT of key
  let candidates = await redis.get(candidateKey);
  if (!candidates) {
    candidates = await opensearch.search(buildRetrievalOnlyQuery(q));  // top-200 IDs + scores
    await redis.set(candidateKey, JSON.stringify(candidates), 'EX', 300);
  } else {
    candidates = JSON.parse(candidates);
  }

  // Re-rank with current user signals, A/B treatment, rerank model version
  return rerank(candidates, await getUserFeatures(userId), getABTreatment(userId));
}
// Effect: shared candidate cache hits ~80% across users for the same query;
//         personalisation still applied per-request without an OpenSearch call.

// Personalize: cache FULL PAYLOAD with full hydration
async function getRecs(userId: string, surface: string) {
  const cohortKey = await getCohortKey(userId);
  const key = `recs:${surface}:${cohortKey}:${SOLUTION_VERSION}`;
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);  // fully hydrated, ready to render
  const fresh = await personalize.getRecommendations({ userId });
  const hydrated = await hydrateItems(fresh.itemList);  // one batched mget per cohort
  await redis.set(key, JSON.stringify(hydrated), 'EX', 1800);
  return hydrated;
}
// Effect: one Personalize call + one mget per cohort per 30min, not per user per request.

Why the asymmetry exists:

Reference: OpenSearch search-pipelines for rerank · Personalize Getting recommendations

Bucket Continuous Filters Before Hashing

Continuous filters — price min=237, distance max=4.3km, date range from=2026-05-15T14:23:11Z — produce essentially infinite key cardinality. Two users searching for the same thing with slider-driven price filters typed min=200 and min=210 get two cache misses, even though the result sets overlap heavily. The fix is to bucket the continuous values before hashing — round prices to €50 increments, distances to 1km increments, dates to day boundaries — so logically-similar requests collide on the same key. The result is reused; the user sees the cached results filtered by the original (un-bucketed) value on the client side or by a quick post-filter on the server.

Incorrect (full-precision filters in the key):

type Filters = {
  priceMin?: number;
  priceMax?: number;
  distanceKm?: number;
  checkIn?: string;   // ISO timestamp
  checkOut?: string;
};

function keyFromFilters(f: Filters): string {
  return sha256(JSON.stringify(f));
}

// User A: priceMin=200, priceMax=600, distanceKm=4.3
// User B: priceMin=210, priceMax=620, distanceKm=4.5
// Different keys, both fetch the same OpenSearch response (modulo a few entries).
// Hit rate: under 10% on filtered queries.

Correct (bucket continuous filters before hashing; filter the full result post-cache):

function bucketise(f: Filters): Filters {
  const PRICE_BUCKET_EUR = 50;
  const DISTANCE_BUCKET_KM = 1;
  return {
    // floor() the min to the bucket below, ceil() the max to the bucket above
    // -> bucketed range contains the user's exact range
    priceMin:  f.priceMin  !== undefined ? Math.floor(f.priceMin  / PRICE_BUCKET_EUR)    * PRICE_BUCKET_EUR    : undefined,
    priceMax:  f.priceMax  !== undefined ? Math.ceil (f.priceMax  / PRICE_BUCKET_EUR)    * PRICE_BUCKET_EUR    : undefined,
    distanceKm: f.distanceKm !== undefined ? Math.ceil(f.distanceKm / DISTANCE_BUCKET_KM) * DISTANCE_BUCKET_KM : undefined,
    // Date: round to day boundary in the user's timezone
    checkIn:   f.checkIn  ? roundToLocalDay(f.checkIn,  'floor') : undefined,
    checkOut:  f.checkOut ? roundToLocalDay(f.checkOut, 'ceil')  : undefined,
  };
}

async function search(q: string, filters: Filters, ctx: Ctx) {
  const bucketed = bucketise(filters);
  const key = sha256(JSON.stringify({ q, ...bucketed, locale: ctx.locale }));
  let candidates = await redis.get(key);
  if (!candidates) {
    // Fetch the bucketed (slightly wider) range
    candidates = await opensearch.search(buildQuery(q, bucketed));
    await redis.set(key, JSON.stringify(candidates), 'EX', 300);
  } else {
    candidates = JSON.parse(candidates);
  }

  // Re-filter post-cache to the user's exact range — cheap in-process work
  return postFilter(candidates, filters);
}
// User A and B now collide on the same key. Hit rate on filtered queries
// rises from 10% to 60-80% depending on bucket size choice.

Choosing bucket size:

• Too narrow: low hit rate, defeats the purpose
• Too wide: cache returns too many entries for client-side filtering, latency/payload grows
• Rule of thumb: bucket should be ~5-10% of the typical filter range
• For prices in marketplaces: €50 for accommodation, €5 for food delivery, €10 for fashion
• For distance: 1km in urban, 5km in regional, 25km in cross-country

Validate: measure cache_payload_size_after_postfilter / size_at_origin. If < 50%, your buckets are too narrow (no reuse) or post-filtering is too aggressive (waste).

The "exact match" exception: equality filters like category_id=42 are already bucketed (each integer is its own bucket). Do not artificially widen them — the user wants exactly that category.

Reference: OpenSearch range queries · Use The Index, Luke — Filtering on bounded ranges

Canonicalise Queries Before Hashing

The same logical query can arrive in dozens of textual forms. "new york", "New York", " new york ", "new york", and "new+york" all describe one search; with naive JSON.stringify(filters) hashing they produce five different cache keys and four guaranteed misses. The same goes for filter arrays — {categories: ["bar","restaurant"]} and {categories: ["restaurant","bar"]} are logically equal but textually distinct. Canonical form is the contract between writer and reader of the cache; without it, hit rates plateau at a fraction of the achievable ceiling, no matter how big the cache.

Incorrect (hash the raw input, accept whatever shape it has):

function buildCacheKey(q: string, filters: Record<string, unknown>) {
  return `search:${md5(q + JSON.stringify(filters))}`;
}
// "new york"               -> key A
// "New York"               -> key B  (different cache line)
// "new  york"              -> key C
// filters {a:1,b:2}        -> key D
// filters {b:2,a:1}        -> key E  (different cache line for same filter)
// Hit rate flattens at ~15% even with a large cache.

Correct (canonical pipeline applied before hashing):

import { createHash } from 'node:crypto';

interface SearchInput {
  q: string;
  filters: Record<string, string | number | boolean | string[]>;
  locale: string;
  sort?: string;
  page?: number;
}

function canonicalise(input: SearchInput): SearchInput {
  return {
    // 1. Trim, collapse whitespace, lowercase, NFC-normalise the query string
    q: input.q.trim().replace(/\s+/g, ' ').toLowerCase().normalize('NFC'),

    // 2. Sort array values; sort object keys via JSON.stringify-with-sort
    filters: Object.fromEntries(
      Object.entries(input.filters)
        .sort(([a], [b]) => a.localeCompare(b))
        .map(([k, v]) => [k, Array.isArray(v) ? [...v].sort() : v])
    ),

    // 3. Locale is part of the result; never assume a default
    locale: input.locale.toLowerCase(),  // "en-GB" -> "en-gb"

    // 4. Default sort and page must be present even when unspecified
    sort: input.sort ?? 'relevance',
    page: input.page ?? 1,
  };
}

function buildCacheKey(input: SearchInput): string {
  const canon = canonicalise(input);
  return `search:${createHash('sha256').update(JSON.stringify(canon)).digest('hex').slice(0, 32)}`;
}

// "new york"          -> canon.q "new york"  -> key A
// "New York"          -> canon.q "new york"  -> key A  (HIT)
// "new  york"         -> canon.q "new york"  -> key A  (HIT)
// filters {a:1,b:2}   -> canon "{a:1,b:2}"   -> key D
// filters {b:2,a:1}   -> canon "{a:1,b:2}"   -> key D  (HIT)

The unicode trap: "café" can be encoded as café (NFD) or café (NFC). Browsers send mixed forms. Without .normalize('NFC') you cache the same word twice. Same for fullwidth/halfwidth characters in Asian locales.

The sort-order trap: Map/Set iteration order in older runtimes isn't insertion-ordered for non-string keys. Always sort explicitly; never trust object iteration order to produce a stable serialisation.

Validate empirically: in observability, track distinct_keys_per_distinct_canonical_query. If > 1.05, your canonicalisation has a leak.

Reference: Unicode Normalization Forms (UAX #15) · Pinterest: Feature Caching for Recommender Systems

Make Locale, Currency, and Timezone Explicit in the Key

Search results, recommendations, and ranked listings depend on the user's locale (language → analysers, synonyms, copy), currency (prices, price filters, "from €99" copy), and timezone (date filters, "open now" status, time-decay scoring). When these aren't in the cache key, the first request "wins" — a user in en-GB populates the cache, and pt-PT requests get English copy, GBP prices, and London times. The bug is silent: results render, no error fires, only the content is wrong. Always-explicit locale/currency/timezone in the key prevents this category of bug entirely.

Incorrect (locale/currency/timezone implicit, derived from request headers but not keyed):

async function search(q: string, ctx: RequestContext) {
  const key = `search:${md5(q + JSON.stringify(ctx.filters))}`;
  const cached = await redis.get(key);
  if (cached) {
    // ctx.locale was 'en-GB' when this was written — we no-op the locale here.
    return JSON.parse(cached);
  }
  const result = await opensearch.search(buildQuery(q, ctx));  // uses ctx.locale internally
  await redis.set(key, JSON.stringify(result), 'EX', 600);
  return result;
}
// Request 1: locale=en-GB, currency=GBP, tz=Europe/London  -> populates the cache
// Request 2: locale=pt-PT, currency=EUR, tz=Europe/Lisbon  -> CACHE HIT, gets en-GB result.
// User in Lisbon sees English copy and GBP prices.

Correct (locale/currency/timezone in the key, explicit):

async function search(q: string, ctx: RequestContext) {
  const canon = canonicalise({
    q,
    filters: ctx.filters,
    locale: ctx.locale,         // 'en-gb' (lowercased in canonicalise)
    currency: ctx.currency,     // 'GBP' — uppercased; ISO 4217
    timezone: ctx.timezone,     // 'Europe/London' — IANA tz name, NOT 'BST'
  });
  const key = `search:${sha256(canon)}`;
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  const result = await opensearch.search(buildQuery(q, ctx));
  await redis.set(key, JSON.stringify(result), 'EX', 600);
  return result;
}
// Different locale/currency/tz produces different keys -> clean separation.

The timezone subtleties: use IANA names (Europe/London), never the local abbreviation (BST vs GMT flips by season — same user, two cache keys 6 months apart). Many libraries (Moment, Day.js without timezone plugins) silently use the server's timezone — always pass an explicit tz.

The currency-formatting trap: even if prices are stored as integers in cents/pence, the rendered price strings differ per currency. If you cache the rendered string, currency MUST be in the key. If you cache only the raw integer, currency can be applied post-cache.

Multi-tenant SaaS variant: include tenantId for the same reason. Same query, different tenant configs (synonyms, boosts, taxonomy) — all keyed separately.

Validation: track (distinct_keys / distinct_locales / distinct_currencies) ≈ 1.0 in observability. If users with different locales/currencies produce the same key, the bug is here.

Reference: Unicode CLDR — locale identifiers · IANA Time Zone Database

Key Recommenders by Cohort When Users Outnumber Cohorts

Per-user caching of recommendations is a contradiction: if the cache key includes userId and you have 5M monthly active users on a homepage with 5 recommenders, the upper bound on cache entries is 25M. The working set for "users active today" alone exceeds memory, and hit rate across cache restarts is near zero. The fix is to recognise that personalisation operates at a coarser grain than the individual user — most users behave like one of K cohorts (K = 100-10,000 typically). Key by cohort × locale × surface × solution-version and the working set shrinks by 2-4 orders of magnitude.

Incorrect (user_id in the cache key):

async function getHomepageRecs(userId: string, locale: string) {
  const key = `homepage-recs:${userId}:${locale}`;          // <-- 5M users -> 5M keys
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);
  const fresh = await personalize.getRecommendations({ userId });
  await redis.set(key, JSON.stringify(fresh), 'EX', 1800);
  return fresh;
}
// Cohorts: implicit (Personalize internally has user embeddings, but you can't see them).
// Hit rate: ~2% — practically no reuse, every user is their own cache miss.
// Cost: Personalize bill scales linearly with active users.

Correct (cohort_id in the cache key, user_id in Personalize call):

// Pre-compute a cohort assignment per user (nightly job from user features)
//   features: country, age_bucket, last_active_bucket, signup_age_months,
//             preferred_category, device_class, ...
// Output:    cohort_id  (numeric, e.g. 1..2000)
// Storage:   small key in Redis: `cohort:${userId}` -> cohortId  (TTL 1 day)

async function getCohortKey(userId: string): Promise<string> {
  const cohortId = await redis.get(`cohort:${userId}`);
  if (cohortId) return cohortId;
  // Compute on the fly for new users (or queue async assignment)
  return await assignCohort(userId);
}

async function getHomepageRecs(userId: string, locale: string) {
  const cohort = await getCohortKey(userId);
  const key = `homepage-recs:c${cohort}:${locale}:${SOLUTION_VERSION}`;
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  // Still call Personalize with the real userId so the model sees the individual,
  // but cache the result against the cohort key for reuse.
  const fresh = await personalize.getRecommendations({ userId });
  await redis.set(key, JSON.stringify(fresh), 'EX', 1800);
  return fresh;
}
// 5M users -> ~2000 cohorts × ~30 locales × 1 solution version = 60k keys
// Hit rate: 85-95% within an hour, because users in the same cohort share recs.

When `userId`-keyed caching IS correct:

• Mutation-write paths (e.g. user-edited preferences) — write-through with user key
• Per-user session vectors that change second-to-second — see pers-session-vector-write-through
• Logged-in marketplace-buyer histories — but TTL stays short (<5 min)

Hybrid pattern for heavy personalisation: cache the cohort-level candidate set (top-200 IDs) for the cohort, then re-rank the top-50 per-user with a lightweight on-the-fly score. This gets cohort-level reuse (~90% hit rate) plus per-user personalisation in the final order.

Choosing K: plot hit rate vs cohort count. K=100 cohorts: ~75% hit rate, very low diversity (homepage looks the same for too many users). K=2000-10000: ~90% hit rate, individual-level diversity preserved. K=100000: per-user, hit rate ~5%. The knee is usually in the 1k-10k range.

Reference: Pinterest: PinnerSage Multi-Modal User Embeddings · Personalize User-Personalization recipe

Use SHA-256 over MD5 for High-Cardinality Keys

MD5's 128-bit output has ~1.8×10¹⁹ values, but the birthday bound makes a collision likely once you've inserted ~2³² (~4.3 billion) keys. A high-traffic marketplace with many cohorts × surfaces × filters can pass that threshold in a few months. Worse, MD5 is not collision-resistant against adversarial input — a request crafted to collide with another's key can return another user's recommendations or A/B treatment. SHA-256 has 256 bits, no known collisions, and the per-call cost on Node.js is ~1.2µs for sub-1KB inputs (negligible against any Redis RTT). Prefer SHA-256 by default; consider BLAKE3 if you need extreme throughput.

Incorrect (MD5 on a high-cardinality keyspace):

import { createHash } from 'node:crypto';

function key(input: object): string {
  // 128 bits. Truncated to 16 hex chars for compactness — even worse.
  return `cache:${createHash('md5').update(JSON.stringify(input)).digest('hex').slice(0, 16)}`;
}
// 16 hex chars = 64 bits. Birthday collision likely at 2^32 = 4B entries.
// A high-traffic marketplace hits that in months — silent cross-user data leaks.
// MD5 itself is also broken against adversarial inputs.

Correct (SHA-256, full digest or first 32 chars):

import { createHash } from 'node:crypto';

function key(input: object): string {
  // SHA-256 of canonicalised JSON. 256 bits full; first 32 hex chars = 128 bits
  // — well above any practical collision risk.
  return `cache:${createHash('sha256').update(JSON.stringify(input)).digest('hex').slice(0, 32)}`;
}

// Benchmark on a 2026-era cloud VM:
//   sha256 of 1KB input:  ~1.2 µs
//   md5 of 1KB input:     ~0.7 µs
//   redis GET round trip: ~1500 µs
// The hash cost is irrelevant.

The "we'll never have that many keys" trap:

// Common misjudgement:
//   "We have 100k SKUs, 5 sort orders, 10 filters — only a few million combos."
// Reality after a year:
//   100k × 5 × 2^10 (filter combos) × 30 locales × 5 device classes
//   = 7.7 trillion possible keys. Working set is smaller, but cardinality of
//   ever-seen keys hits 10B+ easily on viral campaigns.

For non-cryptographic uses where you need speed at extreme scale, BLAKE3 is faster than SHA-256 and equally collision-resistant. But the default should be SHA-256 — it's available in the standard library of every runtime and is fast enough.

Never use:

• String.prototype.hashCode()-style algorithms — 32-bit, collisions guaranteed
• Truncated MD5 below 64 bits — adversarial collisions trivially constructable
• JSON.stringify().length or similar fake hashes — collisions across same-length strings

The fingerprint pattern: if you need to store the full key alongside the hash (for debugging or to disambiguate collisions), use key_hash (the 32-char SHA) plus key_fingerprint (a small array of derived attributes like locale, surface, q-prefix). The hash is the lookup; the fingerprint is the audit trail.

Reference: RFC 6234 — SHA-256 · BLAKE3 spec · Birthday problem analysis for hash collisions

Strip Volatile and Tracking Params Before Hashing

Query params with no effect on the result — UTM tags, request_id, client timestamp, browser fingerprint, A/B-bucket-disambiguation tokens — must never reach the hash. When they leak, every request gets a unique key by construction and the cache hit rate collapses to near-zero. The cache fills up with thousands of variants of the same logical request, evicting useful entries. The bug presents as "hit rate is 4% and the cache is full" — and the fix is upstream of the hash function: a strict allowlist of params that participate in the key.

Incorrect (hash the whole URL or full request object):

function buildCacheKey(req: Request): string {
  // Includes everything: ?utm_source=newsletter&request_id=abc-123&ts=1717012345&q=pizza
  return `search:${sha256(req.url + JSON.stringify(req.body))}`;
}

// Every email-driven request has a different utm_content
// Every request from the frontend has a different request_id
// Every request gets a different ts
// Net effect: each cache entry is read exactly once before eviction.
// Hit rate: 3%.

Correct (allowlist the keying params, ignore everything else):

// What participates in the key:
const KEYABLE_PARAMS = new Set([
  'q',           // the actual search query
  'filters',     // structured filter object (already canonicalised separately)
  'sort',
  'page',
  'pageSize',
  'locale',
  'currency',
  'timezone',
  'tenantId',
]);

// Everything else is explicitly excluded:
const VOLATILE_PARAMS = new Set([
  'utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'utm_id',
  'request_id', 'trace_id', 'span_id',
  'ts', 'timestamp', '_t',
  'cb', 'cache_bust', '_',
  'gclid', 'fbclid', 'mc_cid', 'mc_eid',
  'referrer', 'origin_url',
]);

function buildCacheKey(req: SearchRequest): string {
  // 1. Project to keyable fields only
  const keyed: Record<string, unknown> = {};
  for (const k of KEYABLE_PARAMS) {
    if (req[k] !== undefined) keyed[k] = req[k];
  }

  // 2. Assert: no volatile param leaked into the keyable set
  for (const k of Object.keys(keyed)) {
    if (VOLATILE_PARAMS.has(k)) {
      throw new Error(`Volatile param ${k} in cache key — likely a bug`);
    }
  }

  // 3. Canonicalise + hash (see key-canonicalize-query)
  const canon = canonicalise(keyed);
  return `search:${sha256(JSON.stringify(canon))}`;
}

Allowlist, not denylist. New tracking params get added by marketing, product analytics, and ad networks faster than you can denylist them. Allowlist what should affect the result; the rest is stripped by construction.

The header trap: beware of using User-Agent or Cookie as part of the key. A cookie change (a session refresh, a consent banner click) creates a new key for the same user. Pick the specific cookie fields you need (tenant_id, cohort_id) and ignore the rest of the header.

Frontend cooperation: for client-driven caches (Service Worker, SWR), publish the canonicalisation rules to the frontend so requests are stripped before they leave the browser. Otherwise you pay for the round trip even on hits.

Validation: in observability, plot cache_keys_per_canonical_request. Expected: 1.0. If > 1.1, a volatile param has leaked.

Reference: Cloudflare cache key documentation · HTTP RFC 9111 — Caching

Include the Personalize Solution Version in the Cache Key

A Personalize recommender is bound to a solution version (the trained model). When the model retrains — nightly, weekly, or on demand — the campaign points to a new solutionVersionArn, and the old recommendations are no longer the output of the active model. If your cache key doesn't include the solution version, you keep serving last-week's recommendations until the TTL expires. The retrain happened, the dashboards say "active model: v42", and users see results from v41 for the cached duration. Including the solution version in the key makes the cache self-invalidate on retrain — old entries become orphan, new ones start fresh.

Incorrect (cache key independent of model version):

const KEY = (cohort: string, locale: string) => `homepage:${cohort}:${locale}`;

async function getRecs(cohort: string, locale: string, userId: string) {
  const cached = await redis.get(KEY(cohort, locale));
  if (cached) return JSON.parse(cached);
  const fresh = await personalize.getRecommendations({
    campaignArn: HOMEPAGE_CAMPAIGN,
    userId,
  });
  await redis.set(KEY(cohort, locale), JSON.stringify(fresh), 'EX', 3600);
  return fresh;
}

// 02:00 UTC: nightly retrain finishes, campaign now points to solutionVersion v43.
// 02:00-03:00 UTC: 50% of traffic continues to hit cache entries from v42.
// Some users get v42 recs, some v43. Mixed treatment, untrackable in A/B logs.

Correct (solution version baked into the key):

// Subscribe to Personalize campaign updates (EventBridge or a polling refresher).
// Maintain an in-memory copy of the active solution version per campaign.
let ACTIVE_SOLUTION_VERSION = await fetchActiveSolutionVersion(HOMEPAGE_CAMPAIGN);

setInterval(async () => {
  ACTIVE_SOLUTION_VERSION = await fetchActiveSolutionVersion(HOMEPAGE_CAMPAIGN);
}, 30_000);  // 30s drift acceptable

// Short hash of the ARN — keeps keys compact while uniquely identifying the version
const versionTag = (arn: string) => createHash('sha1').update(arn).digest('hex').slice(0, 8);

const KEY = (cohort: string, locale: string) =>
  `homepage:${cohort}:${locale}:v${versionTag(ACTIVE_SOLUTION_VERSION)}`;

async function getRecs(cohort: string, locale: string, userId: string) {
  const cached = await redis.get(KEY(cohort, locale));
  if (cached) return JSON.parse(cached);
  const fresh = await personalize.getRecommendations({
    campaignArn: HOMEPAGE_CAMPAIGN,
    userId,
  });
  await redis.set(KEY(cohort, locale), JSON.stringify(fresh), 'EX', 3600);
  return fresh;
}

// 02:00 UTC: retrain finishes, active version becomes v43.
// New keys: homepage:c1:en-gb:v8a3c1e9f  (the new versionTag)
// Old keys: homepage:c1:en-gb:v2f9d3e07  — never read again, evict on TTL.
// Every request after 02:00 hits the new model's outputs cleanly.

Apply the same pattern to OpenSearch retrieval pipelines: if your search pipeline includes a learning-to-rank model, your reranker model version, or a query-expansion table version, include them in the key. The principle: any artifact whose change should invalidate the cache belongs in the key, not in the cache value.

A/B testing variant: when running an A/B test that swaps solution versions per user bucket, key includes both the bucket and the version: homepage:c1:en-gb:bucket-${bucket}:v${versionTag}. This naturally segregates the two treatments' caches.

Trade-off — at retrain, hit rate temporarily drops to 0. This is correct behaviour; the cost is one period of full origin traffic. If retrains are frequent (hourly) and origin TPS is the bottleneck, mitigate with strat-async-warm-up immediately after retrain.

Reference: Personalize: Deploying a solution version with a campaign · Personalize EventBridge events

Use a Bloom Filter to Block High-Cardinality Miss Storms

A high-cardinality miss storm is a load pattern where requests target keys that don't exist in either cache or origin — slug-not-found URLs from crawlers, listing IDs from stale links, hash-not-found cache keys from attacks. Each request: cache.GET (miss), origin.GET (404), no cache write, repeat. Negative caching (neg-cache-empty-results) helps but requires Redis lookups per miss. A Bloom filter sits in front of both — set-membership check in ~100ns from in-process memory — and returns "definitely not found" without touching Redis or the origin.

Incorrect (miss storms reach cache and origin):

async function getListingPage(slug: string) {
  const cached = await redis.get(`page:${slug}`);
  if (cached) return JSON.parse(cached);
  const fresh = await db.queryBySlug(slug);  // for nonexistent slugs, returns null
  if (fresh) {
    await redis.set(`page:${slug}`, JSON.stringify(fresh), 'EX', 600);
    return fresh;
  }
  return null;
}
// Attack/crawler hitting /listing/{random-slug} at 1000 req/s:
//   1000 cache misses/s -> 1000 DB queries/s -> all return null
//   Both cache and DB take load for nothing.

Correct (Bloom filter rejects definite-misses before any IO):

import { BloomFilter } from 'bloom-filters';

// Built nightly from the canonical list of all valid slugs
//   - 10M slugs, 1% false-positive rate = ~12 MB
//   - Shipped to all instances at startup; updated via SIGHUP / config reload
let VALID_SLUGS: BloomFilter = await loadBloomFromS3('s3://catalog/valid-slugs.bloom');

async function getListingPage(slug: string) {
  // 1. Bloom filter check — in-process, ~100ns
  if (!VALID_SLUGS.has(slug)) {
    metrics.increment('bloom.rejected', { kind: 'slug' });
    return null;  // never reaches Redis or DB
  }

  // 2. Bloom said "possibly exists" — proceed with normal cache-aside
  const cached = await redis.get(`page:${slug}`);
  if (cached) return JSON.parse(cached);

  const fresh = await db.queryBySlug(slug);
  if (fresh) {
    await redis.set(`page:${slug}`, JSON.stringify(fresh), 'EX', 600);
    return fresh;
  }

  // Bloom false positive — slug doesn't exist after all
  // Cache the negative to avoid repeated DB hits on this slug
  await redis.set(`page:${slug}:miss`, '1', 'EX', 300);
  return null;
}

Sizing the Bloom filter:

• N items, FP rate p: size ~= -N * ln(p) / (ln(2)²)
• 10M items at 1% FP rate: ~12 MB; 0.1% FP rate: ~17 MB
• Memory is cheap; default to 0.1% if you can afford it

Keep the Bloom filter fresh:

• Nightly batch: rebuild from the catalog's authoritative list
• For new items added during the day: maintain a "delta" in Redis (SADD valid-slugs-since-last-bloom-rebuild :slug) and check both
• Or accept that new items have a brief window where Bloom returns "definitely not" — usually fine for SEO content with a 24h indexation lag

The Cloudflare gotcha: Cloudflare's "When Bloom Filters Don't Bloom" post documents that naive Bloom filter implementations have terrible cache locality and can be slower than a hash table. Use a well-implemented library (bloom-filters for Node, pybloom-live for Python, Guava BloomFilter for Java) with cache-aware design.

When NOT to use a Bloom filter:

• The set of valid keys is small and fits in a hash map (just use the hash map)
• Valid keys are highly volatile (real-time additions/removals — Bloom doesn't support deletion)
• Miss storms don't exist in your traffic pattern (web search has them; B2B SaaS often doesn't)

Companion: rate limit by IP for the false-positive tail. Even after Bloom rejection, a determined adversary can construct queries that pass the filter (cache penetration attack). Rate-limit per-IP at the edge as the second line.

Reference: Cloudflare: When Bloom Filters Don't Bloom · Wikipedia: Bloom filter

Cache Empty Search Results With a Short TTL

An empty search result still consumes OpenSearch CPU. The cluster runs the same query, traverses indexes, applies filters, computes scores — only to return zero hits. For a misspelled long-tail query that's not going viral, this is fine. For a common query that returns zero hits (a category page with no inventory, a stale ad linking to a removed listing, a slug-not-found page), repeated empty-result queries can dominate cluster CPU. Cache the "zero results" response like any other — with a deliberately shorter TTL than the positive cache, because empty results often resolve when new inventory arrives.

Incorrect (empty results bypass the cache or get long TTLs):

async function search(q: string, ctx: Ctx) {
  const key = buildKey(q, ctx);
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  const result = await opensearch.search(buildQuery(q, ctx));

  if (result.hits.total === 0) {
    // Some teams: don't cache empty results
    return result;  // every empty-result request hits OpenSearch
  }

  await redis.set(key, JSON.stringify(result), 'EX', 600);
  return result;
}
// Symptom: OpenSearch CPU dominated by zero-result queries from broken links,
//          misconfigured categories, or removed listings.

Correct (cache empties with a short TTL):

async function search(q: string, ctx: Ctx) {
  const key = buildKey(q, ctx);
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  const result = await opensearch.search(buildQuery(q, ctx));

  // Cache positive AND negative — different TTLs reflect their volatility
  const ttl = result.hits.total === 0 ? 60 : 600;
  await redis.set(key, JSON.stringify(result), 'EX', ttl);
  return result;
}
// Empty results cached for 60s. The 100th query for the broken-link slug
// is served from cache. OpenSearch CPU drops accordingly.

Why a shorter TTL on empties:

• Inventory frequently arrives (new listing, new product, restocked item). A 60s TTL on empty means new inventory becomes visible quickly.
• An incorrect empty (e.g. an index bug, a query analyzer regression) self-heals in 60s rather than 10 min.
• Empty results compress extremely well ({"hits":{"total":0,"hits":[]}}) so memory cost is trivial.

Companion pattern — explicit empty signaling:

type CachedSearchResult = {
  isEmpty: boolean;     // explicit
  hits: SearchResult;
  cachedAt: number;
};

The explicit flag means application code can branch on "this is a known-empty result" without parsing the full payload — useful for fallback logic (try a broader query, suggest spelling corrections, route to "did you mean" path).

Don't cache errors as empties. If OpenSearch returned a 5xx, don't write a "zero results" cache entry. The next request would falsely conclude "no inventory" rather than "transient origin issue."

try {
  const result = await opensearch.search(buildQuery(q, ctx));
  // ... cache as above
} catch (err) {
  // Don't cache the failure as zero results. Either propagate or fall back.
  throw err;
}

Personalize equivalent: Personalize doesn't typically return "zero recommendations" for a cold user (it falls back to popular items), but it can throttle or fail. See neg-cache-throttled-personalize — same idea, applied to throttling.

Search-suggestion variant: for autocomplete that frequently returns "no suggestions" for unusual prefixes ("xyz"), cache the empty result aggressively (5-min TTL) — the same prefix is re-typed often and won't change.

Reference: Google Cloud CDN negative caching · Design Gurus: Negative Caching

Serve Last-Known-Good When Personalize Throttles

Personalize auto-scales above minProvisionedTPS, but the scaling has a delay during which excess traffic can be throttled (HTTP 429). This is normal — it happens after deploys, traffic spikes, and during the first minutes after a campaign update. If the application propagates 429s as user-facing errors, recommendation rails go blank during these windows. The correct response is to fall back to a "last-known-good" cached value — even one beyond its TTL — rather than surfacing the throttle. Combined with the circuit breaker (stamp-circuit-breaker-on-origin-error), throttles trigger a graceful degradation that's invisible to users.

Incorrect (let 429s reach the user):

async function getRecs(userId: string, surface: string) {
  const key = `recs:${surface}:${userId}`;
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  try {
    return await personalize.getRecommendations({
      campaignArn: CAMPAIGN_FOR[surface],
      userId,
    });
  } catch (err) {
    // 429 propagates as a 500 to the user
    throw err;
  }
}
// Symptom: post-deploy traffic spike, Personalize 429s, frontend shows
//          "recommendations unavailable" banner. Resolves on its own in 30-60s.

Correct (write a long-TTL "last-known-good" copy; fall back on 429):

async function getRecs(userId: string, surface: string) {
  const cohort = await getCohortKey(userId);
  const liveKey = `recs:${surface}:${cohort}:live`;
  const lkgKey  = `recs:${surface}:${cohort}:lkg`;   // last-known-good

  const cached = await redis.get(liveKey);
  if (cached) return JSON.parse(cached);

  try {
    const fresh = await personalize.getRecommendations({
      campaignArn: CAMPAIGN_FOR[surface],
      userId,
    });

    // Write to both: live (short TTL, typical cache) and LKG (long TTL, fallback only)
    await redis.multi()
      .set(liveKey, JSON.stringify(fresh), 'EX', 1800)
      .set(lkgKey,  JSON.stringify(fresh), 'EX', 86400)  // 24h
      .exec();
    return fresh;

  } catch (err) {
    if (isThrottle(err) || isTransientError(err)) {
      const lkg = await redis.get(lkgKey);
      if (lkg) {
        metrics.increment('cache.fallback.lkg_served', { surface, reason: err.code });
        return JSON.parse(lkg);
      }
      // No LKG — fall back to popularity (no personalisation but no blank)
      metrics.increment('cache.fallback.popularity', { surface });
      return getPopularityRecs(surface);
    }
    throw err;  // genuine error, not a throttle
  }
}

function isThrottle(err: unknown): boolean {
  return err?.name === 'ThrottlingException'
      || err?.$metadata?.httpStatusCode === 429
      || err?.code === 'ProvisionedThroughputExceededException';
}

Live vs LKG decoupling:

• live cache: normal TTL (1800s for cohort recs). Fast invalidation on retrain or content change.
• LKG cache: long TTL (86400s = 24h). Only read on origin failure. Long enough that an extended outage doesn't drain it.
• Cost: 2× memory for cached entries. For cohort caching (small key space) this is negligible.

Track the LKG-served ratio. Persistent LKG serves indicate Personalize is consistently throttled — adjust minProvisionedTPS up or reduce traffic-amplification via better cache keying (decide-amplification-multiplier).

For OpenSearch parallel: cluster overload (5xx, timeouts) follows the same pattern. Cache the last successful result as LKG; serve it during cluster instability. OpenSearch outages are rarer but the cost of a search outage on a marketplace is higher.

Don't fall back to LKG for write paths. A user submitting a search query while Personalize is throttled should get the LKG (they're reading recommendations). A user updating their preferences should NOT silently fall back — that would lose data. Mutations propagate errors normally.

Cold-instance bootstrap: on a new instance with empty LKG, the first throttle has nothing to serve. Mitigate by warming the LKG cache during instance startup (strat-async-warm-up).

Reference: Personalize endpoints and quotas · AWS SDK retry behavior for throttling

Add Random Jitter to TTL to Prevent Synchronized Expiry

When many keys are written at the same time — a nightly batch precomputation, a deploy warm-up, a cohort refresh — they all get the same TTL and expire at the same instant. The next read after expiry triggers N simultaneous origin calls, one per key. Even with per-key stampede protection (each key has ONE origin call), the aggregate origin load spikes by 100-10000× for the duration of the refresh window. Adding random jitter (±10-25% of the TTL) spreads expiry across a window so the origin sees a sustained moderate load rather than a spike.

Incorrect (fixed TTL — synchronized expiry from batch writes):

# Nightly batch at 02:00 writes 10000 cohort × surface entries with TTL=86400
for cohort in cohorts:
    for surface in surfaces:
        redis.set(f'recs:{surface}:{cohort}', json.dumps(recs), ex=86400)

# At 02:00 next day: ALL 10000 entries expire simultaneously.
# First requests after 02:00 trigger 10000 origin calls in seconds.
# Even with single-flight, that's 10000 distinct origin calls.
# Personalize TPS spikes, OpenSearch CPU saturates.

Correct (jittered TTL):

import random

def jittered_ttl(base_seconds: int, jitter_pct: float = 0.15) -> int:
    """TTL with +/- jitter_pct% random jitter."""
    jitter = base_seconds * jitter_pct
    return base_seconds + random.randint(-int(jitter), int(jitter))

for cohort in cohorts:
    for surface in surfaces:
        # 86400 ± 15% = expires uniformly over a ~3.6-hour window
        redis.set(f'recs:{surface}:{cohort}', json.dumps(recs), ex=jittered_ttl(86400))

# Now expiries are spread across 02:00 ± 1.8h.
# Origin load is a smooth ramp, not a cliff.

Application-side cache-aside variant:

const TTL_SEC = 600;
const JITTER_PCT = 0.15;

function jitteredTtl(base: number, pct = JITTER_PCT): number {
  const jitter = base * pct;
  return Math.floor(base + (Math.random() * 2 - 1) * jitter);
}

async function cacheSet<T>(key: string, value: T, baseTtlSec: number) {
  await redis.set(key, JSON.stringify(value), 'EX', jitteredTtl(baseTtlSec));
}

Choosing jitter percentage:

• 5-10%: smooths small bursts, minimal staleness impact
• 15-25%: default for batch-written entries; balances spread vs predictability
• 50%+: aggressive smoothing; only if origin load is the dominant constraint

Avoid the negative-jitter pitfall. random.randint(-15, 15) produces a TTL that may be shorter than expected. The batch job above generates a uniform distribution over [base - jitter, base + jitter]. If the product requires "no entry younger than 5 minutes is served stale," set the jitter range so the minimum is at least 5 minutes.

Combine with soft/hard TTL. The jitter applies to both the soft and hard TTL — keep their ratio constant. Soft = 0.8 * jitteredTtl(), hard = 1.0 * jitteredTtl().

Cohort-specific jitter seed (advanced): instead of pure random, use hash(key) % jitter_range to make a key's TTL deterministic but spread across keys. Useful for debugging — the same key always has the same TTL within a run.

Reference: AWS Architecture Center: TTL jitter patterns · Marc Brooker — Exponential Backoff and Jitter