Frappe Core Cache

Frappe Cache & Locking

Quick Reference

---

Decision Tree

What caching pattern do you need?
│
├─ Cache a function result automatically?
│  ├─ Pure function (same args → same result) → @redis_cache
│  └─ Need custom key/TTL → manual get_value/set_value
│
├─ Cache a document?
│  ├─ Read-only access → frappe.get_cached_doc()
│  └─ Need to invalidate → frappe.clear_document_cache()
│
├─ Cache structured data (multiple fields)?
│  └─ Redis hash → hset/hget/hgetall
│
├─ Per-request cache (avoid repeated DB calls in one request)?
│  └─ frappe.local.cache dict
│
├─ Prevent concurrent execution?
│  └─ Distributed lock → frappe.lock("resource_name")
│
└─ Invalidate cache?
   ├─ Single key → delete_value(key)
   ├─ Pattern → delete_keys("prefix*")
   └─ All site cache → frappe.clear_cache()

---

String Operations

Set and Get

# Set a value (persists until evicted or deleted)
frappe.cache.set_value("exchange_rate_USD", 1.08)

# Set with TTL (expires after N seconds)
frappe.cache.set_value("exchange_rate_USD", 1.08, expires_in_sec=3600)

# Get value (returns None if missing)
rate = frappe.cache.get_value("exchange_rate_USD")

# Get with generator (calls function on cache miss, stores result)
rate = frappe.cache.get_value(
    "exchange_rate_USD",
    generator=lambda: fetch_exchange_rate("USD"),
)

User-Scoped Values

# Store per-user preference
frappe.cache.set_value("dashboard_layout", "compact", user="user@example.com")

# Retrieve for specific user
layout = frappe.cache.get_value("dashboard_layout", user="user@example.com")

Delete

# Single key
frappe.cache.delete_value("exchange_rate_USD")

# Multiple keys
frappe.cache.delete_value(["exchange_rate_USD", "exchange_rate_EUR"])

# Pattern-based deletion (wildcard)
frappe.cache.delete_keys("exchange_rate*")

---

Hash Operations

Use hashes to group related fields under a single key.

# Set hash fields
frappe.cache.hset("config|notifications", "email_enabled", True)
frappe.cache.hset("config|notifications", "sms_enabled", False)
frappe.cache.hset("config|notifications", "max_retries", 3)

# Get single field
email_on = frappe.cache.hget("config|notifications", "email_enabled")

# Get all fields as dict
config = frappe.cache.hgetall("config|notifications")
# {"email_enabled": True, "sms_enabled": False, "max_retries": 3}

# Delete field
frappe.cache.hdel("config|notifications", "sms_enabled")

# Check existence
exists = frappe.cache.hexists("config|notifications", "email_enabled")

Hash with Generator

# hget with generator — calls function on miss
value = frappe.cache.hget(
    "user|permissions",
    "user@example.com",
    generator=lambda: compute_permissions("user@example.com"),
)

---

@redis_cache Decorator

Automatically cache function return values based on arguments.

from frappe.utils.caching import redis_cache

@redis_cache
def get_item_price(item_code, price_list):
    """Expensive query — cached automatically."""
    return frappe.db.get_value("Item Price",
        {"item_code": item_code, "price_list": price_list},
        "price_list_rate",
    )

# First call — hits database, stores in Redis
price = get_item_price("ITEM-001", "Standard Selling")

# Second call — returns from cache
price = get_item_price("ITEM-001", "Standard Selling")

# Clear all cached results for this function
get_item_price.clear_cache()

With TTL

@redis_cache(ttl=300)  # expires after 5 minutes
def get_exchange_rate(from_currency, to_currency):
    return fetch_rate_from_api(from_currency, to_currency)

Rules for @redis_cache:

• ALWAYS ensure arguments are hashable (strings, numbers, tuples). NEVER pass dicts or lists as arguments.
• ALWAYS call .clear_cache() when underlying data changes.
• NEVER use on functions with side effects — the function will NOT execute on cache hits.

---

frappe.local.cache: Request-Scoped Cache

frappe.local.cache is a plain Python dict that lives for the duration of a single HTTP request. It is NOT stored in Redis.

def get_user_settings():
    """Avoid repeated DB calls within a single request."""
    if "user_settings" not in frappe.local.cache:
        frappe.local.cache["user_settings"] = frappe.get_doc(
            "User Settings", frappe.session.user
        )
    return frappe.local.cache["user_settings"]

Use frappe.local.cache when:

• The same data is needed multiple times in one request
• The data does NOT need to persist across requests
• You want zero Redis overhead

---

Document Caching

# Get cached document (read-only, no permission check)
settings = frappe.get_cached_doc("System Settings")
item = frappe.get_cached_doc("Item", "ITEM-001")

# Invalidate when document changes
frappe.clear_document_cache("Item", "ITEM-001")

# Cached single value
val = frappe.db.get_value("Item", "ITEM-001", "item_name", cache=True)

NEVER modify a document returned by frappe.get_cached_doc() — it returns a shared reference. Modifications corrupt the cache for all subsequent reads.

---

Distributed Locking

Prevent concurrent execution of critical sections using Redis-based locks.

# Context manager (recommended)
with frappe.lock("process_payroll"):
    # Only one worker executes this block at a time
    process_all_salary_slips()
    # Lock auto-released on exit

# Manual lock/unlock
frappe.lock("inventory_sync")
try:
    sync_inventory()
finally:
    frappe.unlock("inventory_sync")  # ALWAYS unlock in finally

Rules:

• ALWAYS use with frappe.lock() (context manager) to guarantee release.
• NEVER hold locks for more than a few seconds — long locks cause worker starvation.
• ALWAYS use descriptive lock names to avoid collisions.

---

Cache Invalidation Patterns

Pattern 1: TTL-Based (Time-to-Live)

frappe.cache.set_value("dashboard_stats", compute_stats(), expires_in_sec=300)

Best for: Data that can be slightly stale (exchange rates, dashboard aggregates).

Pattern 2: Event-Based Invalidation

# In hooks.py
doc_events = {
    "Item Price": {
        "on_update": "my_app.cache.invalidate_price_cache",
        "on_trash": "my_app.cache.invalidate_price_cache",
    }
}

# In my_app/cache.py
def invalidate_price_cache(doc, method):
    frappe.cache.delete_keys("item_price*")
    # Or clear specific function cache:
    # get_item_price.clear_cache()

Best for: Data that MUST be fresh immediately after changes.

Pattern 3: Hybrid (TTL + Event)

@redis_cache(ttl=600)
def get_pricing_rules():
    return frappe.get_all("Pricing Rule", fields=["*"])

# Event hook clears cache immediately on change
def on_pricing_rule_update(doc, method):
    get_pricing_rules.clear_cache()

Best for: Frequently read data with occasional updates.

---

Common Cache Keys (Internal)

NEVER write to internal cache keys directly. ALWAYS use the documented API methods (get_cached_doc, clear_document_cache, etc.).

---

Performance Guidelines

1. ALWAYS set TTL on cached values that derive from external data — without TTL, stale data persists until manual invalidation or Redis eviction. 2. NEVER cache large objects (>1 MB) — Redis uses pickle serialization, and large values increase serialization overhead and memory usage. 3. ALWAYS use `frappe.local.cache` for data needed multiple times within a single request — it avoids Redis round-trips entirely. 4. NEVER use `frappe.clear_cache()` as a routine invalidation strategy — it clears ALL cache keys for the site, causing a cold-cache performance hit. 5. ALWAYS prefix custom cache keys with your app name (e.g., myapp|exchange_rate) to avoid collisions with Frappe internals.

---

Redis Configuration

Default config: {bench}/config/redis_cache.conf

---

Key Namespacing

All cache keys are automatically prefixed by Frappe with the site name:

# You write:
frappe.cache.set_value("my_key", "value")

# Redis stores:
# "mysite.localhost|my_key"

frappe.cache.make_key(key, user, shared) handles prefixing. The shared=True parameter removes the site prefix for cross-site keys (rare use case).

---

Version Differences

---

See Also

• references/examples.md — Cache implementation patterns
• references/anti-patterns.md — Common cache mistakes
• references/api-reference.md — Complete API signatures
• frappe-core-database — Database queries that benefit from caching
• frappe-core-permissions — User permission caching

Cache Anti-Patterns

AP-1: No TTL on External Data

Wrong:

frappe.cache.set_value("exchange_rate_USD", fetch_rate("USD"))
# Cached forever — rate becomes stale after minutes

Correct:

frappe.cache.set_value("exchange_rate_USD", fetch_rate("USD"), expires_in_sec=3600)

ALWAYS set expires_in_sec on cached values derived from external sources. Without TTL, stale data persists until manual deletion or Redis eviction.

---

AP-2: Modifying Cached Documents

Wrong:

settings = frappe.get_cached_doc("System Settings")
settings.enable_telemetry = 0  # CORRUPTS the cache for all readers
settings.save()

Correct:

# Get a fresh, mutable copy
settings = frappe.get_doc("System Settings")
settings.enable_telemetry = 0
settings.save()

NEVER modify a document returned by frappe.get_cached_doc(). It returns a shared reference — mutations affect all subsequent cache reads.

---

AP-3: Using frappe.clear_cache() as Routine Invalidation

Wrong:

def on_item_update(doc, method):
    frappe.clear_cache()  # Nukes ALL cache for the entire site

Correct:

def on_item_update(doc, method):
    frappe.cache.delete_value(f"item_data|{doc.name}")
    frappe.clear_document_cache("Item", doc.name)

NEVER use frappe.clear_cache() for targeted invalidation. It clears ALL cache keys (user permissions, boot info, document cache), causing a performance cliff as everything rebuilds.

---

AP-4: Cache Stampede

Wrong:

def get_report_data():
    data = frappe.cache.get_value("expensive_report")
    if data is None:
        # 100 concurrent requests all see cache miss
        # All 100 execute the expensive query simultaneously
        data = run_expensive_query()
        frappe.cache.set_value("expensive_report", data)
    return data

Correct:

def get_report_data():
    # Use generator — only one caller computes, others wait
    return frappe.cache.get_value(
        "expensive_report",
        generator=run_expensive_query,
        expires_in_sec=300,
    )

# Or use locking for critical sections
def get_report_data():
    data = frappe.cache.get_value("expensive_report")
    if data is None:
        with frappe.lock("compute_report"):
            # Double-check after acquiring lock
            data = frappe.cache.get_value("expensive_report")
            if data is None:
                data = run_expensive_query()
                frappe.cache.set_value("expensive_report", data, expires_in_sec=300)
    return data

---

AP-5: Unbounded Cache Keys

Wrong:

def cache_user_data(user, page, filters):
    key = f"user_data|{user}|{page}|{filters}"
    # Creates a unique key for every filter combination
    # Keys accumulate indefinitely — memory bloat
    frappe.cache.set_value(key, compute_data(user, page, filters))

Correct:

def cache_user_data(user, page, filters):
    key = f"user_data|{user}|{page}|{hash(str(sorted(filters.items())))}"
    frappe.cache.set_value(key, compute_data(user, page, filters), expires_in_sec=300)
    # TTL ensures keys expire even if never explicitly deleted

ALWAYS set TTL on dynamically generated cache keys. Without TTL, keys from old filter combinations accumulate and consume Redis memory indefinitely.

---

AP-6: Caching Functions with Side Effects

Wrong:

@redis_cache
def create_and_cache_report(report_name):
    doc = frappe.get_doc({"doctype": "Report Log", "report": report_name}).insert()
    return doc.name
# Second call returns cached name — does NOT create a new Report Log
# But caller expects a new document each time

Correct: NEVER use @redis_cache on functions that create documents, send emails, or perform any side effects. The decorator skips function execution on cache hits.

---

AP-7: Forgetting to Release Locks

Wrong:

frappe.lock("critical_section")
process_data()  # If this raises, lock is never released
frappe.unlock("critical_section")

Correct:

with frappe.lock("critical_section"):
    process_data()  # Lock released even if exception occurs

ALWAYS use the with context manager for distributed locks. Manual lock()/unlock() pairs leak locks on exceptions.

---

AP-8: Non-Hashable Arguments to @redis_cache

Wrong:

@redis_cache
def get_filtered_items(filters):
    return frappe.get_all("Item", filters=filters)

# This fails — dicts are not hashable
get_filtered_items({"item_group": "Products"})

Correct:

@redis_cache
def get_filtered_items(item_group, item_type=None):
    filters = {"item_group": item_group}
    if item_type:
        filters["item_type"] = item_type
    return frappe.get_all("Item", filters=filters)

ALWAYS use hashable arguments (strings, numbers, tuples) with @redis_cache. NEVER pass dicts, lists, or mutable objects.

---

AP-9: Writing to Internal Cache Keys

Wrong:

frappe.cache.set_value("doctype::meta::Item", custom_meta)
# Overwrites Frappe's internal DocType metadata cache
# Causes unpredictable behavior across the entire site

Correct: NEVER write to cache keys matching Frappe's internal patterns (doctype::*, user_permissions::*, bootinfo::*). ALWAYS prefix custom keys with your app name.

Cache API Reference

frappe.cache — RedisWrapper

The frappe.cache object is an instance of frappe.utils.redis_wrapper.RedisWrapper, providing a Python interface to Redis with automatic key namespacing and pickle serialization.

String Operations

frappe.cache.set_value(
    key: str,                    # cache key
    val: Any,                    # value (pickled automatically)
    user: str = None,            # scope to specific user
    expires_in_sec: int = None,  # TTL in seconds (None = no expiry)
    shared: bool = False,        # True = omit site prefix
)

frappe.cache.get_value(
    key: str,                    # cache key
    generator: callable = None,  # called on miss, result is cached
    user: str = None,            # user scope
    expires: bool = False,       # deprecated — use expires_in_sec on set
    shared: bool = False,        # True = omit site prefix
    use_local_cache: bool = True,# check frappe.local.cache first
) -> Any                         # returns None if not found

frappe.cache.delete_value(
    keys: str | list[str],       # key or list of keys to delete
    user: str = None,            # user scope
    make_keys: bool = True,      # True = apply make_key() to keys
    shared: bool = False,        # True = omit site prefix
)

frappe.cache.delete_keys(
    key: str,                    # pattern with wildcard (e.g., "prefix*")
)                                # deletes all matching keys

Hash Operations

frappe.cache.hset(
    name: str,                   # hash name
    key: str,                    # field name
    value: Any,                  # field value (pickled)
    shared: bool = False,        # True = omit site prefix
)

frappe.cache.hget(
    name: str,                   # hash name
    key: str,                    # field name
    generator: callable = None,  # called on miss
    shared: bool = False,        # True = omit site prefix
) -> Any

frappe.cache.hgetall(
    name: str,                   # hash name
) -> dict                        # all fields deserialized

frappe.cache.hdel(
    name: str,                   # hash name
    keys: str | list[str],       # field(s) to delete
    shared: bool = False,
    pipeline = None,             # Redis pipeline for batching
)

frappe.cache.hdel_names(
    names: list[str],            # list of hash names
    key: str,                    # common field to delete from each
)

frappe.cache.hdel_keys(
    name_starts_with: str,       # hash name prefix (wildcard)
    key: str,                    # field to delete from matching hashes
)

frappe.cache.hexists(
    name: str,                   # hash name
    key: str,                    # field name
    shared: bool = False,
) -> bool

Key Management

frappe.cache.make_key(
    key: str,                    # raw key
    user: str = None,            # user scope
    shared: bool = False,        # True = omit site prefix
) -> str                         # returns namespaced key

Format: {site_name}|{key} (or {site_name}|{user}|{key} when user is set).

---

@redis_cache Decorator

from frappe.utils.caching import redis_cache

@redis_cache                      # no TTL — cached indefinitely
def my_function(arg1, arg2): ...

@redis_cache(ttl=300)             # expires after 300 seconds
def my_function(arg1, arg2): ...

# Clear cached results
my_function.clear_cache()         # removes all cached results for this function

Availability: v15+ only. Not available in v14.

Cache key is derived from: function module + function name + serialized arguments.

---

Document Cache

# Get document from cache (read-only, no permission check)
frappe.get_cached_doc(
    doctype: str,
    name: str = None,            # omit for Single DocTypes
) -> Document

# Invalidate cached document
frappe.clear_document_cache(
    doctype: str,
    name: str,
)

# Get single field with cache
frappe.db.get_value(
    doctype: str,
    name: str,
    fieldname: str,
    cache: bool = True,          # enable cache
) -> Any

---

Distributed Locking

# Context manager (recommended)
with frappe.lock(
    name: str,                   # lock name (must be unique per resource)
    timeout: int = None,         # seconds to wait for lock acquisition
):
    ...  # exclusive section

# Manual (use only when context manager is not possible)
frappe.lock(name: str, timeout: int = None)
frappe.unlock(name: str)

On timeout, raises frappe.LockTimeoutError.

---

Site-Wide Cache Operations

# Clear ALL cache for current site (use sparingly)
frappe.clear_cache()

# Clear cache for specific user
frappe.clear_cache(user="user@example.com")

# Clear cache for specific DocType
frappe.clear_cache(doctype="Item")

---

frappe.local.cache

# Per-request dict — not persisted to Redis
frappe.local.cache: dict

# Typical usage
if "my_key" not in frappe.local.cache:
    frappe.local.cache["my_key"] = expensive_computation()
result = frappe.local.cache["my_key"]

Lifetime: created at request start, garbage collected at request end. NEVER assume values persist across requests.

Cache Implementation Examples

1. Cached API Response with TTL

import frappe
import requests

def get_exchange_rate(from_currency, to_currency):
    """Fetch exchange rate with 1-hour cache."""
    key = f"exchange_rate|{from_currency}|{to_currency}"

    rate = frappe.cache.get_value(key)
    if rate is not None:
        return rate

    # Cache miss — fetch from API
    response = requests.get(
        f"https://api.example.com/rates/{from_currency}/{to_currency}"
    )
    rate = response.json()["rate"]

    frappe.cache.set_value(key, rate, expires_in_sec=3600)
    return rate

2. @redis_cache for Expensive Query

from frappe.utils.caching import redis_cache

@redis_cache(ttl=600)
def get_top_customers(company, limit=10):
    """Get top customers by revenue — cached for 10 minutes."""
    return frappe.db.sql("""
        SELECT customer, SUM(grand_total) as total
        FROM `tabSales Invoice`
        WHERE company = %s AND docstatus = 1
        GROUP BY customer
        ORDER BY total DESC
        LIMIT %s
    """, (company, limit), as_dict=True)

# Usage
top = get_top_customers("My Company")

# Invalidate when needed
get_top_customers.clear_cache()

3. Hash-Based Configuration Cache

def get_app_settings():
    """Cache app settings as a Redis hash for fast field-level access."""
    settings = frappe.cache.hgetall("myapp|settings")
    if settings:
        return settings

    # Load from database
    doc = frappe.get_doc("My App Settings")
    settings = {
        "api_key": doc.api_key,
        "max_retries": doc.max_retries,
        "timeout": doc.timeout,
        "enabled": doc.enabled,
    }

    for key, value in settings.items():
        frappe.cache.hset("myapp|settings", key, value)

    return settings

def invalidate_app_settings(doc, method):
    """Hook: clear settings cache on update."""
    frappe.cache.delete_value("myapp|settings")

4. Request-Scoped Cache

def get_current_fiscal_year():
    """Avoid repeated DB calls within a single request."""
    cache_key = "current_fiscal_year"

    if cache_key not in frappe.local.cache:
        frappe.local.cache[cache_key] = frappe.db.get_value(
            "Fiscal Year",
            {"year_start_date": ("<=", frappe.utils.today()),
             "year_end_date": (">=", frappe.utils.today())},
            "name",
        )

    return frappe.local.cache[cache_key]

5. Distributed Lock for Batch Processing

def process_daily_reports():
    """Ensure only one worker processes reports at a time."""
    with frappe.lock("daily_report_processing"):
        pending = frappe.get_all("Report Queue",
            filters={"status": "Pending"},
            limit=50,
        )
        for report in pending:
            generate_report(report.name)
            frappe.db.set_value("Report Queue", report.name, "status", "Completed")
        frappe.db.commit()

6. Cache Warming on Startup

# In hooks.py
after_migrate = ["my_app.cache.warm_cache"]

# In my_app/cache.py
def warm_cache():
    """Pre-populate cache after migration."""
    # Cache frequently accessed documents
    for dt in ["Currency", "Company", "Fiscal Year"]:
        for name in frappe.get_all(dt, pluck="name"):
            frappe.get_cached_doc(dt, name)

    frappe.logger().info("Cache warmed successfully")

7. Conditional Cache Invalidation

def on_item_price_update(doc, method):
    """Invalidate only affected cache entries."""
    # Clear specific item price cache
    frappe.cache.delete_value(f"item_price|{doc.item_code}|{doc.price_list}")

    # Clear aggregated cache that includes this item
    frappe.cache.delete_keys(f"pricing_summary|{doc.price_list}*")

    # Clear function decorator cache
    from my_app.pricing import get_item_price
    get_item_price.clear_cache()

8. Cache with Fallback Chain

def get_setting(key):
    """Three-tier lookup: request cache → Redis → database."""
    # Tier 1: Request-scoped cache
    local_key = f"settings|{key}"
    if local_key in frappe.local.cache:
        return frappe.local.cache[local_key]

    # Tier 2: Redis cache
    value = frappe.cache.get_value(local_key)
    if value is not None:
        frappe.local.cache[local_key] = value
        return value

    # Tier 3: Database
    value = frappe.db.get_single_value("My Settings", key)
    frappe.cache.set_value(local_key, value, expires_in_sec=300)
    frappe.local.cache[local_key] = value
    return value

9. Lock with Timeout Handling

import frappe

def sync_external_data():
    """Sync with timeout protection."""
    try:
        with frappe.lock("external_sync", timeout=10):
            # If lock acquired within 10 seconds, proceed
            perform_sync()
    except frappe.LockTimeoutError:
        frappe.logger().warning("External sync skipped — another process holds the lock")