Cloudflare R2

Cloudflare R2 Object Storage

Status: Production Ready ✅ | Last Verified: 2025-12-27 | v3.0.0

Contents: Quick Start • New Features • Core R2 API • Critical Rules • Agents & Commands • References

---

Quick Start (5 Minutes)

1. Create R2 Bucket

bunx wrangler r2 bucket create my-bucket

Bucket naming: 3-63 chars, lowercase, numbers, hyphens only

2. Configure Binding

Add to wrangler.jsonc:

{
  "name": "my-worker",
  "main": "src/index.ts",
  "compatibility_date": "2025-10-11",
  "r2_buckets": [
    {
      "binding": "MY_BUCKET",          // env.MY_BUCKET
      "bucket_name": "my-bucket",      // Actual bucket
      "preview_bucket_name": "my-bucket-preview"  // Optional: dev bucket
    }
  ]
}

CRITICAL: binding = code access name, bucket_name = actual R2 bucket

3. Basic Upload/Download

import { Hono } from 'hono';

type Bindings = {
  MY_BUCKET: R2Bucket;
};

const app = new Hono<{ Bindings: Bindings }>();

// Upload
app.put('/upload/:filename', async (c) => {
  const filename = c.req.param('filename');
  const body = await c.req.arrayBuffer();

  const object = await c.env.MY_BUCKET.put(filename, body, {
    httpMetadata: {
      contentType: c.req.header('content-type') || 'application/octet-stream',
    },
  });

  return c.json({
    success: true,
    key: object.key,
    size: object.size,
  });
});

// Download
app.get('/download/:filename', async (c) => {
  const object = await c.env.MY_BUCKET.get(c.req.param('filename'));

  if (!object) {
    return c.json({ error: 'Not found' }, 404);
  }

  return new Response(object.body, {
    headers: {
      'Content-Type': object.httpMetadata?.contentType || 'application/octet-stream',
      'ETag': object.httpEtag,
    },
  });
});

export default app;

Load `references/setup-guide.md` for complete setup walkthrough.

---

New R2 Features (2025)

🆕 R2 SQL Integration - Query CSV/Parquet/JSON data with distributed SQL. Analytics without ETL. Load `references/r2-sql-integration.md`

🆕 Data Catalog (Apache Iceberg) - Table versioning, time-travel queries, schema evolution. Spark/Snowflake integration. Load `references/data-catalog-iceberg.md`

🆕 Event Notifications - Trigger Workers on object changes (upload/delete). Automate image processing, backups, webhooks. Load `references/event-notifications.md`

Advanced Features - Storage classes, bucket locks (compliance), tus resumable uploads, SSE-C encryption. Load `references/advanced-features.md`

Zero Trust Security - Cloudflare Access integration with SSO, MFA, identity policies, audit logging. Load `references/cloudflare-access-integration.md`

Performance Tuning - Caching strategies, compression, range requests, ETags, monitoring best practices. Load `references/performance-optimization.md`

---

Core R2 Workers API - Quick Reference

put() - Upload Objects

await env.MY_BUCKET.put(key, data, options?)

Upload with metadata, prevent overwrites with onlyIf. Load `references/workers-api.md` for complete R2PutOptions.

get() - Download Objects

const object = await env.MY_BUCKET.get(key, options?)

Returns R2ObjectBody | null. Supports range requests, conditional operations. Load `references/workers-api.md` for read methods (text(), json(), arrayBuffer(), blob()).

head() - Get Metadata Only

const object = await env.MY_BUCKET.head(key)

Check existence, get size, etag, metadata without downloading body. Useful for validation and caching.

delete() - Delete Objects

await env.MY_BUCKET.delete(key | keys[])  // Single or bulk (max 1000)

Bulk delete up to 1000 keys in single call. Always succeeds (idempotent).

list() - List Objects

const listed = await env.MY_BUCKET.list(options?)

Pagination with cursor, prefix filtering, delimiter for folders. Load `references/workers-api.md` for R2ListOptions.

createMultipartUpload() - Large Files (>100MB)

const multipart = await env.MY_BUCKET.createMultipartUpload(key, options?)

For files >100MB. Load `references/common-patterns.md` for complete multipart workflow with part upload and completion.

Load `references/workers-api.md` when: Need complete API reference, interface definitions (R2Object, R2ObjectBody, R2PutOptions, R2GetOptions), conditional operations, checksums, or advanced options.

---

Critical Rules

Always Do ✅

1. Set contentType on uploads - Files will download as binary otherwise 2. Use batch delete for multiple objects (up to 1000 keys) 3. Set cache headers for static assets (cacheControl) 4. Use presigned URLs for large client uploads 5. Use multipart upload for files >100MB 6. Set CORS policy before browser uploads 7. Set expiry times on presigned URLs (1-24 hours) 8. Handle errors with try/catch 9. Use head() when you only need metadata (not get()) 10. Use conditional operations to prevent overwrites

Never Do ❌

1. Never expose R2 access keys in client-side code 2. Never skip contentType (files will download as binary) 3. Never delete in loops (use batch delete) 4. Never upload without error handling 5. Never skip CORS for browser uploads 6. Never use multipart for small files (<5MB overhead) 7. Never delete >1000 keys in single call (will fail) 8. Never assume uploads succeed (always check response) 9. Never skip presigned URL expiry (security risk) 10. Never hardcode bucket names (use bindings)

---

Top Use Cases

Use Case 1: Image/Asset Storage

app.put('/api/upload/image', async (c) => {
  const file = await c.req.parseBody();
  const image = file['image'] as File;

  await c.env.MY_BUCKET.put(`images/${image.name}`, image.stream(), {
    httpMetadata: {
      contentType: image.type,
      cacheControl: 'public, max-age=31536000, immutable',
    },
  });

  return c.json({ success: true });
});

Use Case 2: Direct Client Upload (Presigned URLs)

Generate secure upload URLs for client-side uploads. See templates/r2-presigned-urls.ts for complete implementation using aws4fetch.

Additional Patterns in References

Load `references/common-patterns.md` for:

• Multipart upload (files >100MB) - Complete workflow with part management
• Bulk operations - Batch delete, cleanup patterns with pagination
• Custom metadata tracking - User files, versions, approval workflows
• Versioned file storage - Version history with latest pointer pattern
• Backup & archive patterns - Automated backups with retention policies
• Thumbnail generation & caching - On-demand image processing
• Static site hosting - SPA fallback and cache strategies
• CDN with origin fallback - R2 as cache layer

Load `templates/r2-multipart-upload.ts` for complete multipart example.

---

Available Agents & Commands

Autonomous Agents

Agents handle complex multi-step workflows automatically:

• r2-setup-automator - Complete R2 setup (bucket creation → binding → TypeScript types → deployment)
• multipart-orchestrator - Large file uploads with chunking, error recovery, and progress tracking
• cors-debugger - Systematic CORS troubleshooting with configuration generation and testing
• s3-migration-planner - AWS S3 to R2 migration planning, data transfer, and cost analysis
• event-notification-setup - Event-driven workflows with Workers, Queues, and automation

Quick Commands

Fast access to common R2 operations:

• /r2-setup - Create bucket and configure binding in wrangler.jsonc
• /r2-presigned-url - Generate presigned URLs for secure client-side uploads/downloads
• /r2-cors-debug - Diagnose and fix CORS configuration issues
• /r2-multipart-init - Initialize multipart upload workflow for large files

---

When to Load References

Core References (Existing Features)

`references/setup-guide.md` - First-time setup, binding configuration, TypeScript types, deployment walkthrough

`references/workers-api.md` - Complete API reference (all methods + options), conditional operations, checksums

`references/common-patterns.md` - Multipart uploads, retry logic with backoff, batch operations, cache strategies

`references/s3-compatibility.md` - S3 migration guide, S3 client library usage, aws4fetch presigned URL signing

`references/cors-configuration.md` - Browser access setup, CORS debugging, security policies, Dashboard configuration

New Features References (2025)

`references/event-notifications.md` - Event-driven automation, Queue integration, image processing, webhook triggers

`references/advanced-features.md` - Storage classes (cost optimization), bucket locks (compliance), tus resumable uploads, SSE-C encryption

`references/r2-sql-integration.md` - SQL queries on R2 data (CSV/Parquet/JSON), analytics patterns, performance tuning

`references/data-catalog-iceberg.md` - Apache Iceberg tables, time-travel queries, schema evolution, Spark/Snowflake integration

`references/cloudflare-access-integration.md` - Zero Trust security, SSO (Google/Okta/Azure AD), identity policies, MFA, audit logging

`references/performance-optimization.md` - Caching (browser/CDN/Workers), compression (gzip/Brotli), range requests, ETags, monitoring

---

Using Bundled Resources

References (references/)

• setup-guide.md - Complete setup walkthrough (bucket creation → deployment)
• workers-api.md - Complete Workers API reference (all methods + options)
• common-patterns.md - Advanced patterns (multipart, retry, batch, performance)
• s3-compatibility.md - S3 compatibility guide (migration, aws4fetch, S3 clients)
• cors-configuration.md - CORS setup guide (Dashboard, scenarios, troubleshooting, security)

Templates (templates/)

• r2-simple-upload.ts - Basic upload/download Worker
• r2-multipart-upload.ts - Complete multipart upload implementation
• r2-presigned-urls.ts - Presigned URL generation (upload + download)
• r2-cors-config.json - CORS configuration examples
• wrangler-r2-config.jsonc - Complete wrangler.jsonc with R2 binding

---

CORS Configuration

Configure CORS for browser uploads/downloads. Load `references/cors-configuration.md` for complete guide including Dashboard setup, common scenarios, troubleshooting, and security best practices.

---

Error Handling

try {
  await env.MY_BUCKET.put(key, data);
} catch (error: any) {
  const message = error.message;

  if (message.includes('R2_ERROR')) {
    // Generic R2 error
  } else if (message.includes('exceeded')) {
    // Quota exceeded
  } else if (message.includes('precondition')) {
    // Conditional operation failed (onlyIf)
  }

  console.error('R2 Error:', message);
  return c.json({ error: 'Storage operation failed' }, 500);
}

Load `references/common-patterns.md` for retry logic with exponential backoff, circuit breaker patterns, and advanced error recovery.

---

Known Issues Prevented

---

Wrangler Commands

# Bucket management
wrangler r2 bucket create <BUCKET_NAME>
wrangler r2 bucket list
wrangler r2 bucket delete <BUCKET_NAME>

# Object management
wrangler r2 object put <BUCKET>/<KEY> --file=<PATH>
wrangler r2 object get <BUCKET>/<KEY> --file=<OUTPUT>
wrangler r2 object delete <BUCKET>/<KEY>

# List objects
wrangler r2 object list <BUCKET>
wrangler r2 object list <BUCKET> --prefix="folder/"

---

Official Documentation

• R2 Overview: https://developers.cloudflare.com/r2/
• Workers API: https://developers.cloudflare.com/r2/api/workers/workers-api-reference/
• Multipart Upload: https://developers.cloudflare.com/r2/api/workers/workers-multipart-usage/
• Presigned URLs: https://developers.cloudflare.com/r2/api/s3/presigned-urls/
• CORS Configuration: https://developers.cloudflare.com/r2/buckets/cors/

---

Questions? Issues?

1. Check references/setup-guide.md for setup walkthrough 2. Review references/workers-api.md for API reference 3. See references/common-patterns.md for advanced patterns 4. Load templates/ for working code examples

R2 Advanced Features

Last Updated: 2025-12-27

Advanced R2 capabilities including storage classes, bucket locks, tus protocol resumable uploads, and server-side encryption with customer-provided keys (SSE-C).

---

Storage Classes

Storage classes optimize costs based on object access patterns. Choose the appropriate class for different data lifecycle stages.

Available Storage Classes

Note: Storage class support is currently in development. Check official docs for availability.

Configuring Storage Class (Future)

// Upload with storage class (when available)
await env.MY_BUCKET.put('archive/old-data.zip', data, {
  storageClass: 'ARCHIVE',
  httpMetadata: {
    contentType: 'application/zip',
  },
});

Lifecycle Policies (Future)

Automatically transition objects between storage classes:

// wrangler.jsonc (when available)
{
  "r2_buckets": [
    {
      "binding": "MY_BUCKET",
      "bucket_name": "my-bucket",
      "lifecycle_rules": [
        {
          "id": "archive-old-backups",
          "prefix": "backups/",
          "transitions": [
            {
              "days": 30,
              "storage_class": "INFREQUENT_ACCESS"
            },
            {
              "days": 90,
              "storage_class": "ARCHIVE"
            }
          ]
        },
        {
          "id": "delete-temp-files",
          "prefix": "temp/",
          "expiration": {
            "days": 7
          }
        }
      ]
    }
  ]
}

---

Bucket Locks (Compliance Mode)

Bucket locks prevent accidental or malicious deletion of objects, enforcing retention policies for compliance and data protection.

Object Lock Modes

Compliance Mode:

• Objects cannot be deleted or modified until retention period expires
• Even account administrators cannot override
• Required for regulatory compliance (FINRA, SEC, HIPAA, etc.)

Governance Mode:

• Objects protected but can be overridden with special permissions
• Useful for internal policies without strict regulatory requirements

Enabling Bucket Lock

Dashboard Setup: 1. Navigate to R2 → Select bucket 2. Click "Settings" tab 3. Scroll to "Object Lock" section 4. Enable "Object Lock" (cannot be disabled once enabled) 5. Set default retention period

Wrangler Configuration:

# Enable object lock on bucket (irreversible!)
bunx wrangler r2 bucket update my-bucket --object-lock

# Set retention mode
bunx wrangler r2 bucket update my-bucket \
  --object-lock-mode COMPLIANCE \
  --object-lock-days 90

Uploading with Retention

// Upload with retention policy
await env.MY_BUCKET.put('legal/document.pdf', data, {
  httpMetadata: {
    contentType: 'application/pdf',
  },
  // Object cannot be deleted for 365 days
  objectLock: {
    mode: 'COMPLIANCE',
    retainUntilDate: new Date(Date.now() + 365 * 24 * 60 * 60 * 1000).toISOString(),
  },
});

Use Cases

• Financial Records: SEC/FINRA require 7-year retention
• Healthcare Data: HIPAA compliance requires immutable audit logs
• Legal Documents: E-discovery and litigation hold requirements
• Backup Protection: Prevent ransomware from deleting backups
• Audit Trails: Immutable logs for compliance investigations

---

tus Protocol (Resumable Uploads)

The tus protocol enables resumable file uploads, allowing uploads to continue after network interruptions without starting over. Better than multipart for unreliable connections.

Why Use tus Instead of Multipart?

Enabling tus Support

Wrangler Configuration:

{
  "r2_buckets": [
    {
      "binding": "MY_BUCKET",
      "bucket_name": "my-bucket",
      "tus_enabled": true  // Enable tus protocol
    }
  ]
}

Client-Side tus Upload

JavaScript Client:

import * as tus from 'tus-js-client';

function uploadWithTus(file, onProgress) {
  const upload = new tus.Upload(file, {
    // tus endpoint (served by your Worker)
    endpoint: '/api/upload/tus',

    // Retry on failure
    retryDelays: [0, 1000, 3000, 5000],

    // Custom metadata
    metadata: {
      filename: file.name,
      filetype: file.type,
      userId: currentUserId,
    },

    // Progress tracking
    onProgress: (bytesUploaded, bytesTotal) => {
      const percentage = ((bytesUploaded / bytesTotal) * 100).toFixed(2);
      onProgress(percentage);
      console.log(`Uploaded ${bytesUploaded} of ${bytesTotal} bytes (${percentage}%)`);
    },

    // Upload complete
    onSuccess: () => {
      console.log('Upload complete!');
    },

    // Upload failed
    onError: (error) => {
      console.error('Upload failed:', error);
    },
  });

  // Start upload
  upload.start();

  // Return upload object for controls
  return {
    pause: () => upload.abort(),
    resume: () => upload.start(),
    abort: () => upload.abort(true),
  };
}

Worker tus Handler

import { Hono } from 'hono';
import { tusMiddleware } from '@tus/server';

type Bindings = {
  MY_BUCKET: R2Bucket;
};

const app = new Hono<{ Bindings: Bindings }>();

// tus upload endpoint
app.all('/api/upload/tus/*', async (c) => {
  const tusHandler = tusMiddleware({
    // Store upload metadata in R2
    datastore: new R2DataStore(c.env.MY_BUCKET),

    // Allow upload resume
    namingFunction: (req) => {
      return `${Date.now()}-${crypto.randomUUID()}`;
    },

    // Maximum upload size (5GB)
    maxSize: 5 * 1024 * 1024 * 1024,

    // Handle upload complete
    onUploadComplete: async (req, upload) => {
      console.log('Upload complete:', upload.id);

      // Move from temp to final location
      const finalKey = `uploads/${upload.metadata.filename}`;
      await c.env.MY_BUCKET.put(finalKey, upload.stream, {
        httpMetadata: {
          contentType: upload.metadata.filetype,
        },
        customMetadata: {
          uploadId: upload.id,
          userId: upload.metadata.userId,
        },
      });
    },
  });

  return tusHandler(c.req.raw);
});

export default app;

tus Best Practices

1. Set reasonable chunk sizes - 1-10MB chunks for optimal performance 2. Implement authentication - Secure tus endpoints 3. Monitor storage usage - Clean up incomplete uploads 4. Set expiry times - Delete abandoned uploads after 24-48 hours 5. Use progress callbacks - Provide user feedback during upload 6. Handle network changes - Detect disconnections and auto-resume

---

Server-Side Encryption with Customer Keys (SSE-C)

SSE-C allows you to manage your own encryption keys while Cloudflare handles encryption/decryption operations. Keys never leave your control.

How SSE-C Works

1. Client provides encryption key with upload request 2. Cloudflare encrypts object using provided key 3. Cloudflare discards key after encryption 4. Client must provide same key for download 5. Without key, object cannot be decrypted

Uploading with SSE-C

import crypto from 'crypto';

// Generate or retrieve encryption key (32 bytes)
const encryptionKey = crypto.randomBytes(32);
const keyBase64 = encryptionKey.toString('base64');
const keyMD5 = crypto.createHash('md5').update(encryptionKey).digest('base64');

// Upload with customer-provided encryption
await env.MY_BUCKET.put('secure/document.pdf', data, {
  httpMetadata: {
    contentType: 'application/pdf',
  },
  encryption: {
    algorithm: 'AES256',
    key: keyBase64,
    keyMD5: keyMD5,
  },
});

// Store encryption key securely (e.g., in KV or external key management system)
await env.ENCRYPTION_KEYS.put('document.pdf', keyBase64);

Downloading with SSE-C

// Retrieve encryption key
const encryptionKey = await env.ENCRYPTION_KEYS.get('document.pdf');

if (!encryptionKey) {
  return c.json({ error: 'Encryption key not found' }, 404);
}

const keyBuffer = Buffer.from(encryptionKey, 'base64');
const keyMD5 = crypto.createHash('md5').update(keyBuffer).digest('base64');

// Download with encryption key
const object = await env.MY_BUCKET.get('secure/document.pdf', {
  encryption: {
    algorithm: 'AES256',
    key: encryptionKey,
    keyMD5: keyMD5,
  },
});

if (!object) {
  return c.json({ error: 'Object not found or decryption failed' }, 404);
}

return new Response(object.body, {
  headers: {
    'Content-Type': 'application/pdf',
  },
});

SSE-C Key Management

Option 1: Cloudflare Workers KV

// Store keys in KV (encrypted at rest by Cloudflare)
await env.ENCRYPTION_KEYS.put(`key:${objectKey}`, encryptionKey);

// Retrieve keys
const key = await env.ENCRYPTION_KEYS.get(`key:${objectKey}`);

Option 2: External Key Management Service (KMS)

// Store keys in AWS KMS, Azure Key Vault, or HashiCorp Vault
async function getEncryptionKey(objectKey: string): Promise<string> {
  const response = await fetch(`https://kms.example.com/keys/${objectKey}`, {
    headers: {
      'Authorization': `Bearer ${env.KMS_API_KEY}`,
    },
  });

  const data = await response.json();
  return data.encryptionKey;
}

Option 3: Derived Keys (Password-Based)

import { pbkdf2 } from 'crypto';

// Derive encryption key from user password
function deriveEncryptionKey(password: string, salt: string): string {
  return pbkdf2(password, salt, 100000, 32, 'sha256').toString('base64');
}

// Use derived key for encryption
const userPassword = 'user-secret-password';
const salt = 'unique-salt-per-user';
const encryptionKey = deriveEncryptionKey(userPassword, salt);

SSE-C Best Practices

1. Never log encryption keys - Keys are sensitive credentials 2. Use strong key generation - Crypto-secure random number generators 3. Rotate keys periodically - Re-encrypt objects with new keys 4. Backup keys securely - Losing keys means losing data permanently 5. Use separate keys per object - Limit blast radius of key compromise 6. Implement key derivation - Use password-based keys for user data 7. Monitor key access - Audit who retrieves encryption keys

SSE-C Use Cases

• PII/PHI Data: HIPAA compliance with customer-managed keys
• Financial Records: SOC 2 compliance requirements
• Intellectual Property: Source code, patents, trade secrets
• User Data Encryption: Per-user encryption keys
• Zero-Knowledge Architecture: Server cannot decrypt user data

---

Combining Advanced Features

Example: Encrypted, Locked, Resumable Upload

// Upload sensitive document with all protections
async function uploadSecureDocument(
  file: File,
  env: Bindings,
  userId: string
) {
  // Generate encryption key
  const encryptionKey = crypto.randomBytes(32);
  const keyBase64 = encryptionKey.toString('base64');
  const keyMD5 = crypto.createHash('md5').update(encryptionKey).digest('base64');

  // Upload with tus (resumable), SSE-C (encrypted), and object lock (immutable)
  const upload = new tus.Upload(file, {
    endpoint: '/api/upload/tus',
    metadata: {
      filename: file.name,
      filetype: file.type,
      userId: userId,
      encrypted: 'true',
      locked: 'true',
    },
    headers: {
      // SSE-C headers
      'x-amz-server-side-encryption-customer-algorithm': 'AES256',
      'x-amz-server-side-encryption-customer-key': keyBase64,
      'x-amz-server-side-encryption-customer-key-md5': keyMD5,
      // Object lock headers
      'x-amz-object-lock-mode': 'COMPLIANCE',
      'x-amz-object-lock-retain-until-date': new Date(
        Date.now() + 365 * 24 * 60 * 60 * 1000
      ).toISOString(),
    },
    onSuccess: async () => {
      // Store encryption key securely
      await env.ENCRYPTION_KEYS.put(`user:${userId}:${file.name}`, keyBase64);
      console.log('Secure upload complete!');
    },
  });

  upload.start();
}

---

Migration Strategies

Enabling Advanced Features on Existing Buckets

Storage Classes: Can be enabled retroactively, objects keep current class until changed

Bucket Locks: CANNOT be enabled on existing buckets with data (create new bucket)

tus Protocol: Can be enabled anytime, doesn't affect existing objects

SSE-C: Can be enabled anytime, only new uploads will be encrypted

Migration Steps for Bucket Lock

# 1. Create new bucket with object lock
bunx wrangler r2 bucket create my-bucket-locked --object-lock

# 2. Copy objects to new bucket (Workers script)
# 3. Update application to use new bucket
# 4. Verify all data migrated
# 5. Delete old bucket

---

Troubleshooting

Storage Class Errors

Error: "Storage class not supported" Solution: Feature in development, use Standard class for now

Bucket Lock Issues

Error: "Cannot enable object lock on existing bucket" Solution: Create new bucket with object lock enabled, migrate data

Error: "Cannot delete object - retention period active" Solution: Wait for retention period to expire, or use Governance mode with override permissions

tus Upload Failures

Error: "tus endpoint not found" Solution: Enable tus in wrangler.jsonc, implement tus handler in Worker

Error: "Upload not resuming" Solution: Check tus client configuration, ensure upload ID is being stored

SSE-C Decryption Failures

Error: "Decryption failed - wrong key" Solution: Verify encryption key matches upload key, check key storage

Error: "Cannot download - key required" Solution: Provide encryption key in download request headers

---

Official Documentation

• Storage Classes (In Development): https://developers.cloudflare.com/r2/
• Bucket Locks: https://developers.cloudflare.com/r2/buckets/object-lock/
• tus Protocol: https://tus.io/protocols/resumable-upload
• SSE-C: https://developers.cloudflare.com/r2/api/s3/encryption/

---

Advanced features for enterprise-grade R2 deployments!

Cloudflare Access Integration with R2

Last Updated: 2025-12-27

Restrict R2 bucket access using Cloudflare Access for identity-based authentication. Implement Zero Trust security with user, group, and application-level policies.

---

Overview

Cloudflare Access provides identity-based access control for R2 buckets, allowing you to restrict access to specific users, groups, or applications within your organization. This enables Zero Trust security for internal tools, employee-only content, and sensitive data.

Key Benefits:

• Zero Trust Security - Verify identity before granting access
• SSO Integration - SAML, OAuth, OIDC providers (Google, Okta, Azure AD)
• Granular Policies - User, group, IP, device, and location-based rules
• Audit Logging - Track all access attempts
• No VPN Required - Secure access from anywhere
• Multi-factor Authentication - Enforce MFA for sensitive data

Use Cases:

• Internal file storage (HR documents, financials)
• Employee-only resources (training videos, company files)
• Partner/vendor access (restricted file sharing)
• Development/staging environments
• Compliance-required access controls

---

Architecture Overview

┌─────────────────┐
│ User Browser    │
│ (Employee)      │
└────────┬────────┘
         │ 1. Request r2.example.com/file.pdf
         ↓
┌─────────────────┐
│ Cloudflare      │
│ Access          │◄──── 2. Check policy
│                 │
└────────┬────────┘
         │ 3. Redirect to IdP if not authenticated
         ↓
┌─────────────────┐
│ Identity        │
│ Provider        │ (Google, Okta, Azure AD)
│ (SSO)           │
└────────┬────────┘
         │ 4. User authenticates
         ↓
┌─────────────────┐
│ Cloudflare      │
│ Access          │◄──── 5. Validate identity
└────────┬────────┘
         │ 6. Issue JWT token
         ↓
┌─────────────────┐
│ R2 Bucket       │
│ (Protected)     │◄──── 7. Access granted
└─────────────────┘

---

Setup Cloudflare Access for R2

Step 1: Enable Cloudflare Access

Dashboard Steps: 1. Navigate to Zero Trust → Access → Applications 2. Click Add an application 3. Select Self-hosted 4. Configure application details:

• Application name: "Employee File Storage"
• Session duration: 24 hours
• Application domain: files.example.com (custom domain for R2)

Step 2: Configure Identity Provider

Add SSO Provider: 1. Go to Zero Trust → Settings → Authentication 2. Click Add new under Login methods 3. Select provider (Google, Okta, Azure AD, etc.) 4. Configure OAuth/SAML settings 5. Test authentication

Supported Providers:

• Google Workspace
• Okta
• Azure AD (Microsoft Entra)
• OneLogin
• GitHub
• Generic SAML/OIDC

Step 3: Create Access Policies

Policy Types:

• Allow: Grant access to matching users
• Block: Deny access to matching users
• Bypass: Skip Access for specific scenarios
• Service Auth: API tokens for service-to-service

Example Policy: Allow Employees

Policy Name: Employee Access
Action: Allow

Include:
- Emails ending in @company.com

Exclude:
- Email: contractor@company.com

Require:
- Multi-factor authentication

Dashboard Configuration: 1. In application settings, click Add a policy 2. Configure include rules (who should have access) 3. Configure exclude rules (who should be denied) 4. Add require rules (additional requirements like MFA)

Step 4: Configure R2 Custom Domain

Map Custom Domain to R2 Bucket:

# Create custom domain for R2 bucket
bunx wrangler r2 bucket domain add my-bucket files.example.com

Dashboard Steps: 1. R2 → Select bucket → Settings 2. Scroll to Custom Domains 3. Add domain: files.example.com 4. Configure DNS (CNAME to <bucket>.r2.cloudflarestorage.com)

Step 5: Apply Access Policy to R2 Domain

Dashboard Steps: 1. Zero Trust → Access → Applications → Your application 2. Verify Application domain matches R2 custom domain 3. Policies will automatically protect all requests to domain

---

Access Policies Examples

Policy 1: Employees Only

Action: Allow

Include:
- Emails ending in @company.com

Require:
- Country: United States
- Multi-factor authentication

Use Case: Restrict access to US-based employees with MFA

Policy 2: Specific Groups

Action: Allow

Include:
- Group: Finance Team
- Group: Executives

Require:
- Device Posture: Corporate managed

Use Case: Only finance team and executives on corporate devices

Policy 3: IP Allowlist

Action: Allow

Include:
- IP range: 203.0.113.0/24 (Office IP)
- Email: remote-worker@company.com

Require:
- Multi-factor authentication (for remote workers)

Use Case: Office network or specific remote workers with MFA

Policy 4: Partner Access

Action: Allow

Include:
- Email: partner1@vendor.com
- Email: partner2@vendor.com

Require:
- Valid client certificate
- Session duration: 1 hour

Use Case: Temporary partner access with certificate authentication

Policy 5: Service Tokens (API Access)

Action: Service Auth

Service Tokens:
- Name: Backup Service
- Name: Analytics Pipeline

Use Case: Automated systems accessing R2 with service tokens

---

Workers Integration with Access

Validate Access JWT in Workers

import { Hono } from 'hono';

type Bindings = {
  MY_BUCKET: R2Bucket;
  ACCESS_AUD: string;  // Cloudflare Access audience tag
};

const app = new Hono<{ Bindings: Bindings }>();

// Validate Cloudflare Access JWT
async function validateAccessToken(
  request: Request,
  env: Bindings
): Promise<{ valid: boolean; email?: string; groups?: string[] }> {
  const cookieHeader = request.headers.get('Cookie');

  if (!cookieHeader) {
    return { valid: false };
  }

  // Extract Access JWT from cookie
  const cfAccessToken = cookieHeader
    .split(';')
    .find(c => c.trim().startsWith('CF_Authorization='))
    ?.split('=')[1];

  if (!cfAccessToken) {
    return { valid: false };
  }

  // Verify JWT with Cloudflare's public keys
  const certsUrl = 'https://example.cloudflareaccess.com/cdn-cgi/access/certs';
  const certsResponse = await fetch(certsUrl);
  const certs = await certsResponse.json();

  // Decode and verify JWT (use jose library)
  try {
    const { payload } = await verifyJWT(cfAccessToken, certs, env.ACCESS_AUD);

    return {
      valid: true,
      email: payload.email,
      groups: payload.groups || [],
    };
  } catch (error) {
    console.error('JWT validation failed:', error);
    return { valid: false };
  }
}

// Protected endpoint
app.get('/files/:filename', async (c) => {
  // Validate Access token
  const auth = await validateAccessToken(c.req.raw, c.env);

  if (!auth.valid) {
    return c.json({ error: 'Unauthorized - Cloudflare Access required' }, 401);
  }

  // Check group-based permissions
  const filename = c.req.param('filename');

  if (filename.startsWith('finance/') && !auth.groups?.includes('Finance Team')) {
    return c.json({ error: 'Forbidden - Finance Team access required' }, 403);
  }

  // Fetch file from R2
  const object = await c.env.MY_BUCKET.get(filename);

  if (!object) {
    return c.json({ error: 'File not found' }, 404);
  }

  // Log access for audit trail
  console.log(`File accessed: ${filename} by ${auth.email}`);

  return new Response(object.body, {
    headers: {
      'Content-Type': object.httpMetadata?.contentType || 'application/octet-stream',
    },
  });
});

export default app;

Service Token Authentication

// Service-to-service access with service tokens
app.get('/api/files/:filename', async (c) => {
  const serviceToken = c.req.header('CF-Access-Client-Id');
  const serviceSecret = c.req.header('CF-Access-Client-Secret');

  if (!serviceToken || !serviceSecret) {
    return c.json({ error: 'Service token required' }, 401);
  }

  // Validate service token with Access
  const validation = await fetch(
    'https://example.cloudflareaccess.com/cdn-cgi/access/token',
    {
      headers: {
        'CF-Access-Client-Id': serviceToken,
        'CF-Access-Client-Secret': serviceSecret,
      },
    }
  );

  if (!validation.ok) {
    return c.json({ error: 'Invalid service token' }, 401);
  }

  // Proceed with file access
  const filename = c.req.param('filename');
  const object = await c.env.MY_BUCKET.get(filename);

  if (!object) {
    return c.json({ error: 'File not found' }, 404);
  }

  return new Response(object.body);
});

---

Audit Logging

Enable Access Logs

Dashboard Steps: 1. Zero Trust → Logs → Access 2. Enable Access audit logs 3. Configure log destination:

• R2 bucket (store logs long-term)
• Third-party SIEM (Splunk, DataDog)
• Cloudflare Logpush

Query Access Logs

-- R2 SQL query on Access logs
SELECT
  timestamp,
  user_email,
  action,
  resource,
  country,
  device_posture_check_passed
FROM r2('access-logs-bucket/access/*.json')
WHERE action = 'login'
  AND timestamp >= '2025-01-15'
ORDER BY timestamp DESC;

-- Failed access attempts
SELECT
  user_email,
  COUNT(*) as failed_attempts,
  MAX(timestamp) as last_attempt
FROM r2('access-logs-bucket/access/*.json')
WHERE action = 'login'
  AND success = false
  AND timestamp >= CURRENT_DATE - INTERVAL '7' DAYS
GROUP BY user_email
HAVING failed_attempts > 5
ORDER BY failed_attempts DESC;

---

Advanced Patterns

Pattern 1: Per-User File Access

// Ensure users can only access their own files
app.get('/user-files/:filename', async (c) => {
  const auth = await validateAccessToken(c.req.raw, c.env);

  if (!auth.valid) {
    return c.json({ error: 'Unauthorized' }, 401);
  }

  const filename = c.req.param('filename');
  const userId = auth.email?.split('@')[0]; // Extract user ID from email

  // Ensure file belongs to user
  if (!filename.startsWith(`users/${userId}/`)) {
    return c.json({ error: 'Access denied - not your file' }, 403);
  }

  const object = await c.env.MY_BUCKET.get(filename);

  if (!object) {
    return c.json({ error: 'File not found' }, 404);
  }

  return new Response(object.body);
});

Pattern 2: Time-Based Access

// Restrict access to business hours
app.get('/restricted/:filename', async (c) => {
  const auth = await validateAccessToken(c.req.raw, c.env);

  if (!auth.valid) {
    return c.json({ error: 'Unauthorized' }, 401);
  }

  // Check business hours (9 AM - 5 PM EST)
  const now = new Date();
  const hour = now.getUTCHours() - 5; // EST offset

  if (hour < 9 || hour >= 17) {
    return c.json({
      error: 'Access restricted to business hours (9 AM - 5 PM EST)',
    }, 403);
  }

  const filename = c.req.param('filename');
  const object = await c.env.MY_BUCKET.get(filename);

  if (!object) {
    return c.json({ error: 'File not found' }, 404);
  }

  return new Response(object.body);
});

Pattern 3: Download Limits

// Track and limit file downloads per user
app.get('/limited/:filename', async (c) => {
  const auth = await validateAccessToken(c.req.raw, c.env);

  if (!auth.valid) {
    return c.json({ error: 'Unauthorized' }, 401);
  }

  const filename = c.req.param('filename');
  const downloadKey = `downloads:${auth.email}:${filename}`;

  // Check download count (stored in KV)
  const downloads = parseInt(await c.env.DOWNLOAD_LIMITS.get(downloadKey) || '0');

  if (downloads >= 3) {
    return c.json({
      error: 'Download limit exceeded (max 3 per file)',
    }, 429);
  }

  // Increment download count
  await c.env.DOWNLOAD_LIMITS.put(downloadKey, String(downloads + 1), {
    expirationTtl: 86400, // Reset after 24 hours
  });

  const object = await c.env.MY_BUCKET.get(filename);

  if (!object) {
    return c.json({ error: 'File not found' }, 404);
  }

  return new Response(object.body);
});

---

Security Best Practices

1. Always enforce MFA for sensitive data access 2. Use device posture checks to ensure corporate-managed devices 3. Implement IP allowlists for office networks when possible 4. Rotate service tokens regularly (every 90 days) 5. Monitor audit logs for suspicious activity 6. Use short session durations (4-8 hours) for sensitive apps 7. Implement least privilege - grant minimum necessary access 8. Review policies quarterly - remove stale access 9. Use groups for access management (easier than individual emails) 10. Test policies before deploying to production

---

Troubleshooting

Access Denied Errors

Error: "Forbidden - Access policy does not allow"

Solutions:

• Check policy includes user's email or group
• Verify user authenticated with correct IdP
• Check exclude rules aren't blocking user
• Ensure MFA is enabled if required

JWT Validation Failures

Error: "Invalid JWT signature"

Solutions:

• Verify Access audience tag matches (env.ACCESS_AUD)
• Check JWT expiration (max session duration)
• Ensure using latest public keys from /certs endpoint
• Validate cookie is being sent with request

Service Token Issues

Error: "Service token authentication failed"

Solutions:

• Verify service token is still valid (not expired or revoked)
• Check client ID and secret are correct
• Ensure service token is in correct application policy
• Regenerate token if compromised

Custom Domain Not Protected

Problem: Access policies not applied to R2 custom domain

Solutions:

• Verify custom domain configured in Access application
• Check DNS records point to correct R2 endpoint
• Wait for DNS propagation (up to 24 hours)
• Test with incognito/private browsing to clear cache

---

Official Documentation

• Cloudflare Access: https://developers.cloudflare.com/cloudflare-one/policies/access/
• Identity Providers: https://developers.cloudflare.com/cloudflare-one/identity/idp-integration/
• Service Tokens: https://developers.cloudflare.com/cloudflare-one/identity/service-tokens/
• JWT Validation: https://developers.cloudflare.com/cloudflare-one/identity/authorization-cookie/validating-json/
• Audit Logs: https://developers.cloudflare.com/cloudflare-one/insights/logs/

---

Secure R2 access with Zero Trust and identity-based policies!

R2 Common Patterns

Last Updated: 2025-10-21

---

Image Upload & Serving

Upload with Automatic Content-Type Detection

import { Hono } from 'hono';

type Bindings = {
  IMAGES: R2Bucket;
};

const app = new Hono<{ Bindings: Bindings }>();

app.post('/upload/image', async (c) => {
  const formData = await c.req.formData();
  const file = formData.get('image') as File;

  if (!file) {
    return c.json({ error: 'No file provided' }, 400);
  }

  // Validate file type
  const allowedTypes = ['image/jpeg', 'image/png', 'image/webp', 'image/gif'];
  if (!allowedTypes.includes(file.type)) {
    return c.json({ error: 'Invalid file type' }, 400);
  }

  // Generate unique filename
  const extension = file.name.split('.').pop();
  const filename = `${crypto.randomUUID()}.${extension}`;
  const key = `images/${filename}`;

  // Upload to R2
  const arrayBuffer = await file.arrayBuffer();
  const object = await c.env.IMAGES.put(key, arrayBuffer, {
    httpMetadata: {
      contentType: file.type,
      cacheControl: 'public, max-age=31536000, immutable',
    },
    customMetadata: {
      originalFilename: file.name,
      uploadedAt: new Date().toISOString(),
    },
  });

  return c.json({
    success: true,
    url: `/images/${filename}`,
    key: object.key,
    size: object.size,
  });
});

// Serve image
app.get('/images/:filename', async (c) => {
  const filename = c.req.param('filename');
  const key = `images/${filename}`;

  const object = await c.env.IMAGES.get(key);

  if (!object) {
    return c.json({ error: 'Image not found' }, 404);
  }

  return new Response(object.body, {
    headers: {
      'Content-Type': object.httpMetadata?.contentType || 'image/jpeg',
      'Cache-Control': 'public, max-age=31536000, immutable',
      'ETag': object.httpEtag,
    },
  });
});

export default app;

---

User File Storage with Folder Organization

app.post('/users/:userId/files', async (c) => {
  const userId = c.req.param('userId');
  const formData = await c.req.formData();
  const file = formData.get('file') as File;

  if (!file) {
    return c.json({ error: 'No file provided' }, 400);
  }

  // Organize by user ID and date
  const date = new Date().toISOString().split('T')[0]; // YYYY-MM-DD
  const filename = file.name;
  const key = `users/${userId}/${date}/${filename}`;

  const arrayBuffer = await file.arrayBuffer();
  const object = await c.env.MY_BUCKET.put(key, arrayBuffer, {
    httpMetadata: {
      contentType: file.type,
      contentDisposition: `attachment; filename="${filename}"`,
    },
    customMetadata: {
      userId,
      uploadDate: date,
      originalSize: file.size.toString(),
    },
  });

  return c.json({
    success: true,
    fileId: object.key,
    size: object.size,
  });
});

// List user's files
app.get('/users/:userId/files', async (c) => {
  const userId = c.req.param('userId');
  const cursor = c.req.query('cursor');

  const listed = await c.env.MY_BUCKET.list({
    prefix: `users/${userId}/`,
    limit: 100,
    cursor: cursor || undefined,
  });

  return c.json({
    files: listed.objects.map(obj => ({
      key: obj.key,
      filename: obj.key.split('/').pop(),
      size: obj.size,
      uploaded: obj.uploaded,
      metadata: obj.customMetadata,
    })),
    hasMore: listed.truncated,
    cursor: listed.cursor,
  });
});

---

Thumbnail Generation & Caching

app.get('/thumbnails/:filename', async (c) => {
  const filename = c.req.param('filename');
  const width = parseInt(c.req.query('w') || '200');
  const height = parseInt(c.req.query('h') || '200');

  const thumbnailKey = `thumbnails/${width}x${height}/${filename}`;

  // Check if thumbnail already exists
  let thumbnail = await c.env.IMAGES.get(thumbnailKey);

  if (!thumbnail) {
    // Get original image
    const original = await c.env.IMAGES.get(`images/${filename}`);

    if (!original) {
      return c.json({ error: 'Image not found' }, 404);
    }

    // Generate thumbnail (using Cloudflare Images or external service)
    // This is a placeholder - use actual image processing
    const thumbnailData = await generateThumbnail(
      await original.arrayBuffer(),
      width,
      height
    );

    // Store thumbnail for future requests
    await c.env.IMAGES.put(thumbnailKey, thumbnailData, {
      httpMetadata: {
        contentType: 'image/jpeg',
        cacheControl: 'public, max-age=31536000, immutable',
      },
    });

    thumbnail = await c.env.IMAGES.get(thumbnailKey);
  }

  return new Response(thumbnail!.body, {
    headers: {
      'Content-Type': 'image/jpeg',
      'Cache-Control': 'public, max-age=31536000, immutable',
    },
  });
});

async function generateThumbnail(
  imageData: ArrayBuffer,
  width: number,
  height: number
): Promise<ArrayBuffer> {
  // Use Cloudflare Images API, sharp, or other image processing library
  // This is a placeholder
  return imageData;
}

---

Versioned File Storage

app.put('/files/:filename', async (c) => {
  const filename = c.req.param('filename');
  const body = await c.req.arrayBuffer();

  // Get current version number
  const versionKey = `versions/${filename}/latest`;
  const currentVersion = await c.env.MY_BUCKET.head(versionKey);

  let version = 1;
  if (currentVersion?.customMetadata?.version) {
    version = parseInt(currentVersion.customMetadata.version) + 1;
  }

  // Store new version
  const versionedKey = `versions/${filename}/v${version}`;
  await c.env.MY_BUCKET.put(versionedKey, body, {
    httpMetadata: {
      contentType: c.req.header('content-type') || 'application/octet-stream',
    },
    customMetadata: {
      version: version.toString(),
      createdAt: new Date().toISOString(),
    },
  });

  // Update "latest" pointer
  await c.env.MY_BUCKET.put(versionKey, body, {
    httpMetadata: {
      contentType: c.req.header('content-type') || 'application/octet-stream',
    },
    customMetadata: {
      version: version.toString(),
      latestVersion: 'true',
    },
  });

  return c.json({
    success: true,
    version,
    key: versionedKey,
  });
});

// Get specific version
app.get('/files/:filename/v/:version', async (c) => {
  const filename = c.req.param('filename');
  const version = c.req.param('version');

  const key = `versions/${filename}/v${version}`;
  const object = await c.env.MY_BUCKET.get(key);

  if (!object) {
    return c.json({ error: 'Version not found' }, 404);
  }

  return new Response(object.body, {
    headers: {
      'Content-Type': object.httpMetadata?.contentType || 'application/octet-stream',
    },
  });
});

---

Backup & Archive Pattern

// Daily database backup to R2
async function backupDatabase(env: Bindings) {
  const date = new Date().toISOString().split('T')[0];
  const key = `backups/database/${date}/dump.sql.gz`;

  // Generate backup (placeholder)
  const backupData = await generateDatabaseDump();

  await env.BACKUPS.put(key, backupData, {
    httpMetadata: {
      contentType: 'application/gzip',
      contentEncoding: 'gzip',
    },
    customMetadata: {
      backupDate: date,
      backupType: 'full',
      database: 'production',
    },
  });

  // Delete backups older than 30 days
  await cleanupOldBackups(env, 30);
}

async function cleanupOldBackups(env: Bindings, retentionDays: number) {
  const cutoffDate = new Date();
  cutoffDate.setDate(cutoffDate.getDate() - retentionDays);

  const listed = await env.BACKUPS.list({
    prefix: 'backups/database/',
  });

  const oldBackups = listed.objects.filter(
    obj => obj.uploaded < cutoffDate
  );

  if (oldBackups.length > 0) {
    const keysToDelete = oldBackups.map(obj => obj.key);
    await env.BACKUPS.delete(keysToDelete);
  }
}

---

Static Site Hosting with SPA Fallback

app.get('/*', async (c) => {
  const url = new URL(c.req.url);
  let key = url.pathname.slice(1); // Remove leading slash

  if (key === '' || key.endsWith('/')) {
    key += 'index.html';
  }

  let object = await c.env.STATIC.get(key);

  // SPA fallback: if file not found, try index.html
  if (!object && !key.includes('.')) {
    object = await c.env.STATIC.get('index.html');
  }

  if (!object) {
    return c.json({ error: 'Not found' }, 404);
  }

  const headers = new Headers();
  object.writeHttpMetadata(headers);

  // Set appropriate cache headers
  if (key.match(/\.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2)$/)) {
    headers.set('Cache-Control', 'public, max-age=31536000, immutable');
  } else {
    headers.set('Cache-Control', 'public, max-age=3600, must-revalidate');
  }

  return new Response(object.body, { headers });
});

---

CDN with Origin Fallback

// Use R2 as CDN with external origin fallback
app.get('/cdn/*', async (c) => {
  const url = new URL(c.req.url);
  const key = url.pathname.replace('/cdn/', '');

  // Check R2 cache first
  let object = await c.env.CDN_CACHE.get(key);

  if (!object) {
    // Fetch from origin
    const originUrl = `https://origin.example.com/${key}`;
    const response = await fetch(originUrl);

    if (!response.ok) {
      return c.json({ error: 'Not found on origin' }, 404);
    }

    const data = await response.arrayBuffer();
    const contentType = response.headers.get('content-type') || 'application/octet-stream';

    // Cache in R2
    await c.env.CDN_CACHE.put(key, data, {
      httpMetadata: {
        contentType,
        cacheControl: 'public, max-age=31536000',
      },
    });

    object = await c.env.CDN_CACHE.get(key);
  }

  return new Response(object!.body, {
    headers: {
      'Content-Type': object!.httpMetadata?.contentType || 'application/octet-stream',
      'Cache-Control': 'public, max-age=31536000',
      'X-Cache': object ? 'HIT' : 'MISS',
    },
  });
});

---

Signed Upload with Quota Limits

app.post('/request-upload', async (c) => {
  const { userId, filename, fileSize } = await c.req.json();

  // Check user's quota
  const quota = await getUserQuota(userId);

  if (quota.used + fileSize > quota.total) {
    return c.json({ error: 'Quota exceeded' }, 403);
  }

  // Generate presigned URL
  const r2Client = new AwsClient({
    accessKeyId: c.env.R2_ACCESS_KEY_ID,
    secretAccessKey: c.env.R2_SECRET_ACCESS_KEY,
  });

  const key = `users/${userId}/${filename}`;
  const url = new URL(
    `https://my-bucket.${c.env.ACCOUNT_ID}.r2.cloudflarestorage.com/${key}`
  );

  url.searchParams.set('X-Amz-Expires', '3600');

  const signed = await r2Client.sign(
    new Request(url, { method: 'PUT' }),
    { aws: { signQuery: true } }
  );

  return c.json({
    uploadUrl: signed.url,
    expiresIn: 3600,
  });
});

async function getUserQuota(userId: string) {
  // Query database for user quota
  return {
    used: 1024 * 1024 * 100, // 100MB used
    total: 1024 * 1024 * 1024, // 1GB total
  };
}

---

Best Practices Summary

1. Use meaningful key prefixes for organization (users/{id}/, images/, backups/) 2. Set appropriate cache headers for static assets 3. Store metadata for tracking and filtering 4. Use bulk delete instead of loops 5. Implement cleanup for old/temporary files 6. Add authentication before presigned URL generation 7. Validate file types before uploading 8. Use UUIDs for unique filenames 9. Set expiry times on presigned URLs 10. Monitor quota to prevent overages

---

Retry Logic with Exponential Backoff

async function r2WithRetry<T>(
  operation: () => Promise<T>,
  maxRetries = 3
): Promise<T> {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      return await operation();
    } catch (error: any) {
      const isRetryable =
        error.message.includes('network') ||
        error.message.includes('timeout') ||
        error.message.includes('temporary');

      if (!isRetryable || attempt === maxRetries - 1) {
        throw error;
      }

      // Exponential backoff: 1s, 2s, 4s
      const delay = Math.min(1000 * Math.pow(2, attempt), 5000);
      await new Promise(resolve => setTimeout(resolve, delay));
    }
  }
  throw new Error('Retry failed');
}

// Usage
const object = await r2WithRetry(() =>
  env.MY_BUCKET.put('file.txt', data)
);

---

Circuit Breaker Pattern

class R2CircuitBreaker {
  private failures = 0;
  private lastFailure = 0;
  private readonly threshold = 5;
  private readonly timeout = 60000; // 1 minute

  async execute<T>(operation: () => Promise<T>): Promise<T> {
    if (this.isOpen()) {
      throw new Error('Circuit breaker open');
    }

    try {
      const result = await operation();
      this.onSuccess();
      return result;
    } catch (error) {
      this.onFailure();
      throw error;
    }
  }

  private isOpen(): boolean {
    return (
      this.failures >= this.threshold &&
      Date.now() - this.lastFailure < this.timeout
    );
  }

  private onSuccess() {
    this.failures = 0;
  }

  private onFailure() {
    this.failures++;
    this.lastFailure = Date.now();
  }
}

R2 CORS Configuration Guide

Last Updated: 2025-11-26 Official Docs: https://developers.cloudflare.com/r2/buckets/cors/

---

Dashboard Configuration

1. Cloudflare Dashboard → R2 → Your bucket 2. Settings tab → CORS Policy → Add CORS policy

---

Common Scenarios

Public Assets (Read-Only)

{
  "CORSRules": [
    {
      "AllowedOrigins": ["*"],
      "AllowedMethods": ["GET", "HEAD"],
      "AllowedHeaders": ["Range"],
      "MaxAgeSeconds": 3600
    }
  ]
}

Use when: Serving public images, videos, or static assets from R2.

Upload/Download (Full Access)

{
  "CORSRules": [
    {
      "AllowedOrigins": ["https://app.example.com"],
      "AllowedMethods": ["GET", "PUT", "POST", "DELETE", "HEAD"],
      "AllowedHeaders": ["Content-Type", "Content-MD5"],
      "ExposeHeaders": ["ETag"],
      "MaxAgeSeconds": 3600
    }
  ]
}

Use when: Allowing browser uploads and downloads from specific domain.

Development Environment

{
  "CORSRules": [
    {
      "AllowedOrigins": ["http://localhost:3000", "http://localhost:5173"],
      "AllowedMethods": ["GET", "PUT", "POST", "DELETE", "HEAD"],
      "AllowedHeaders": ["*"],
      "ExposeHeaders": ["ETag"],
      "MaxAgeSeconds": 3600
    }
  ]
}

Use when: Local development and testing.

---

Troubleshooting CORS Errors

Error: "CORS policy: No 'Access-Control-Allow-Origin' header"

Cause: CORS not configured or origin not allowed.

Solution: 1. Add origin to AllowedOrigins in bucket CORS policy 2. Verify method is in AllowedMethods 3. For * origin, ensure credentials mode is not 'include'

Error: "CORS policy: Request header field X is not allowed"

Cause: Custom header not in AllowedHeaders.

Solution: Add header to AllowedHeaders or use ["*"] for development.

Error: Presigned URLs failing with CORS

Cause: Presigned URLs bypass worker CORS headers.

Solution: Configure CORS at bucket level (Dashboard), not in Worker.

---

Security Best Practices

1. *Never use `[""]` origin in production - Always specify exact domains 2. Limit methods - Only allow methods your app actually uses 3. Set reasonable MaxAgeSeconds - 3600 (1 hour) is typically sufficient 4. Use ExposeHeaders - Include ETag for caching validation 5. Consider preflight caching** - Higher MaxAgeSeconds reduces preflight requests

---

Testing CORS Configuration

# Test GET request
curl -H "Origin: https://example.com" \
     -H "Access-Control-Request-Method: GET" \
     -X OPTIONS \
     https://bucket.account.r2.cloudflarestorage.com/file.txt

# Should return Access-Control-Allow-Origin header

See templates/r2-cors-config.json for more examples.

R2 Data Catalog with Apache Iceberg

Last Updated: 2025-12-27 Status: NEW FEATURE (2025)

Manage structured data in R2 using Apache Iceberg table format with versioning, time-travel queries, schema evolution, and integration with analytics tools.

---

Overview

R2 Data Catalog provides built-in Apache Iceberg support for managing structured data in object storage. Iceberg is an open table format that enables ACID transactions, schema evolution, and efficient query performance on large datasets.

Key Features:

• Table versioning - Track changes over time with snapshots
• Time-travel queries - Query data as it existed at any point in time
• Schema evolution - Add/remove columns without rewriting data
• Partition evolution - Change partitioning without data migration
• ACID transactions - Atomic writes with snapshot isolation
• Metadata management - Efficient file tracking and pruning

Use Cases:

• Data lakes and lakehouses
• Analytics on large datasets
• Compliance and audit trails (time-travel)
• Schema evolution without downtime
• Integration with Spark, Snowflake, Trino, Presto

---

Apache Iceberg Basics

What is Apache Iceberg?

Iceberg is a high-performance table format for huge analytic datasets. It solves problems with traditional data lake formats:

Iceberg Table Structure

r2://my-bucket/warehouse/
├── database1/
│   └── table1/
│       ├── metadata/
│       │   ├── v1.metadata.json      # Table schema v1
│       │   ├── v2.metadata.json      # Table schema v2
│       │   ├── snap-123.avro         # Snapshot manifest
│       │   └── snap-456.avro
│       └── data/
│           ├── part-00001.parquet
│           ├── part-00002.parquet
│           └── ...

Metadata files track schema, snapshots, and data files Data files contain actual table data (Parquet format) Snapshots represent table state at a point in time

---

Creating Iceberg Tables

Option 1: Using R2 Data Catalog API

import { Hono } from 'hono';

type Bindings = {
  DATA_BUCKET: R2Bucket;
  CATALOG: DataCatalog; // Iceberg catalog binding
};

const app = new Hono<{ Bindings: Bindings }>();

// Create Iceberg table
app.post('/catalog/create-table', async (c) => {
  const { namespace, tableName, schema } = await c.req.json();

  const table = await c.env.CATALOG.createTable({
    namespace: namespace,     // e.g., "sales" or "logs"
    name: tableName,          // e.g., "transactions"
    schema: {
      fields: [
        { id: 1, name: 'transaction_id', type: 'long', required: true },
        { id: 2, name: 'user_id', type: 'long', required: true },
        { id: 3, name: 'amount', type: 'decimal(10,2)', required: true },
        { id: 4, name: 'timestamp', type: 'timestamptz', required: true },
        { id: 5, name: 'status', type: 'string', required: false },
      ],
    },
    partitionSpec: {
      fields: [
        { sourceId: 4, transform: 'day', name: 'transaction_day' },  // Partition by day
      ],
    },
    location: `r2://data-bucket/warehouse/${namespace}/${tableName}`,
  });

  return c.json({
    success: true,
    table: table.name,
    location: table.location,
    snapshot: table.currentSnapshotId,
  });
});

export default app;

Option 2: Using Spark SQL

from pyspark.sql import SparkSession

# Configure Spark for R2 with Iceberg
spark = SparkSession.builder \
    .config('spark.sql.catalog.r2', 'org.apache.iceberg.spark.SparkCatalog') \
    .config('spark.sql.catalog.r2.type', 'rest') \
    .config('spark.sql.catalog.r2.uri', 'https://catalog-api.example.workers.dev') \
    .config('spark.sql.catalog.r2.warehouse', 'r2://my-bucket/warehouse') \
    .getOrCreate()

# Create table with SQL
spark.sql("""
    CREATE TABLE r2.sales.transactions (
        transaction_id BIGINT,
        user_id BIGINT,
        amount DECIMAL(10,2),
        timestamp TIMESTAMP,
        status STRING
    )
    USING iceberg
    PARTITIONED BY (days(timestamp))
    LOCATION 'r2://data-bucket/warehouse/sales/transactions'
""")

---

Inserting Data

Append New Data

# Spark DataFrame to Iceberg table
from pyspark.sql import functions as F

# Create sample data
transactions = spark.createDataFrame([
    (1, 101, 99.99, '2025-01-15 10:00:00', 'completed'),
    (2, 102, 149.50, '2025-01-15 11:30:00', 'completed'),
    (3, 103, 79.99, '2025-01-15 12:00:00', 'pending'),
], ['transaction_id', 'user_id', 'amount', 'timestamp', 'status'])

# Append to Iceberg table (creates new snapshot)
transactions.writeTo('r2.sales.transactions').append()

Overwrite Data

# Overwrite specific partition
transactions.writeTo('r2.sales.transactions') \
    .overwritePartitions()  # Overwrites only affected partitions

Upsert (Merge)

# Merge/upsert data
spark.sql("""
    MERGE INTO r2.sales.transactions t
    USING updates u
    ON t.transaction_id = u.transaction_id
    WHEN MATCHED THEN
        UPDATE SET t.status = u.status
    WHEN NOT MATCHED THEN
        INSERT *
""")

---

Time-Travel Queries

Query Historical Data

# Query table as of specific timestamp
spark.sql("""
    SELECT *
    FROM r2.sales.transactions
    FOR SYSTEM_TIME AS OF '2025-01-15 10:00:00'
    WHERE amount > 100
""")

# Query specific snapshot by ID
spark.sql("""
    SELECT *
    FROM r2.sales.transactions
    FOR SYSTEM_VERSION AS OF 12345678
    WHERE status = 'completed'
""")

# Query snapshot 3 days ago
spark.sql("""
    SELECT *
    FROM r2.sales.transactions
    FOR SYSTEM_TIME AS OF CURRENT_TIMESTAMP - INTERVAL 3 DAYS
""")

List Snapshots

# View snapshot history
spark.sql("""
    SELECT
        made_current_at,
        snapshot_id,
        parent_id,
        operation,
        summary
    FROM r2.sales.transactions.snapshots
    ORDER BY made_current_at DESC
""")

---

Schema Evolution

Add Columns

# Add new column without rewriting data
spark.sql("""
    ALTER TABLE r2.sales.transactions
    ADD COLUMNS (
        payment_method STRING COMMENT 'Payment type (card, cash, crypto)',
        discount_applied DECIMAL(10,2) DEFAULT 0.0
    )
""")

# Existing data: new columns are NULL
# New data: columns populated

Rename Columns

# Rename column (metadata-only operation)
spark.sql("""
    ALTER TABLE r2.sales.transactions
    RENAME COLUMN status TO transaction_status
""")

Drop Columns

# Drop column (metadata-only, data not deleted)
spark.sql("""
    ALTER TABLE r2.sales.transactions
    DROP COLUMN discount_applied
""")

Change Column Type

# Promote int to long (safe operation)
spark.sql("""
    ALTER TABLE r2.sales.transactions
    ALTER COLUMN user_id TYPE BIGINT
""")

---

Partition Evolution

Add Partitioning

# Change partitioning scheme
spark.sql("""
    ALTER TABLE r2.sales.transactions
    DROP PARTITION FIELD transaction_day
""")

spark.sql("""
    ALTER TABLE r2.sales.transactions
    ADD PARTITION FIELD hours(timestamp)  -- Partition by hour instead
""")

Note: Iceberg handles partition evolution automatically - old data stays in old partitions, new data uses new partitions.

---

Maintenance Operations

Expire Old Snapshots

# Remove snapshots older than 7 days
spark.sql("""
    CALL r2.system.expire_snapshots(
        table => 'sales.transactions',
        older_than => TIMESTAMP '2025-01-08 00:00:00',
        retain_last => 5  -- Keep at least 5 snapshots
    )
""")

Remove Orphan Files

# Delete data files not referenced by any snapshot
spark.sql("""
    CALL r2.system.remove_orphan_files(
        table => 'sales.transactions',
        older_than => TIMESTAMP '2025-01-08 00:00:00'
    )
""")

Rewrite Data Files

# Compact small files into larger ones
spark.sql("""
    CALL r2.system.rewrite_data_files(
        table => 'sales.transactions',
        strategy => 'binpack',
        options => map('target-file-size-bytes', '536870912')  -- 512MB files
    )
""")

Rewrite Manifests

# Optimize metadata files
spark.sql("""
    CALL r2.system.rewrite_manifests('sales.transactions')
""")

---

Integration with Analytics Tools

Snowflake External Tables

-- Create external table pointing to Iceberg in R2
CREATE EXTERNAL TABLE sales.transactions
WITH LOCATION = 'r2://data-bucket/warehouse/sales/transactions/'
FILE_FORMAT = (TYPE = PARQUET)
CATALOG = 'iceberg_catalog'
AUTO_REFRESH = TRUE;

-- Query from Snowflake
SELECT * FROM sales.transactions WHERE amount > 100;

Trino/Presto Queries

-- Configure Iceberg catalog
-- In catalog/iceberg.properties:
connector.name=iceberg
iceberg.catalog.type=rest
iceberg.rest.uri=https://catalog-api.example.workers.dev
iceberg.rest.warehouse=r2://my-bucket/warehouse

-- Query from Trino
SELECT
    DATE_TRUNC('day', timestamp) as day,
    COUNT(*) as transactions,
    SUM(amount) as total_amount
FROM iceberg.sales.transactions
WHERE timestamp >= CURRENT_DATE - INTERVAL '7' DAY
GROUP BY 1
ORDER BY 1;

DuckDB Integration

import duckdb

# Connect to Iceberg table in R2
conn = duckdb.connect()

conn.execute("""
    INSTALL iceberg;
    LOAD iceberg;
""")

# Query Iceberg table
result = conn.execute("""
    SELECT *
    FROM iceberg_scan('r2://data-bucket/warehouse/sales/transactions')
    WHERE amount > 100
""").fetchdf()

print(result)

---

Workers Integration

Query Iceberg from Workers

type Bindings = {
  CATALOG: DataCatalog;
  R2_SQL: R2SQLEngine;
};

const app = new Hono<{ Bindings: Bindings }>();

// Query Iceberg table with R2 SQL
app.get('/analytics/recent-transactions', async (c) => {
  const sql = `
    SELECT
      transaction_id,
      user_id,
      amount,
      timestamp,
      status
    FROM iceberg.sales.transactions
    WHERE timestamp >= CURRENT_TIMESTAMP - INTERVAL '1' HOUR
    ORDER BY timestamp DESC
    LIMIT 100
  `;

  const result = await c.env.R2_SQL.execute(sql);

  return c.json({
    transactions: result.rows,
    count: result.rowCount,
  });
});

// Get table schema
app.get('/catalog/tables/:namespace/:table/schema', async (c) => {
  const namespace = c.req.param('namespace');
  const tableName = c.req.param('table');

  const table = await c.env.CATALOG.loadTable(namespace, tableName);

  return c.json({
    name: table.name,
    schema: table.schema,
    partitionSpec: table.partitionSpec,
    currentSnapshot: table.currentSnapshotId,
    location: table.location,
  });
});

// List table snapshots
app.get('/catalog/tables/:namespace/:table/snapshots', async (c) => {
  const namespace = c.req.param('namespace');
  const tableName = c.req.param('table');

  const snapshots = await c.env.CATALOG.listSnapshots(namespace, tableName);

  return c.json({
    snapshots: snapshots.map(s => ({
      snapshotId: s.snapshotId,
      parentId: s.parentId,
      timestamp: s.timestampMillis,
      operation: s.operation,
      summary: s.summary,
    })),
  });
});

export default app;

---

Performance Best Practices

1. Choose Appropriate Partitioning

# Good: Partition by date for time-series data
PARTITIONED BY (days(timestamp))

# Bad: Too many partitions (high cardinality)
PARTITIONED BY (user_id)  # Millions of partitions

# Good: Multi-level partitioning
PARTITIONED BY (days(timestamp), bucket(16, user_id))

2. Hidden Partitioning

Iceberg automatically handles partitioning:

# Write data without specifying partitions
transactions.writeTo('r2.sales.transactions').append()

# Iceberg automatically partitions by days(timestamp)
# No need to organize files manually

3. File Size Optimization

# Configure target file size (512MB recommended)
spark.conf.set('write.target-file-size-bytes', 536870912)

# Compact small files regularly
spark.sql("CALL r2.system.rewrite_data_files('sales.transactions')")

4. Metadata Caching

# Cache table metadata for faster queries
spark.conf.set('iceberg.metadata-cache-enabled', 'true')
spark.conf.set('iceberg.metadata-cache-expiration-interval-ms', '300000')  # 5 minutes

---

Migration from Parquet to Iceberg

Step 1: Analyze Existing Data

# Read existing Parquet files
parquet_df = spark.read.parquet('r2://data-bucket/legacy-data/*.parquet')

# Infer schema
parquet_df.printSchema()

Step 2: Create Iceberg Table

# Create Iceberg table with matching schema
spark.sql("""
    CREATE TABLE r2.sales.transactions_iceberg
    USING iceberg
    PARTITIONED BY (days(timestamp))
    AS SELECT * FROM parquet_df
    WHERE 1=0  -- Create schema only
""")

Step 3: Migrate Data

# Migrate data in batches
parquet_df.writeTo('r2.sales.transactions_iceberg').append()

# Verify row counts
iceberg_count = spark.table('r2.sales.transactions_iceberg').count()
parquet_count = parquet_df.count()

assert iceberg_count == parquet_count, "Row count mismatch!"

Step 4: Switch Applications

# Update application queries to use Iceberg table
# Old: spark.read.parquet('r2://data-bucket/legacy-data/*.parquet')
# New: spark.table('r2.sales.transactions_iceberg')

Step 5: Clean Up

# After verification, delete old Parquet files
# Keep legacy data for rollback period (e.g., 30 days)

---

Troubleshooting

Table Not Found Errors

Error: "Table 'sales.transactions' does not exist"

Solutions:

• Check namespace and table name spelling
• Verify catalog configuration (REST URI, warehouse location)
• Ensure table was created successfully

Schema Evolution Errors

Error: "Cannot change column type from STRING to INT"

Solutions:

• Only safe type promotions allowed (int→long, float→double)
• Use ALTER COLUMN for safe changes
• Rewrite data for unsafe changes

Snapshot Expiration Warnings

Error: "Snapshot not found: 12345678"

Solutions:

• Snapshot may have been expired
• Check retention policy with retain_last
• Increase snapshot retention period

Performance Issues

Problem: Slow queries on large tables

Solutions:

• Add appropriate partitioning
• Compact small files with rewrite_data_files
• Enable metadata caching
• Use column pruning (SELECT specific columns)
• Add partition pruning (WHERE on partition columns)

---

Official Documentation

• Apache Iceberg: https://iceberg.apache.org/
• R2 Data Catalog: https://developers.cloudflare.com/r2/data-catalog/
• Iceberg Spark: https://iceberg.apache.org/docs/latest/spark/
• Schema Evolution: https://iceberg.apache.org/docs/latest/evolution/
• Maintenance: https://iceberg.apache.org/docs/latest/maintenance/

---

Manage structured data in R2 with Apache Iceberg - versioning, time-travel, and schema evolution!

R2 Event Notifications

Last Updated: 2025-12-27

Configure Workers to automatically respond to R2 object changes (uploads, deletions) using event notifications and Cloudflare Queues integration.

---

Overview

R2 event notifications enable event-driven workflows by automatically triggering Workers when objects are created, updated, or deleted in R2 buckets. Events are delivered through Cloudflare Queues for reliable, asynchronous processing.

Common Use Cases:

• Resize images on upload
• Generate thumbnails automatically
• Update database indexes when files are added
• Backup files to secondary storage on upload
• Clean up related resources when files are deleted
• Trigger webhooks for external systems
• Analytics and audit logging

---

Event Subscription Setup

1. Create Queue for Events

# Create queue to receive R2 events
bunx wrangler queues create r2-events

2. Configure Event Notifications

Option A: Wrangler Configuration

Add to wrangler.jsonc:

{
  "name": "r2-event-handler",
  "main": "src/index.ts",
  "compatibility_date": "2025-10-11",

  // R2 bucket binding
  "r2_buckets": [
    {
      "binding": "MY_BUCKET",
      "bucket_name": "my-bucket",
      // Configure event notifications
      "event_notification_rules": [
        {
          "queue": "r2-events",
          "rules": [
            {
              "prefix": "images/",     // Only images folder
              "suffix": ".jpg"          // Only JPG files
            }
          ]
        }
      ]
    }
  ],

  // Queue consumer binding
  "queues": {
    "consumers": [
      {
        "queue": "r2-events",
        "max_batch_size": 10,
        "max_batch_timeout": 5,
        "max_retries": 3,
        "dead_letter_queue": "r2-events-dlq"
      }
    ]
  }
}

Option B: Dashboard Configuration

1. Navigate to R2 → Select bucket 2. Click "Settings" tab 3. Scroll to "Event Notifications" 4. Click "Create notification rule" 5. Configure:

• Queue name: Select your queue
• Event types: object-create, object-delete
• Prefix filter: Optional path prefix (e.g., "uploads/")
• Suffix filter: Optional file extension (e.g., ".png")

---

Event Payload Structure

Object Created Event

interface R2ObjectCreatedEvent {
  account: string;           // Cloudflare account ID
  action: 'PutObject';       // Event type
  bucket: string;            // Bucket name
  object: {
    key: string;             // Object key
    size: number;            // File size in bytes
    eTag: string;            // Object ETag
  };
  eventTime: string;         // ISO 8601 timestamp
}

Object Deleted Event

interface R2ObjectDeletedEvent {
  account: string;
  action: 'DeleteObject';
  bucket: string;
  object: {
    key: string;             // Deleted object key
  };
  eventTime: string;
}

---

Event Handler Worker

Basic Event Handler

import { Hono } from 'hono';

type Bindings = {
  MY_BUCKET: R2Bucket;
};

interface Env extends Bindings {
  // Queue will be automatically available
}

export default {
  async queue(batch: MessageBatch<R2Event>, env: Env): Promise<void> {
    for (const message of batch.messages) {
      const event = message.body;

      try {
        await handleR2Event(event, env);
        message.ack(); // Acknowledge successful processing
      } catch (error) {
        console.error('Failed to process event:', error);
        message.retry(); // Retry on failure
      }
    }
  }
};

async function handleR2Event(event: R2Event, env: Env) {
  console.log(`Event: ${event.action} on ${event.object.key}`);

  if (event.action === 'PutObject') {
    await handleObjectCreated(event, env);
  } else if (event.action === 'DeleteObject') {
    await handleObjectDeleted(event, env);
  }
}

async function handleObjectCreated(event: R2ObjectCreatedEvent, env: Env) {
  const key = event.object.key;

  // Example: Process only images
  if (key.startsWith('images/') && key.match(/\.(jpg|png|webp)$/)) {
    console.log(`New image uploaded: ${key} (${event.object.size} bytes)`);

    // Trigger image processing (resize, thumbnail generation, etc.)
    await processImage(key, env);
  }
}

async function handleObjectDeleted(event: R2ObjectDeletedEvent, env: Env) {
  const key = event.object.key;
  console.log(`Object deleted: ${key}`);

  // Clean up related resources
  await cleanupRelatedResources(key, env);
}

type R2Event = R2ObjectCreatedEvent | R2ObjectDeletedEvent;

---

Common Patterns

Pattern 1: Image Resize on Upload

async function processImage(key: string, env: Env) {
  // Get original image
  const original = await env.MY_BUCKET.get(key);

  if (!original) {
    console.error(`Object not found: ${key}`);
    return;
  }

  const imageData = await original.arrayBuffer();

  // Resize image (using Cloudflare Images or external service)
  const resized = await resizeImage(imageData, { width: 800, height: 600 });

  // Store resized version
  const resizedKey = key.replace('images/', 'images/resized/');
  await env.MY_BUCKET.put(resizedKey, resized, {
    httpMetadata: {
      contentType: original.httpMetadata?.contentType || 'image/jpeg',
      cacheControl: 'public, max-age=31536000, immutable',
    },
    customMetadata: {
      originalKey: key,
      processedAt: new Date().toISOString(),
    },
  });

  console.log(`Resized image stored: ${resizedKey}`);
}

async function resizeImage(data: ArrayBuffer, options: { width: number; height: number }): Promise<ArrayBuffer> {
  // Use Cloudflare Images API, sharp, or external image service
  // Placeholder implementation
  return data;
}

Pattern 2: Thumbnail Generation

async function generateThumbnails(key: string, env: Env) {
  const original = await env.MY_BUCKET.get(key);

  if (!original) return;

  const imageData = await original.arrayBuffer();

  // Generate multiple thumbnail sizes
  const sizes = [
    { name: 'small', width: 150, height: 150 },
    { name: 'medium', width: 300, height: 300 },
    { name: 'large', width: 600, height: 600 },
  ];

  for (const size of sizes) {
    const thumbnail = await resizeImage(imageData, size);
    const thumbnailKey = `thumbnails/${size.name}/${key}`;

    await env.MY_BUCKET.put(thumbnailKey, thumbnail, {
      httpMetadata: {
        contentType: 'image/jpeg',
        cacheControl: 'public, max-age=31536000, immutable',
      },
      customMetadata: {
        size: size.name,
        originalKey: key,
      },
    });
  }

  console.log(`Thumbnails generated for: ${key}`);
}

Pattern 3: Database Index Update

async function updateDatabaseIndex(event: R2ObjectCreatedEvent, env: Env) {
  // Update database with new file metadata
  await env.DB.prepare(
    `INSERT INTO files (key, size, uploaded_at, etag)
     VALUES (?, ?, ?, ?)`
  )
    .bind(
      event.object.key,
      event.object.size,
      event.eventTime,
      event.object.eTag
    )
    .run();

  console.log(`Database updated for: ${event.object.key}`);
}

async function cleanupDatabaseIndex(event: R2ObjectDeletedEvent, env: Env) {
  // Remove file from database
  await env.DB.prepare(`DELETE FROM files WHERE key = ?`)
    .bind(event.object.key)
    .run();

  console.log(`Database cleaned up for: ${event.object.key}`);
}

Pattern 4: Backup to Secondary Storage

async function backupToSecondary(key: string, env: Env) {
  const object = await env.MY_BUCKET.get(key);

  if (!object) return;

  const data = await object.arrayBuffer();

  // Store in backup bucket
  await env.BACKUP_BUCKET.put(key, data, {
    httpMetadata: object.httpMetadata,
    customMetadata: {
      ...object.customMetadata,
      backedUpAt: new Date().toISOString(),
      originalBucket: env.MY_BUCKET.name,
    },
  });

  console.log(`Backed up to secondary: ${key}`);
}

Pattern 5: Webhook Notification

async function sendWebhook(event: R2Event, env: Env) {
  const webhookUrl = env.WEBHOOK_URL;

  await fetch(webhookUrl, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'X-Event-Type': event.action,
    },
    body: JSON.stringify({
      action: event.action,
      bucket: event.bucket,
      object: event.object,
      timestamp: event.eventTime,
    }),
  });

  console.log(`Webhook sent for: ${event.object.key}`);
}

---

Error Handling and Retries

Retry Logic with Exponential Backoff

async function handleEventWithRetry(event: R2Event, env: Env, maxRetries = 3) {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      await handleR2Event(event, env);
      return; // Success
    } catch (error) {
      const isLastAttempt = attempt === maxRetries - 1;

      if (isLastAttempt) {
        console.error(`Failed after ${maxRetries} attempts:`, error);
        throw error; // Will go to dead letter queue
      }

      // Exponential backoff: 1s, 2s, 4s
      const delay = Math.min(1000 * Math.pow(2, attempt), 5000);
      await new Promise(resolve => setTimeout(resolve, delay));

      console.log(`Retry attempt ${attempt + 1}/${maxRetries}`);
    }
  }
}

Dead Letter Queue Handling

// Separate worker for processing failed events
export default {
  async queue(batch: MessageBatch<R2Event>, env: Env): Promise<void> {
    for (const message of batch.messages) {
      const event = message.body;

      // Log failed event for manual investigation
      console.error('Event processing failed multiple times:', {
        action: event.action,
        key: event.object.key,
        timestamp: event.eventTime,
        retries: message.attempts,
      });

      // Store in persistent storage for later retry
      await env.FAILED_EVENTS.put(
        `failed/${Date.now()}-${event.object.key}`,
        JSON.stringify(event)
      );

      message.ack(); // Acknowledge to prevent further retries
    }
  }
};

---

Event Filtering

Prefix and Suffix Filters

Filter events by object path:

{
  "event_notification_rules": [
    {
      "queue": "image-processing",
      "rules": [
        {
          "prefix": "uploads/images/",
          "suffix": ".jpg"
        },
        {
          "prefix": "uploads/images/",
          "suffix": ".png"
        }
      ]
    },
    {
      "queue": "video-processing",
      "rules": [
        {
          "prefix": "uploads/videos/",
          "suffix": ".mp4"
        }
      ]
    }
  ]
}

Application-Level Filtering

Filter events in Worker code:

async function handleR2Event(event: R2Event, env: Env) {
  const key = event.object.key;

  // Filter by file type
  if (key.match(/\.(jpg|png|webp)$/)) {
    await processImage(key, env);
  } else if (key.match(/\.(mp4|mov|avi)$/)) {
    await processVideo(key, env);
  } else if (key.match(/\.(pdf|doc|docx)$/)) {
    await processDocument(key, env);
  } else {
    console.log(`Ignoring file: ${key}`);
  }
}

---

Performance Considerations

Batch Processing

Process events in batches for efficiency:

export default {
  async queue(batch: MessageBatch<R2Event>, env: Env): Promise<void> {
    // Group events by type
    const createEvents: R2ObjectCreatedEvent[] = [];
    const deleteEvents: R2ObjectDeletedEvent[] = [];

    for (const message of batch.messages) {
      if (message.body.action === 'PutObject') {
        createEvents.push(message.body as R2ObjectCreatedEvent);
      } else {
        deleteEvents.push(message.body as R2ObjectDeletedEvent);
      }
    }

    // Process batches
    if (createEvents.length > 0) {
      await processBatchCreated(createEvents, env);
    }

    if (deleteEvents.length > 0) {
      await processBatchDeleted(deleteEvents, env);
    }

    // Acknowledge all messages
    batch.messages.forEach(m => m.ack());
  }
};

async function processBatchCreated(events: R2ObjectCreatedEvent[], env: Env) {
  // Process multiple events in parallel
  await Promise.all(
    events.map(event => handleObjectCreated(event, env))
  );
}

Async Processing

Use Durable Objects for long-running tasks:

async function handleLargeFileUpload(event: R2ObjectCreatedEvent, env: Env) {
  // For large files, use Durable Object for processing
  const id = env.FILE_PROCESSOR.idFromName(event.object.key);
  const processor = env.FILE_PROCESSOR.get(id);

  await processor.fetch(new Request('https://dummy/process', {
    method: 'POST',
    body: JSON.stringify(event),
  }));

  console.log(`Async processing started for: ${event.object.key}`);
}

---

Monitoring and Debugging

Logging Best Practices

async function handleR2Event(event: R2Event, env: Env) {
  const startTime = Date.now();

  console.log('Event received:', {
    action: event.action,
    key: event.object.key,
    size: event.object.size,
    timestamp: event.eventTime,
  });

  try {
    await processEvent(event, env);

    const duration = Date.now() - startTime;
    console.log('Event processed successfully:', {
      key: event.object.key,
      duration: `${duration}ms`,
    });
  } catch (error) {
    console.error('Event processing failed:', {
      key: event.object.key,
      error: error.message,
      stack: error.stack,
    });
    throw error;
  }
}

Testing Events Locally

Trigger test events manually:

// Development endpoint to simulate events
app.post('/test/event', async (c) => {
  const testEvent: R2ObjectCreatedEvent = {
    account: 'test-account',
    action: 'PutObject',
    bucket: 'test-bucket',
    object: {
      key: 'test/image.jpg',
      size: 1024,
      eTag: 'test-etag',
    },
    eventTime: new Date().toISOString(),
  };

  await handleR2Event(testEvent, c.env);

  return c.json({ success: true, message: 'Test event processed' });
});

---

Security Best Practices

1. Validate event payloads - Verify event structure before processing 2. Use dead letter queues - Catch failed events for investigation 3. Implement idempotency - Handle duplicate events safely 4. Set reasonable timeouts - Prevent infinite processing loops 5. Monitor queue depth - Alert on backlog buildup 6. Use environment variables - Keep sensitive config out of code 7. Limit batch sizes - Prevent memory exhaustion 8. Add authentication - Secure webhook endpoints

---

Troubleshooting

Events Not Triggering

Check:

• Event notification rule configured correctly in wrangler.jsonc or Dashboard
• Queue exists and is bound to Worker
• Prefix/suffix filters match your objects
• Worker is deployed and queue consumer is active

Test:

# Check queue status
bunx wrangler queues list

# View queue consumer
bunx wrangler queues consumer list r2-events

Events Stuck in Queue

Check:

• Worker queue handler is processing messages
• No infinite retry loops (check logs)
• Dead letter queue configured for failed messages
• Batch size and timeout settings appropriate

Monitor:

# View queue metrics in Dashboard
# R2 → Queues → [your-queue] → Metrics

Duplicate Event Processing

Solution: Implement idempotency using event.object.eTag:

// Track processed ETags to prevent duplicates
const processedEvents = new Set<string>();

async function handleR2Event(event: R2Event, env: Env) {
  const eventId = `${event.object.key}-${event.object.eTag}`;

  if (processedEvents.has(eventId)) {
    console.log(`Duplicate event ignored: ${eventId}`);
    return;
  }

  processedEvents.add(eventId);
  await processEvent(event, env);
}

---

Official Documentation

• Event Notifications: https://developers.cloudflare.com/r2/buckets/event-notifications/
• Queues: https://developers.cloudflare.com/queues/
• Queue Consumers: https://developers.cloudflare.com/queues/configuration/consumer-concurrency/
• Dead Letter Queues: https://developers.cloudflare.com/queues/configuration/dead-letter-queues/

---

Ready to automate R2 workflows with event-driven architecture!

R2 Performance Optimization

Last Updated: 2025-12-27

Optimize R2 performance with caching strategies, compression, range requests, ETags, and monitoring best practices.

---

Overview

R2 provides excellent baseline performance, but following optimization patterns can significantly improve response times, reduce bandwidth costs, and enhance user experience.

Key Optimization Areas:

• Caching (browser, CDN, Workers)
• Compression and content encoding
• Range requests for large files
• ETags and conditional requests
• Connection pooling and reuse
• Bandwidth optimization
• Monitoring and metrics

---

Caching Strategies

Browser Caching

Set appropriate Cache-Control headers for client-side caching:

// Immutable assets (hashed filenames)
await env.MY_BUCKET.put('assets/app-abc123.js', data, {
  httpMetadata: {
    contentType: 'application/javascript',
    cacheControl: 'public, max-age=31536000, immutable',  // 1 year
  },
});

// Frequently changing content
await env.MY_BUCKET.put('api/data.json', data, {
  httpMetadata: {
    contentType: 'application/json',
    cacheControl: 'public, max-age=300, must-revalidate',  // 5 minutes
  },
});

// Private user data
await env.MY_BUCKET.put('user/profile.jpg', data, {
  httpMetadata: {
    contentType: 'image/jpeg',
    cacheControl: 'private, max-age=3600',  // 1 hour, browser only
  },
});

// No caching (sensitive or dynamic)
await env.MY_BUCKET.put('sensitive/document.pdf', data, {
  httpMetadata: {
    contentType: 'application/pdf',
    cacheControl: 'no-store, no-cache, must-revalidate',
  },
});

Cache-Control Directives:

• public - Can be cached by browsers and CDNs
• private - Only browser caching (not CDNs)
• max-age=N - Cache for N seconds
• immutable - Content never changes (perfect for hashed files)
• must-revalidate - Check with server before using stale cache
• no-store - Never cache
• no-cache - Cache but always validate

Cloudflare CDN Caching

R2 integrates seamlessly with Cloudflare's CDN:

import { Hono } from 'hono';

type Bindings = {
  ASSETS: R2Bucket;
};

const app = new Hono<{ Bindings: Bindings }>();

// Serve assets with CDN caching
app.get('/assets/*', async (c) => {
  const key = c.req.param('*');
  const object = await c.env.ASSETS.get(key);

  if (!object) {
    return c.json({ error: 'Not found' }, 404);
  }

  const headers = new Headers();
  object.writeHttpMetadata(headers);

  // Override cache headers for CDN
  headers.set('Cache-Control', 'public, max-age=31536000, immutable');
  headers.set('CDN-Cache-Control', 'public, max-age=86400');  // CDN caches for 24 hours

  return new Response(object.body, { headers });
});

export default app;

CDN-Cache-Control Header:

// Different caching for browser vs CDN
headers.set('Cache-Control', 'public, max-age=3600');          // Browser: 1 hour
headers.set('CDN-Cache-Control', 'public, max-age=86400');    // CDN: 24 hours

Workers KV Caching Layer

Use KV as a caching layer for frequently accessed small objects:

type Bindings = {
  MY_BUCKET: R2Bucket;
  CACHE: KVNamespace;
};

const app = new Hono<{ Bindings: Bindings }>();

app.get('/cached/:filename', async (c) => {
  const filename = c.req.param('filename');
  const cacheKey = `r2:${filename}`;

  // Check KV cache first
  const cached = await c.env.CACHE.get(cacheKey, { type: 'arrayBuffer' });

  if (cached) {
    console.log('Cache HIT:', filename);
    return new Response(cached, {
      headers: {
        'X-Cache': 'HIT',
        'Content-Type': 'application/octet-stream',
      },
    });
  }

  console.log('Cache MISS:', filename);

  // Fetch from R2
  const object = await c.env.MY_BUCKET.get(filename);

  if (!object) {
    return c.json({ error: 'Not found' }, 404);
  }

  const data = await object.arrayBuffer();

  // Store in KV cache (if < 25MB)
  if (data.byteLength < 25 * 1024 * 1024) {
    await c.env.CACHE.put(cacheKey, data, {
      expirationTtl: 3600,  // Cache for 1 hour
    });
  }

  return new Response(data, {
    headers: {
      'X-Cache': 'MISS',
      'Content-Type': object.httpMetadata?.contentType || 'application/octet-stream',
    },
  });
});

---

Compression and Content Encoding

Gzip/Brotli Compression

Compress files before uploading to R2:

import { gzip, brotli } from 'zlib';
import { promisify } from 'util';

const gzipAsync = promisify(gzip);
const brotliAsync = promisify(brotli);

// Upload with gzip compression
async function uploadCompressed(key: string, data: Buffer, env: Bindings) {
  const compressed = await gzipAsync(data);

  await env.MY_BUCKET.put(key, compressed, {
    httpMetadata: {
      contentType: 'application/javascript',
      contentEncoding: 'gzip',
    },
  });

  console.log(
    `Compression ratio: ${((compressed.length / data.length) * 100).toFixed(1)}%`
  );
}

// Upload with Brotli (better compression)
async function uploadBrotliCompressed(key: string, data: Buffer, env: Bindings) {
  const compressed = await brotliAsync(data);

  await env.MY_BUCKET.put(key, compressed, {
    httpMetadata: {
      contentType: 'text/html',
      contentEncoding: 'br',  // Brotli
    },
  });

  console.log(
    `Brotli compression: ${data.length} → ${compressed.length} bytes`
  );
}

Compression Tips:

• Gzip: Good for most text files (HTML, CSS, JS, JSON)
• Brotli: Better compression but slower (use for static assets)
• Don't compress images/videos (already compressed)
• Pre-compress at build time, not on upload

Automatic Compression in Workers

app.get('/assets/:filename', async (c) => {
  const filename = c.req.param('filename');
  const acceptEncoding = c.req.header('Accept-Encoding') || '';

  // Determine best compression
  const supportsBrotli = acceptEncoding.includes('br');
  const supportsGzip = acceptEncoding.includes('gzip');

  let key = filename;
  let contentEncoding = '';

  if (supportsBrotli) {
    key = `${filename}.br`;
    contentEncoding = 'br';
  } else if (supportsGzip) {
    key = `${filename}.gz`;
    contentEncoding = 'gzip';
  }

  // Try compressed version first
  let object = await c.env.ASSETS.get(key);

  if (!object) {
    // Fallback to uncompressed
    object = await c.env.ASSETS.get(filename);
    contentEncoding = '';
  }

  if (!object) {
    return c.json({ error: 'Not found' }, 404);
  }

  return new Response(object.body, {
    headers: {
      'Content-Type': object.httpMetadata?.contentType || 'text/plain',
      'Content-Encoding': contentEncoding,
      'Vary': 'Accept-Encoding',
    },
  });
});

---

Range Requests

Support partial content delivery for large files (videos, downloads):

app.get('/video/:filename', async (c) => {
  const filename = c.req.param('filename');
  const rangeHeader = c.req.header('Range');

  const object = await c.env.VIDEOS.get(filename);

  if (!object) {
    return c.json({ error: 'Not found' }, 404);
  }

  // No range request - return entire file
  if (!rangeHeader) {
    return new Response(object.body, {
      headers: {
        'Content-Type': object.httpMetadata?.contentType || 'video/mp4',
        'Content-Length': object.size.toString(),
        'Accept-Ranges': 'bytes',
      },
    });
  }

  // Parse range header: "bytes=0-1023"
  const match = rangeHeader.match(/bytes=(\d+)-(\d*)/);

  if (!match) {
    return c.json({ error: 'Invalid range header' }, 400);
  }

  const start = parseInt(match[1]);
  const end = match[2] ? parseInt(match[2]) : object.size - 1;

  // Validate range
  if (start >= object.size || end >= object.size || start > end) {
    return new Response(null, {
      status: 416,
      headers: {
        'Content-Range': `bytes */${object.size}`,
      },
    });
  }

  // Fetch range from R2
  const rangeObject = await c.env.VIDEOS.get(filename, {
    range: { offset: start, length: end - start + 1 },
  });

  if (!rangeObject) {
    return c.json({ error: 'Range not satisfiable' }, 416);
  }

  return new Response(rangeObject.body, {
    status: 206,
    headers: {
      'Content-Type': object.httpMetadata?.contentType || 'video/mp4',
      'Content-Length': (end - start + 1).toString(),
      'Content-Range': `bytes ${start}-${end}/${object.size}`,
      'Accept-Ranges': 'bytes',
    },
  });
});

Benefits of Range Requests:

• Video streaming (start playback before full download)
• Resume interrupted downloads
• Parallel chunk downloads
• Reduce bandwidth for partial reads

---

ETags and Conditional Requests

Use ETags to avoid transferring unchanged data:

app.get('/files/:filename', async (c) => {
  const filename = c.req.param('filename');
  const ifNoneMatch = c.req.header('If-None-Match');

  const object = await c.env.MY_BUCKET.get(filename);

  if (!object) {
    return c.json({ error: 'Not found' }, 404);
  }

  // Check if client's cached version matches
  if (ifNoneMatch === object.httpEtag) {
    return new Response(null, {
      status: 304,  // Not Modified
      headers: {
        'ETag': object.httpEtag,
        'Cache-Control': 'public, max-age=3600',
      },
    });
  }

  return new Response(object.body, {
    headers: {
      'Content-Type': object.httpMetadata?.contentType || 'application/octet-stream',
      'ETag': object.httpEtag,
      'Cache-Control': 'public, max-age=3600',
    },
  });
});

Conditional Uploads (prevent race conditions):

// Update only if ETag matches (optimistic locking)
app.put('/files/:filename', async (c) => {
  const filename = c.req.param('filename');
  const ifMatch = c.req.header('If-Match');

  const data = await c.req.arrayBuffer();

  // Use conditional put to prevent overwrites
  try {
    await c.env.MY_BUCKET.put(filename, data, {
      httpMetadata: {
        contentType: c.req.header('content-type') || 'application/octet-stream',
      },
      onlyIf: {
        etagMatches: ifMatch,  // Only write if ETag matches
      },
    });

    return c.json({ success: true });
  } catch (error: any) {
    if (error.message.includes('precondition')) {
      return c.json({
        error: 'File was modified by another process',
      }, 412);
    }
    throw error;
  }
});

---

Bandwidth Optimization

Lazy Loading and Pagination

// List objects with pagination
app.get('/files', async (c) => {
  const cursor = c.req.query('cursor');
  const limit = parseInt(c.req.query('limit') || '100');

  const listed = await c.env.MY_BUCKET.list({
    limit: Math.min(limit, 1000),  // Cap at 1000
    cursor: cursor || undefined,
  });

  return c.json({
    files: listed.objects.map(obj => ({
      key: obj.key,
      size: obj.size,
      uploaded: obj.uploaded,
    })),
    hasMore: listed.truncated,
    cursor: listed.cursor,
  });
});

Head Requests for Metadata

Use head() to check existence without downloading:

// Check if file exists before download
app.get('/check/:filename', async (c) => {
  const filename = c.req.param('filename');

  const object = await c.env.MY_BUCKET.head(filename);

  if (!object) {
    return c.json({ exists: false }, 404);
  }

  return c.json({
    exists: true,
    size: object.size,
    contentType: object.httpMetadata?.contentType,
    etag: object.httpEtag,
    uploaded: object.uploaded,
  });
});

---

Monitoring and Metrics

Performance Logging

app.get('/download/:filename', async (c) => {
  const startTime = Date.now();
  const filename = c.req.param('filename');

  try {
    const object = await c.env.MY_BUCKET.get(filename);

    if (!object) {
      return c.json({ error: 'Not found' }, 404);
    }

    const downloadTime = Date.now() - startTime;

    // Log performance metrics
    console.log(JSON.stringify({
      event: 'r2_download',
      filename,
      size: object.size,
      duration_ms: downloadTime,
      throughput_mbps: (object.size / downloadTime / 1000 * 8).toFixed(2),
    }));

    return new Response(object.body, {
      headers: {
        'X-Response-Time': `${downloadTime}ms`,
      },
    });
  } catch (error) {
    const errorTime = Date.now() - startTime;

    console.error(JSON.stringify({
      event: 'r2_download_error',
      filename,
      duration_ms: errorTime,
      error: error.message,
    }));

    throw error;
  }
});

Analytics with Workers Analytics Engine

type Bindings = {
  MY_BUCKET: R2Bucket;
  ANALYTICS: AnalyticsEngineDataset;
};

app.get('/files/:filename', async (c) => {
  const startTime = Date.now();
  const filename = c.req.param('filename');

  const object = await c.env.MY_BUCKET.get(filename);

  if (!object) {
    return c.json({ error: 'Not found' }, 404);
  }

  const duration = Date.now() - startTime;

  // Write to Analytics Engine
  c.env.ANALYTICS.writeDataPoint({
    blobs: [filename],
    doubles: [object.size, duration],
    indexes: ['r2_access'],
  });

  return new Response(object.body);
});

---

Performance Best Practices

1. Use Correct Content-Type

// Always set correct content-type
await env.BUCKET.put(key, data, {
  httpMetadata: {
    contentType: 'image/jpeg',  // Not 'application/octet-stream'
  },
});

2. Compress Text Files

// Compress JS/CSS/HTML at build time
const compressed = await gzipAsync(data);
await env.BUCKET.put('app.js', compressed, {
  httpMetadata: {
    contentType: 'application/javascript',
    contentEncoding: 'gzip',
  },
});

3. Set Aggressive Caching for Static Assets

// Hash-based filenames enable long caching
await env.BUCKET.put('app-abc123.js', data, {
  httpMetadata: {
    cacheControl: 'public, max-age=31536000, immutable',
  },
});

4. Use CDN for Global Distribution

// Cloudflare CDN automatically caches R2 objects
// Set appropriate Cache-Control headers

5. Batch Operations

// Good: Batch delete
await env.BUCKET.delete(['file1.txt', 'file2.txt', 'file3.txt']);

// Bad: Loop delete
for (const file of files) {
  await env.BUCKET.delete(file);  // Slow!
}

6. Use Metadata for Filtering

// Store metadata for efficient filtering
await env.BUCKET.put(key, data, {
  customMetadata: {
    category: 'images',
    public: 'true',
  },
});

// Filter in application
const listed = await env.BUCKET.list({ prefix: 'images/' });
const publicImages = listed.objects.filter(
  obj => obj.customMetadata?.public === 'true'
);

7. Monitor and Alert

// Set up alerts for slow requests
if (duration > 1000) {
  console.error(`Slow R2 request: ${filename} took ${duration}ms`);
  // Send alert to monitoring system
}

---

Troubleshooting Performance Issues

Slow Downloads

Problem: Downloads taking longer than expected

Solutions:

• Check object size - large files naturally take longer
• Verify compression is enabled for text files
• Ensure CDN caching is configured
• Check network latency to Cloudflare edge
• Use range requests for large files

High Bandwidth Costs

Problem: Unexpectedly high bandwidth usage

Solutions:

• Enable compression (can reduce by 70%+)
• Set aggressive caching headers
• Use CDN to reduce origin requests
• Implement head() checks before downloads
• Lazy load images and files

Cache Misses

Problem: Low cache hit ratio

Solutions:

• Increase cache TTL (max-age)
• Use consistent URLs (avoid query parameters)
• Set Vary header correctly
• Check CDN purge frequency
• Monitor cache hit ratio with analytics

---

Official Documentation

• R2 Performance: https://developers.cloudflare.com/r2/performance/
• Caching: https://developers.cloudflare.com/cache/
• Range Requests: https://developers.cloudflare.com/r2/api/workers/workers-api-reference/#ranged-reads
• ETags: https://developers.cloudflare.com/r2/api/workers/workers-api-reference/#conditional-operations
• Analytics Engine: https://developers.cloudflare.com/analytics/analytics-engine/

---

Optimize R2 performance for faster, cheaper, better user experience!

R2 SQL Integration

Last Updated: 2025-12-27 Status: NEW FEATURE (2025)

Query data stored in R2 using distributed SQL without extraction or ETL pipelines. Analyze structured data directly in object storage.

---

Overview

R2 SQL is a distributed SQL engine that allows querying data stored in R2 buckets without moving it. Query CSV, JSON, Parquet, and other structured formats directly from R2 using standard SQL syntax.

Key Benefits:

• No ETL required - Query data in place without extraction
• Cost-effective - No data egress fees, pay only for compute
• Scalable - Distributed execution across Cloudflare's network
• Standard SQL - Use familiar SQL syntax and tools
• Real-time analytics - Query fresh data instantly

Use Cases:

• Log analysis and monitoring
• Business intelligence and reporting
• Data exploration and ad-hoc queries
• Analytics on large datasets
• Serverless data warehousing

---

Supported Data Formats

R2 SQL can query these file formats:

Recommended: Parquet for best performance and compression.

---

Getting Started

1. Store Queryable Data in R2

import { Hono } from 'hono';

type Bindings = {
  DATA_BUCKET: R2Bucket;
};

const app = new Hono<{ Bindings: Bindings }>();

// Upload CSV data
app.post('/upload/logs', async (c) => {
  const csvData = `timestamp,level,message,user_id
2025-01-15T10:00:00Z,INFO,"User login",123
2025-01-15T10:01:00Z,ERROR,"Failed payment",456
2025-01-15T10:02:00Z,INFO,"User logout",123`;

  await c.env.DATA_BUCKET.put('logs/2025-01-15.csv', csvData, {
    httpMetadata: {
      contentType: 'text/csv',
    },
    customMetadata: {
      format: 'csv',
      schema: 'timestamp,level,message,user_id',
    },
  });

  return c.json({ success: true });
});

// Upload Parquet data (binary format)
app.post('/upload/analytics', async (c) => {
  const parquetData = await c.req.arrayBuffer(); // From analytics pipeline

  await c.env.DATA_BUCKET.put('analytics/2025-01-15.parquet', parquetData, {
    httpMetadata: {
      contentType: 'application/octet-stream',
    },
    customMetadata: {
      format: 'parquet',
      partitionDate: '2025-01-15',
    },
  });

  return c.json({ success: true });
});

export default app;

2. Query Data with R2 SQL

SQL Query Syntax:

-- Query CSV files
SELECT
  timestamp,
  level,
  message,
  user_id
FROM r2('my-bucket/logs/*.csv')
WHERE level = 'ERROR'
  AND timestamp >= '2025-01-15'
ORDER BY timestamp DESC
LIMIT 100;

-- Query Parquet files
SELECT
  date,
  COUNT(*) as events,
  SUM(revenue) as total_revenue,
  AVG(session_duration) as avg_duration
FROM r2('my-bucket/analytics/*.parquet')
WHERE date BETWEEN '2025-01-01' AND '2025-01-31'
GROUP BY date
ORDER BY date;

-- Join multiple datasets
SELECT
  u.user_id,
  u.username,
  COUNT(e.event_id) as event_count,
  MAX(e.timestamp) as last_active
FROM r2('my-bucket/users/*.csv') u
LEFT JOIN r2('my-bucket/events/*.parquet') e
  ON u.user_id = e.user_id
WHERE e.timestamp >= '2025-01-01'
GROUP BY u.user_id, u.username
ORDER BY event_count DESC;

3. Execute Queries from Workers

type Bindings = {
  DATA_BUCKET: R2Bucket;
  R2_SQL: R2SQLEngine; // SQL engine binding
};

const app = new Hono<{ Bindings: Bindings }>();

// Execute SQL query
app.post('/query', async (c) => {
  const { sql, params } = await c.req.json();

  try {
    const result = await c.env.R2_SQL.execute(sql, params);

    return c.json({
      success: true,
      rows: result.rows,
      rowCount: result.rowCount,
      columns: result.columns,
    });
  } catch (error: any) {
    return c.json({
      success: false,
      error: error.message,
    }, 400);
  }
});

// Predefined analytics query
app.get('/analytics/daily-summary', async (c) => {
  const date = c.req.query('date') || new Date().toISOString().split('T')[0];

  const sql = `
    SELECT
      level,
      COUNT(*) as count,
      COUNT(DISTINCT user_id) as unique_users
    FROM r2('data-bucket/logs/${date}*.csv')
    GROUP BY level
    ORDER BY count DESC
  `;

  const result = await c.env.R2_SQL.execute(sql);

  return c.json({
    date,
    summary: result.rows,
  });
});

export default app;

---

SQL Syntax and Functions

Supported SQL Operations

SELECT Statements:

SELECT column1, column2, aggregate_func(column3)
FROM r2('bucket/path/*.format')
WHERE condition
GROUP BY column1, column2
HAVING aggregate_condition
ORDER BY column1 DESC
LIMIT 1000;

Aggregate Functions:

• COUNT(*), COUNT(DISTINCT column)
• SUM(column), AVG(column)
• MIN(column), MAX(column)
• STDDEV(column), VARIANCE(column)

String Functions:

• CONCAT(str1, str2)
• SUBSTRING(str, start, length)
• UPPER(str), LOWER(str)
• TRIM(str), LTRIM(str), RTRIM(str)
• REGEXP_MATCHES(str, pattern)

Date/Time Functions:

• DATE(timestamp)
• EXTRACT(field FROM timestamp)
• DATE_TRUNC(precision, timestamp)
• NOW(), CURRENT_DATE, CURRENT_TIME

Conditional Logic:

SELECT
  CASE
    WHEN level = 'ERROR' THEN 'Critical'
    WHEN level = 'WARN' THEN 'Important'
    ELSE 'Normal'
  END as priority,
  COUNT(*) as count
FROM r2('bucket/logs/*.csv')
GROUP BY priority;

---

Performance Optimization

Partitioning Data

Organize data by date/category for faster queries:

data-bucket/
├── logs/
│   ├── 2025-01-15/
│   │   ├── application.csv
│   │   └── api.csv
│   ├── 2025-01-16/
│   │   ├── application.csv
│   │   └── api.csv
└── analytics/
    ├── 2025-01/
    │   ├── users.parquet
    │   └── events.parquet

Query specific partitions:

-- Query only January 15 data
SELECT *
FROM r2('data-bucket/logs/2025-01-15/*.csv')
WHERE level = 'ERROR';

-- Query entire month (slower, scans all files)
SELECT *
FROM r2('data-bucket/logs/2025-01-*/*.csv')
WHERE level = 'ERROR';

Use Parquet for Large Datasets

Convert CSV to Parquet for 10x better performance:

// Example: Convert CSV logs to Parquet daily
import parquet from 'parquetjs';

async function convertCSVToParquet(csvKey: string, env: Bindings) {
  // Fetch CSV data
  const csvObject = await env.DATA_BUCKET.get(csvKey);
  const csvText = await csvObject?.text();

  if (!csvText) return;

  // Parse CSV
  const rows = parseCSV(csvText);

  // Define Parquet schema
  const schema = new parquet.ParquetSchema({
    timestamp: { type: 'TIMESTAMP_MILLIS' },
    level: { type: 'UTF8' },
    message: { type: 'UTF8' },
    user_id: { type: 'INT64' },
  });

  // Write Parquet file
  const writer = await parquet.ParquetWriter.openStream(schema);

  for (const row of rows) {
    await writer.appendRow(row);
  }

  await writer.close();

  // Upload Parquet to R2
  const parquetKey = csvKey.replace('.csv', '.parquet');
  await env.DATA_BUCKET.put(parquetKey, writer.outputStream, {
    httpMetadata: {
      contentType: 'application/octet-stream',
    },
  });

  console.log(`Converted ${csvKey} to ${parquetKey}`);
}

Columnar Projection

Select only needed columns for faster queries:

-- Good: Select specific columns
SELECT timestamp, user_id, revenue
FROM r2('bucket/analytics/*.parquet')
WHERE date = '2025-01-15';

-- Bad: Select all columns (slower)
SELECT *
FROM r2('bucket/analytics/*.parquet')
WHERE date = '2025-01-15';

Predicate Pushdown

Filter early for better performance:

-- Good: Filter in WHERE clause
SELECT COUNT(*)
FROM r2('bucket/logs/*.csv')
WHERE timestamp >= '2025-01-15'
  AND level = 'ERROR';

-- Bad: Filter in application code (processes all rows)
SELECT *
FROM r2('bucket/logs/*.csv');
-- Then filter in application

---

Common Patterns

Pattern 1: Daily Log Analysis

-- Analyze application errors by hour
SELECT
  DATE_TRUNC('hour', timestamp) as hour,
  level,
  COUNT(*) as error_count,
  COUNT(DISTINCT user_id) as affected_users
FROM r2('logs-bucket/app-logs/2025-01-15/*.csv')
WHERE level IN ('ERROR', 'FATAL')
GROUP BY hour, level
ORDER BY hour DESC, error_count DESC;

Pattern 2: User Activity Dashboard

-- User engagement metrics
SELECT
  date,
  COUNT(DISTINCT user_id) as daily_active_users,
  COUNT(*) as total_events,
  SUM(CASE WHEN event_type = 'purchase' THEN 1 ELSE 0 END) as purchases,
  SUM(CASE WHEN event_type = 'purchase' THEN amount ELSE 0 END) as revenue
FROM r2('analytics-bucket/events/*.parquet')
WHERE date BETWEEN '2025-01-01' AND '2025-01-31'
GROUP BY date
ORDER BY date;

Pattern 3: Funnel Analysis

-- Conversion funnel from signup to purchase
WITH funnel AS (
  SELECT
    user_id,
    MAX(CASE WHEN event_type = 'signup' THEN 1 ELSE 0 END) as signed_up,
    MAX(CASE WHEN event_type = 'add_to_cart' THEN 1 ELSE 0 END) as added_cart,
    MAX(CASE WHEN event_type = 'checkout' THEN 1 ELSE 0 END) as checked_out,
    MAX(CASE WHEN event_type = 'purchase' THEN 1 ELSE 0 END) as purchased
  FROM r2('analytics-bucket/events/*.parquet')
  WHERE date >= '2025-01-01'
  GROUP BY user_id
)
SELECT
  SUM(signed_up) as signups,
  SUM(added_cart) as cart_adds,
  SUM(checked_out) as checkouts,
  SUM(purchased) as purchases,
  ROUND(100.0 * SUM(added_cart) / SUM(signed_up), 2) as signup_to_cart_pct,
  ROUND(100.0 * SUM(checked_out) / SUM(added_cart), 2) as cart_to_checkout_pct,
  ROUND(100.0 * SUM(purchased) / SUM(checked_out), 2) as checkout_to_purchase_pct
FROM funnel;

Pattern 4: Time-Series Analysis

-- 7-day moving average of revenue
WITH daily_revenue AS (
  SELECT
    date,
    SUM(amount) as revenue
  FROM r2('sales-bucket/transactions/*.parquet')
  WHERE date >= DATE_SUB(CURRENT_DATE, INTERVAL 30 DAY)
  GROUP BY date
)
SELECT
  date,
  revenue,
  AVG(revenue) OVER (
    ORDER BY date
    ROWS BETWEEN 6 PRECEDING AND CURRENT ROW
  ) as moving_avg_7d
FROM daily_revenue
ORDER BY date;

---

Integration with Analytics Tools

Grafana Dashboard

// Grafana data source endpoint
app.post('/grafana/query', async (c) => {
  const { targets, range } = await c.req.json();

  const results = [];

  for (const target of targets) {
    const sql = buildSQLFromTarget(target, range);
    const result = await c.env.R2_SQL.execute(sql);

    results.push({
      target: target.target,
      datapoints: result.rows.map(row => [row.value, row.timestamp]),
    });
  }

  return c.json(results);
});

function buildSQLFromTarget(target: any, range: any) {
  return `
    SELECT
      timestamp,
      ${target.metric} as value
    FROM r2('${target.bucket}/${target.path}')
    WHERE timestamp BETWEEN '${range.from}' AND '${range.to}'
    ORDER BY timestamp
  `;
}

Jupyter Notebooks (Python)

import requests
import pandas as pd

def query_r2_sql(sql):
    response = requests.post(
        'https://my-worker.example.workers.dev/query',
        json={'sql': sql}
    )
    data = response.json()
    return pd.DataFrame(data['rows'])

# Query and analyze in Pandas
df = query_r2_sql("""
    SELECT date, revenue, users
    FROM r2('analytics-bucket/daily-metrics/*.parquet')
    WHERE date >= '2025-01-01'
    ORDER BY date
""")

# Pandas analysis
print(df.describe())
print(df.corr())

# Plotting
import matplotlib.pyplot as plt
df.plot(x='date', y='revenue')
plt.show()

---

Limitations and Constraints

Query Limits

• Maximum query time: 30 seconds
• Maximum result size: 10 MB
• Maximum row count: 100,000 rows per query
• File size limit: No limit, but larger files take longer to scan

Unsupported Features

• Mutations: INSERT, UPDATE, DELETE not supported (read-only)
• Transactions: No transaction support
• User-defined functions: Custom SQL functions not available
• Stored procedures: Not supported
• Views: Cannot create persistent views (use CTEs)

Data Format Requirements

• CSV: Must have header row
• JSON: Must be valid JSON or NDJSON
• Parquet: Must be valid Parquet format
• Compression: Gzip and Snappy supported, others may not work

---

Cost Optimization

Query Cost Factors

1. Data scanned: Pay per GB scanned 2. Query complexity: Complex joins cost more 3. Result size: Large result sets cost more 4. Compression: Compressed files reduce scan costs

Optimization Strategies

// 1. Query only needed date ranges
const sql = `
  SELECT * FROM r2('logs/2025-01-15/*.csv')  -- Good: specific date
  -- Not: r2('logs/*/*.csv')  -- Bad: scans everything
`;

// 2. Use columnar formats (Parquet)
// Parquet scans only queried columns vs CSV scans entire file

// 3. Partition data by date
// Store: logs/YYYY-MM-DD/app.parquet
// Not: logs/app-YYYY-MM-DD.parquet

// 4. Compress files
await env.BUCKET.put('data.parquet.gz', compressedData, {
  httpMetadata: {
    contentEncoding: 'gzip',
  },
});

// 5. Cache frequent queries
const cacheKey = `query-result:${hashSQL(sql)}`;
const cached = await env.CACHE.get(cacheKey);

if (cached) {
  return c.json(JSON.parse(cached));
}

const result = await c.env.R2_SQL.execute(sql);
await env.CACHE.put(cacheKey, JSON.stringify(result), { expirationTtl: 3600 });

---

Troubleshooting

Query Timeout Errors

Error: "Query exceeded maximum execution time (30s)"

Solutions:

• Add date filters to reduce data scanned
• Use Parquet instead of CSV
• Partition data by date
• Limit result size with LIMIT clause

Schema Mismatch Errors

Error: "Column 'user_id' not found"

Solutions:

• Check CSV header row matches query
• Verify Parquet schema with parquet-tools
• Ensure all files have consistent schema

Out of Memory Errors

Error: "Query exceeded memory limit"

Solutions:

• Reduce result size with LIMIT
• Use aggregation instead of returning raw rows
• Query smaller date ranges
• Split large queries into smaller chunks

---

Official Documentation

• R2 SQL Overview: https://developers.cloudflare.com/r2/sql/
• SQL Syntax Reference: https://developers.cloudflare.com/r2/sql/syntax/
• Supported Functions: https://developers.cloudflare.com/r2/sql/functions/
• Performance Guide: https://developers.cloudflare.com/r2/sql/performance/

---

Query your R2 data with SQL - no ETL required!

R2 S3 API Compatibility

Last Updated: 2025-10-21 Official Docs: https://developers.cloudflare.com/r2/api/s3/api/

---

Overview

R2 implements a large portion of the Amazon S3 API, allowing you to use existing S3 SDKs and tools.

S3 Endpoint Format:

https://<account_id>.r2.cloudflarestorage.com

---

Supported S3 Operations

Bucket Operations

• ✅ ListBuckets
• ❌ CreateBucket (use Cloudflare Dashboard or Wrangler)
• ❌ DeleteBucket (use Cloudflare Dashboard or Wrangler)

Object Operations

• ✅ GetObject
• ✅ PutObject
• ✅ DeleteObject
• ✅ DeleteObjects (bulk delete, max 1000)
• ✅ HeadObject
• ✅ ListObjectsV2
• ✅ CopyObject
• ✅ UploadPart
• ✅ CreateMultipartUpload
• ✅ CompleteMultipartUpload
• ✅ AbortMultipartUpload
• ✅ ListMultipartUploads
• ✅ ListParts

Presigned URLs

• ✅ GetObject (download)
• ✅ PutObject (upload)
• ✅ UploadPart (multipart)

Not Supported

• ❌ Versioning
• ❌ Object Lock
• ❌ ACLs (use CORS instead)
• ❌ Bucket policies
• ❌ Object tagging (use custom metadata)
• ❌ Server-side encryption config (use SSE-C instead)

---

Using AWS SDK for JavaScript

Installation

bun add @aws-sdk/client-s3  # preferred
# or: npm install @aws-sdk/client-s3
bun add @aws-sdk/s3-request-presigner  # preferred
# or: npm install @aws-sdk/s3-request-presigner

Basic Usage

import { S3Client, PutObjectCommand, GetObjectCommand } from '@aws-sdk/client-s3';

// Create S3 client for R2
const s3Client = new S3Client({
  region: 'auto',
  endpoint: `https://<ACCOUNT_ID>.r2.cloudflarestorage.com`,
  credentials: {
    accessKeyId: '<R2_ACCESS_KEY_ID>',
    secretAccessKey: '<R2_SECRET_ACCESS_KEY>',
  },
});

// Upload object
const uploadParams = {
  Bucket: 'my-bucket',
  Key: 'path/to/file.txt',
  Body: 'Hello, R2!',
  ContentType: 'text/plain',
};

await s3Client.send(new PutObjectCommand(uploadParams));

// Download object
const downloadParams = {
  Bucket: 'my-bucket',
  Key: 'path/to/file.txt',
};

const response = await s3Client.send(new GetObjectCommand(downloadParams));
const text = await response.Body.transformToString();

Presigned URLs with AWS SDK

import { getSignedUrl } from '@aws-sdk/s3-request-presigner';
import { PutObjectCommand, GetObjectCommand } from '@aws-sdk/client-s3';

// Generate presigned upload URL
const uploadCommand = new PutObjectCommand({
  Bucket: 'my-bucket',
  Key: 'uploads/file.jpg',
});

const uploadUrl = await getSignedUrl(s3Client, uploadCommand, {
  expiresIn: 3600, // 1 hour
});

// Generate presigned download URL
const downloadCommand = new GetObjectCommand({
  Bucket: 'my-bucket',
  Key: 'uploads/file.jpg',
});

const downloadUrl = await getSignedUrl(s3Client, downloadCommand, {
  expiresIn: 3600,
});

---

Using aws4fetch (Lightweight Alternative)

Installation

bun add aws4fetch  # preferred
# or: npm install aws4fetch

Usage

import { AwsClient } from 'aws4fetch';

const r2Client = new AwsClient({
  accessKeyId: '<R2_ACCESS_KEY_ID>',
  secretAccessKey: '<R2_SECRET_ACCESS_KEY>',
});

const endpoint = `https://<ACCOUNT_ID>.r2.cloudflarestorage.com`;

// Upload object
await r2Client.fetch(`${endpoint}/my-bucket/file.txt`, {
  method: 'PUT',
  body: 'Hello, R2!',
  headers: {
    'Content-Type': 'text/plain',
  },
});

// Download object
const response = await r2Client.fetch(`${endpoint}/my-bucket/file.txt`);
const text = await response.text();

// Delete object
await r2Client.fetch(`${endpoint}/my-bucket/file.txt`, {
  method: 'DELETE',
});

// List objects
const listResponse = await r2Client.fetch(
  `${endpoint}/my-bucket?list-type=2&max-keys=100`
);
const xml = await listResponse.text();

Presigned URLs with aws4fetch

import { AwsClient } from 'aws4fetch';

const r2Client = new AwsClient({
  accessKeyId: '<R2_ACCESS_KEY_ID>',
  secretAccessKey: '<R2_SECRET_ACCESS_KEY>',
});

const url = new URL(
  `https://<ACCOUNT_ID>.r2.cloudflarestorage.com/my-bucket/file.txt`
);

// Set expiry (in seconds)
url.searchParams.set('X-Amz-Expires', '3600');

// Sign for PUT (upload)
const signedUpload = await r2Client.sign(
  new Request(url, { method: 'PUT' }),
  { aws: { signQuery: true } }
);

console.log(signedUpload.url);

// Sign for GET (download)
const signedDownload = await r2Client.sign(
  new Request(url, { method: 'GET' }),
  { aws: { signQuery: true } }
);

console.log(signedDownload.url);

---

S3 vs R2 Workers API Comparison

---

When to Use S3 API vs Workers API

Use S3 API when:

• ✅ Migrating from AWS S3
• ✅ Using existing S3 tools (aws-cli, s3cmd)
• ✅ Generating presigned URLs
• ✅ Need S3 compatibility for external systems

Use Workers API when:

• ✅ Building new applications on Cloudflare
• ✅ Need better performance (native binding)
• ✅ Don't want to manage access keys
• ✅ Using R2 from Workers

---

R2-Specific Extensions

R2 adds some extensions to the S3 API:

Conditional Operations

// Only upload if file doesn't exist
await s3Client.send(new PutObjectCommand({
  Bucket: 'my-bucket',
  Key: 'file.txt',
  Body: data,
  IfUnmodifiedSince: new Date('2020-01-01'), // Before R2 existed
}));

Storage Class

R2 currently only supports 'Standard' storage class.

await s3Client.send(new PutObjectCommand({
  Bucket: 'my-bucket',
  Key: 'file.txt',
  Body: data,
  StorageClass: 'STANDARD',
}));

---

Migration from S3

1. Update Endpoint

const s3Client = new S3Client({
  region: 'auto',
- endpoint: 'https://s3.amazonaws.com',
+ endpoint: 'https://<ACCOUNT_ID>.r2.cloudflarestorage.com',
  credentials: {
-   accessKeyId: process.env.AWS_ACCESS_KEY_ID,
-   secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
+   accessKeyId: process.env.R2_ACCESS_KEY_ID,
+   secretAccessKey: process.env.R2_SECRET_ACCESS_KEY,
  },
});

2. Remove Unsupported Features

await s3Client.send(new PutObjectCommand({
  Bucket: 'my-bucket',
  Key: 'file.txt',
  Body: data,
- ACL: 'public-read',  // ❌ Not supported
- Tagging: 'key=value', // ❌ Not supported (use custom metadata)
+ Metadata: {            // ✅ Use custom metadata instead
+   visibility: 'public',
+ },
}));

3. Use CORS Instead of ACLs

R2 doesn't support S3 ACLs. Use CORS policies instead for browser access.

---

Common Issues

Issue: SignatureDoesNotMatch

Cause: Incorrect access keys or endpoint URL

Fix:

• Verify access key ID and secret
• Ensure endpoint includes your account ID
• Check region is set to 'auto'

Issue: Presigned URLs Don't Work with Custom Domains

Cause: Presigned URLs only work with R2 S3 endpoint

Fix:

• Use <ACCOUNT_ID>.r2.cloudflarestorage.com endpoint
• Or use Worker with R2 binding for custom domains

Issue: Upload Size Exceeds Limit

Cause: S3 API PUT has 5GB limit, but R2 Workers has 100-500MB limit

Fix:

• Use multipart upload for large files
• Or use S3 API directly (not through Worker)

---

Official Resources

• S3 API Compatibility: https://developers.cloudflare.com/r2/api/s3/api/
• AWS SDK Examples: https://developers.cloudflare.com/r2/examples/aws/
• Presigned URLs: https://developers.cloudflare.com/r2/api/s3/presigned-urls/