Aws Lambda Managed Instances

AWS Lambda Managed Instances (LMI)

Runs Lambda functions on EC2 instances in the user's account while AWS manages provisioning, patching, scaling, routing, and load balancing. Combines Lambda's developer experience with EC2's pricing and hardware options.

Works best with the AWS MCP server for sandboxed CLI execution and audit logging. All guidance also works with standard AWS CLI or SAM CLI.

Note: Confirm regional availability, quotas, and instance type offerings against current AWS documentation before production deployment.

Quick Decision: Is LMI Right for This Workload?

Routing

Read ONLY the single reference file that matches the user's task. Do not preload multiple references.

Troubleshooting quick facts (always mention when diagnosing issues):

• Capacity provider stuck in CREATING → most common cause is private subnets missing a NAT gateway route (instances need outbound internet for image pull and Lambda service communication)
• Function not scaling → check that a version is published (PublishToLatestPublished: true)
• Memory errors → LMI minimum is 2048 MB

Workflow

Step 1: Assess the Workload

Gather these signals before recommending:

1. Traffic pattern: Steady vs bursty? Requests per second? 2. Current costs: Monthly Lambda spend? Existing Savings Plans? 3. Runtime: Node.js, Java, .NET, or Python? 4. Memory/CPU: How much memory? CPU-bound or I/O-bound? 5. Execution duration: Average and P99? 6. Concurrency readiness: Thread safety? Shared /tmp paths? Per-invocation DB connections? 7. VPC: Already in a VPC? Private resource access needed?

When recommending LMI, ALWAYS mention: minimum 3 execution environments for AZ resiliency (cannot go below 3 in production).

Step 2: Build the Cost Comparison

REQUIRED: Present a cost comparison before recommending LMI.

Rule of thumb: LMI becomes cost-competitive at 50-100M+ req/month with steady traffic. Use the LMI Pricing Calculator for accurate comparisons.

Step 3: Configure the Deployment

• Instance families (400+ types, .large and up): C-series (compute), M-series (general), R-series (memory). ARM (Graviton) for best price-performance.
• When using Graviton instances, MUST set `Architectures: [arm64]` in the function configuration to match.
• Memory-to-vCPU ratios: 2:1 (compute), 4:1 (general, default), 8:1 (memory). Min 2 GB, max 32 GB.
• Multi-concurrency per-vCPU maximums: Node.js 64, Java 32, .NET 32, Python 16. These are system caps — the actual setting is PerExecutionEnvironmentMaxConcurrency (per execution environment, not per vCPU).
• For I/O-bound workloads: use the runtime default or higher PerExecutionEnvironmentMaxConcurrency (e.g., 10 for Node.js) since each request uses minimal CPU while waiting on network.
• For CPU-bound workloads: set PerExecutionEnvironmentMaxConcurrency to 1-2 per vCPU since each request saturates CPU.
• Scaling: MinExecutionEnvironments (default 3), MaxVCpuCount (optional, default 400 — set explicitly as best practice), TargetResourceUtilization.

Step 4: Migrate the Code

Review code for concurrency safety. LMI runs multiple invocations concurrently per execution environment:

• Python: Process-based isolation — globals are NOT shared. No thread-safety changes needed. Focus on /tmp conflicts and memory sizing.
• Node.js: Worker threads — globals shared within a worker. Requires async safety.
• Java/.NET: OS threads/Tasks — handler shared across threads. Requires full thread safety.

Step 5: Set Up Infrastructure

1. Create two IAM roles: execution role (for the function) and operator role (for capacity provider EC2 management) 2. Configure VPC with subnets across 3+ AZs 3. Create capacity provider with VPC config and scaling limits 4. Create or update function with capacity provider attachment 5. Publish a version (triggers instance provisioning)

Step 6: Validate and Cut Over

1. Deploy to a non-production environment first 2. Monitor CloudWatch: CPU utilization, memory, concurrency, throttle rate 3. Gradual traffic shift with weighted aliases (10% → 50% → 100%) 4. Compare costs after 1-2 weeks of production data 5. Decommission standard Lambda once stable

Best Practices

Pricing (always mention when discussing costs)

• Three components: EC2 instance hours + 15% management fee + $0.20/1M requests
• Savings Plans: Compute Savings Plans apply to the EC2 portion (up to 60-72% discount)
• The 15% fee is charged on top of EC2 cost for AWS managing provisioning, patching, scaling, lifecycle

Scaling (always mention when discussing scaling or traffic)

• LMI absorbs a 50% traffic spike immediately and doubles capacity within 5 minutes — if traffic more than doubles faster, requests throttle
• Standard Lambda bursts to 3000 instantly — LMI cannot match this
• Pre-warm with MinExecutionEnvironments before known spikes
• MaxVCpuCount (default 400) — set explicitly as a cost ceiling
• Shape: Reduce MinExecutionEnvironments to lower capacity during off-hours (minimum 3 for AZ resiliency)

Instance Sizing

• 1 vCPU + 1 GB reserved per instance for OS overhead (not available to your function)
• Usable capacity = total - overhead

Configuration

• Start with 4:1 ratio and runtime default concurrency
• Use ARM (Graviton) unless x86 dependencies exist
• Let Lambda choose instance types unless specific hardware needed
• Set MaxVCpuCount to control cost ceiling
• Never set MinExecutionEnvironments below 3 (breaks AZ resiliency)

Migration

• Start with I/O-heavy functions (benefit most from multi-concurrency)
• Review code for concurrency safety before attaching to capacity provider
• Use weighted aliases for gradual traffic shift
• Include request IDs in all log statements
• Initialize DB pools and SDK clients outside the handler

Operations

• Set CloudWatch alarms on throttle rate > 1% and CPU > 80%
• Plan for 14-day instance rotation (automatic)
• Never manually terminate LMI EC2 instances (delete the capacity provider instead)
• Always publish a version — unpublished functions cannot run on LMI

Limits Quick Reference

Security Considerations

• Operator role scoping: Add aws:SourceAccount and aws:SourceArn conditions to trust policies to prevent confused deputy attacks.
• VPC egress: Scope security group egress to VPC endpoint security groups or AWS prefix lists rather than 0.0.0.0/0.
• Credentials: Use AWS Secrets Manager or Parameter Store for database credentials — never environment variables for secrets.
• Encryption: Enable SQS SSE, CloudWatch Logs encryption (KMS), and S3 default encryption for any data at rest.
• Logging: Set CloudWatch Log group retention policies. Avoid logging PII or credentials. Enable CloudTrail data events for Lambda.
• Instance rotation: The 14-day automatic rotation ensures security patches are applied without manual intervention.
• References: Lambda Security Best Practices, IAM Best Practices

Files

LMI Configuration Guide

Instance Type Decision Tree

• CPU-intensive (encoding, ML, compression) → C-series, 2:1 ratio, concurrency=1/vCPU
• Memory-intensive (caching, large datasets) → R-series, 8:1 ratio
• Network-intensive (streaming, data transfer) → Use AllowedInstanceTypes for n-suffix types, 4:1 ratio
• General/balanced (web APIs, microservices) → M-series, 4:1 ratio, default concurrency

Architecture: ARM (Graviton, g-suffix) for price-performance. x86 (i=Intel, a=AMD) when dependencies require it.

Memory-to-vCPU Ratios

Min: 2 GB / 1 vCPU. Max: 32 GB. Memory must align with ratio multiples.

Memory Sizing from Existing Lambda

Concurrency Tuning

Total capacity = MinExecutionEnvironments × PerExecutionEnvironmentMaxConcurrency

Capacity Provider Scaling Controls

Monitoring Thresholds

• CPU > 80%: reduce concurrency or add vCPUs
• CPU < 20%: increase concurrency for better utilization
• Throttle rate (429s) > 1%: increase MinExecutionEnvironments or reduce utilization target
• Memory > 90%: increase memory or reduce concurrency
• ExecutionEnvironmentConcurrency near limit: saturation — reduce concurrency or scale out

CloudWatch Metrics Dimensions

LMI metrics are split across two CloudWatch dimensions:

• Alias (live): Invocations, Errors, Throttles, Duration
• Version ($LATEST or numbered): CPUUtilization, MemoryUtilization, ExecutionEnvironmentConcurrency, ExecutionEnvironmentCount

Create a unified dashboard combining both views to monitor LMI performance effectively.

Lambda vs LMI Cost Comparison

Use the LMI Pricing Calculator for accurate, up-to-date cost comparisons based on your specific workload parameters (region, instance type, request volume, duration).

When building a cost comparison for a user, gather: region, runtime, requests/month, average duration, memory, and architecture (x86 vs ARM). Plug these into the calculator rather than relying on hardcoded estimates.

When LMI is NOT Cheaper

• low number of requests/month (fixed 3-instance cost exceeds Lambda)
• Very short functions (< 100ms duration)
• Highly bursty, unpredictable traffic
• Workloads needing scale-to-zero

Tools

• LMI Pricing Calculator — interactive comparison tool
• AWS Pricing Calculator — general AWS cost estimation

LMI Infrastructure Setup

IAM Roles (Two Required)

1. Execution Role (for the function)

Trust policy:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"Service": "lambda.amazonaws.com"},
    "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "aws:SourceAccount": "<ACCOUNT_ID>"
        }
      }
  }]
}

Minimum permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:*:*:log-group:/aws/lambda/*"
    }
  ]
}

Add VPC permissions only if the function accesses VPC resources:

{
  "Effect": "Allow",
  "Action": [
    "ec2:CreateNetworkInterface",
    "ec2:DescribeNetworkInterfaces",
    "ec2:DeleteNetworkInterface"
  ],
  "Resource": "*",
  "Condition": {
    "StringEquals": {
      "ec2:Vpc": "arn:aws:ec2:<REGION>:<ACCOUNT_ID>:vpc/<VPC_ID>"
    }
  }
}

2. Operator Role (for capacity provider EC2 management)

Trust policy:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"Service": "lambda.amazonaws.com"},
    "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "aws:SourceAccount": "<ACCOUNT_ID>"
        }
      }
  }]
}

Minimum permissions (scoped with conditions). Use the AWS managed policy `AWSLambdaManagedEC2ResourceOperator` or the equivalent:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["ec2:RunInstances", "ec2:CreateTags", "ec2:AttachNetworkInterface"],
      "Resource": [
        "arn:aws:ec2:*:*:instance/*",
        "arn:aws:ec2:*:*:network-interface/*",
        "arn:aws:ec2:*:*:volume/*"
      ],
      "Condition": {
        "StringEquals": {
          "ec2:ManagedResourceOperator": "scaler.lambda.amazonaws.com"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeAvailabilityZones",
        "ec2:DescribeCapacityReservations",
        "ec2:DescribeInstances",
        "ec2:DescribeInstanceStatus",
        "ec2:DescribeInstanceTypeOfferings",
        "ec2:DescribeInstanceTypes",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeSubnets",
        "ec2:DescribeVpcEncryptionControls"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": ["ec2:RunInstances", "ec2:CreateNetworkInterface"],
      "Resource": [
        "arn:aws:ec2:*:*:subnet/*",
        "arn:aws:ec2:*:*:security-group/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": "ec2:RunInstances",
      "Resource": "arn:aws:ec2:*:*:image/*",
      "Condition": {
        "StringEquals": { "ec2:Owner": "amazon" }
      }
    }
  ]
}

The ec2:ManagedResourceOperator condition ensures RunInstances/CreateTags only apply to Lambda-managed instances.

VPC Requirements

LMI runs functions on EC2 instances inside the VPC. These instances need VPC endpoints or NAT to reach AWS services.

• 3+ subnets across different AZs (for default 3-instance fleet)
• Security groups: HTTPS egress (port 443) scoped to VPC endpoint security groups or AWS prefix lists (avoid 0.0.0.0/0); no ingress needed
• Required VPC endpoints:

CLI Workflow

Required Parameters

Manual Steps

# 1. Create capacity provider
aws lambda create-capacity-provider \
  --capacity-provider-name $CP_NAME \
  --vpc-config "SubnetIds=[$SUBNET_IDS],SecurityGroupIds=[$SECURITY_GROUP_ID]" \
  --permissions-config "CapacityProviderOperatorRoleArn=$OPERATOR_ROLE_ARN" \
  --instance-requirements "Architectures=[$ARCHITECTURE]" \
  --capacity-provider-scaling-config "MaxVCpuCount=30"

# 2. Create function
aws lambda create-function --function-name $FUNCTION_NAME --runtime python3.13 \
  --handler app.handler --zip-file fileb://function.zip \
  --role $EXECUTION_ROLE_ARN --architectures $ARCHITECTURE \
  --memory-size 4096 \
  --capacity-provider-config \
    "LambdaManagedInstancesCapacityProviderConfig={CapacityProviderArn=arn:aws:lambda:$AWS_REGION:$ACCOUNT_ID:capacity-provider:$CP_NAME}"

# 3. Publish version (triggers provisioning — takes several minutes)
aws lambda publish-version --function-name $FUNCTION_NAME

# 4. Invoke (must use versioned ARN)
aws lambda invoke --function-name $FUNCTION_NAME:1 --payload '{}' response.json

Architecture must match between function and capacity provider.

SAM Template

Resources:
  MyCP:
    Type: AWS::Lambda::CapacityProvider
    Properties:
      CapacityProviderName: my-cp
      VpcConfig:
        SubnetIds: [!Ref Sub1, !Ref Sub2, !Ref Sub3]
        SecurityGroupIds: [!Ref SG]
      PermissionsConfig:
        CapacityProviderOperatorRoleArn: !GetAtt OpRole.Arn
      InstanceRequirements:
        Architectures: [arm64]
      CapacityProviderScalingConfig:
        MaxVCpuCount: 30

  MyFn:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: python3.13
      Handler: app.handler
      MemorySize: 4096
      Architectures: [arm64]
      CapacityProviderConfig:
        LambdaManagedInstancesCapacityProviderConfig:
          CapacityProviderArn: !GetAtt MyCP.Arn

Cleanup

aws lambda delete-function --function-name my-fn
aws lambda delete-capacity-provider --capacity-provider-name my-cp

Deleting the capacity provider destroys all associated EC2 instances.

LMI Migration Patterns

Before/after code examples for migrating to multi-concurrency.

Node.js

Global State

// BEFORE (race condition)
let requestCount = 0;
exports.handler = async (event) => {
  requestCount++;
  return { count: requestCount };
};

// AFTER (request-isolated)
const { AsyncLocalStorage } = require('node:async_hooks');
const als = new AsyncLocalStorage();
exports.handler = async (event, context) => {
  return als.run({ id: context.awsRequestId }, async () => {
    return await processEvent(event);
  });
};

File I/O

// BEFORE (shared path)
fs.writeFileSync('/tmp/output.json', JSON.stringify(data));

// AFTER (request-unique path)
const path = `/tmp/output-${context.awsRequestId}.json`;
try { fs.writeFileSync(path, JSON.stringify(data)); }
finally { fs.unlinkSync(path); }

Database

// BEFORE (per-invocation connection)
exports.handler = async (event) => {
  const conn = await mysql.createConnection({/*...*/});
  const [rows] = await conn.execute('SELECT ...');
  await conn.end();
};

// AFTER (shared pool)
// For production: retrieve credentials from AWS Secrets Manager
// const { SecretsManagerClient, GetSecretValueCommand } = require('@aws-sdk/client-secrets-manager');
const pool = mysql.createPool({ connectionLimit: 10, /*...*/ });
exports.handler = async (event) => {
  const [rows] = await pool.execute('SELECT ...');
  return rows;
};

Python

Python on LMI uses process-based isolation. Each concurrent invocation runs in its own process with independent memory. Global state is NOT shared, so no locking is needed. The main migration concerns are /tmp conflicts, memory sizing, and connection pooling.

Global State (No Changes Needed)

# This is SAFE on LMI — each process has its own copy of cache
cache = {}
def handler(event, context):
    cache[event['key']] = compute(event)
    return cache[event['key']]

# Module-level clients are also safe (isolated per process)
s3_client = boto3.client('s3')
dynamodb = boto3.resource('dynamodb')

File I/O (Change Required — /tmp is shared across processes)

# BEFORE (conflict — all processes share /tmp)
with open('/tmp/data.json', 'w') as f: json.dump(event, f)

# AFTER (request-unique path)
path = f'/tmp/data-{context.aws_request_id}.json'
try:
    with open(path, 'w') as f: json.dump(event, f)
finally:
    os.unlink(path)

Database (Change Required — each process needs pooled connections)

# BEFORE (per-invocation connection — exhausts limits at concurrency)
def handler(event, context):
    conn = psycopg2.connect(host='...')

# AFTER (pool per process — initialized at module level)
from psycopg2 import pool
db_pool = pool.SimpleConnectionPool(1, 3, host=os.environ['DB_HOST'])
def handler(event, context):
    conn = db_pool.getconn()
    try: return query(conn, event)
    finally: db_pool.putconn(conn)
# Note: total connections = pool_size × concurrency (e.g., 3 × 16 = 48)

# For production: retrieve credentials from Secrets Manager, not environment variables
# import boto3
# secret = boto3.client("secretsmanager").get_secret_value(SecretId="my-db-creds")

Memory Sizing

# A function using 200 MB per process with default concurrency of 16:
# Total memory ≈ 200 MB × 16 = 3.2 GB
# Use 4:1 or 8:1 memory-to-vCPU ratio to accommodate
# Monitor MemoryUtilization metric and adjust as needed

Java

Global State

// BEFORE (race condition)
private static Map<String, String> cache = new HashMap<>();

// AFTER (thread-safe)
private static final ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>();
// Use cache.computeIfAbsent(key, k -> compute(k));

Database

// BEFORE (per-invocation)
Connection conn = DriverManager.getConnection("jdbc:...");

// AFTER (HikariCP pool, static init)
private static final HikariDataSource ds;
static {
    HikariConfig c = new HikariConfig();
    // For production: retrieve credentials from Secrets Manager, not env vars
    c.setJdbcUrl(System.getenv("DB_URL"));
    c.setMaximumPoolSize(10);
    ds = new HikariDataSource(c);
}
// Use: try (Connection conn = ds.getConnection()) { ... }

Concurrency Safety for LMI

LMI runs multiple invocations concurrently in the same execution environment. The concurrency model differs by runtime — some require thread safety, others provide process isolation.

Code Review Checklist

When reviewing a function for LMI readiness, check each item:

• [ ] No shared /tmp paths (use request ID in filenames, clean up after — shared across ALL runtimes)
• [ ] Database connections use pools (initialized outside handler, not per-invocation)
• [ ] SDK clients outside handler (module-level singletons are fine — they are thread-safe)
• [ ] Logging includes request ID (for tracing concurrent requests)
• [ ] Node.js/Java/.NET only: No global/static mutable variables (use immutable or request-local state)
• [ ] Node.js/Java/.NET only: Thread-safe libraries only (check DB drivers, HTTP clients, caching libs)
• [ ] Node.js/Java/.NET only: No request state in global scope (use AsyncLocalStorage, ThreadLocal, AsyncLocal<T>)
• [ ] Node.js/Java/.NET only: No environment variable mutation during requests
• [ ] Python only: Memory budget accounts for per-process multiplication (memory × concurrency)

Runtime-Specific Guidance

Python (Process-Based Isolation)

Python uses multiple independent processes, each with its own interpreter and memory space. Global variables, module-level caches, and singleton objects are duplicated per process, not shared. If a function works on standard Lambda today, it works on LMI without code changes related to shared state.

Key concerns:

• Memory consumption: total footprint ≈ per-process memory × concurrency. A 200 MB function with 16 concurrent processes can consume 3+ GB.
• /tmp filesystem is shared across all processes — use context.aws_request_id in filenames
• Each process needs its own connection pool — size pools per-process, not globally
• Prefer 4:1 or 8:1 memory-to-vCPU ratio to accommodate memory multiplication
• Monitor MemoryUtilization metric and adjust ratio if needed

Safe patterns (no locking needed):

• Module-level mutable globals (isolated per process)
• Module-level SDK clients and caches
• os.environ reads

Node.js (Worker Threads + Async/Await)

Uses worker threads combined with async/await event loops. The handler and global state are shared across concurrent invocations within a worker thread.

The await keyword yields control to the event loop, which may execute another invocation that overwrites shared state before the first resumes.

Key concerns:

• Use AsyncLocalStorage from node:async_hooks for request context
• Keep mutable state within handler local scope
• Initialize SDK clients and DB pools at module level (they are thread-safe)
• Avoid module-level mutable state (let count = 0 is a race condition)
• Callback-based handlers are NOT supported on Node.js 22 — use async handlers

Java (OS Threads)

Uses OS-level threads. Lambda loads the handler class once and invokes handleRequest from multiple threads simultaneously.

Key concerns:

• Use immutable objects and thread-safe collections (ConcurrentHashMap, Collections.synchronizedList)
• Initialize SDK clients and connection pools in constructor or static block
• Avoid mutable static fields
• Use ThreadLocal<T> for request-specific state
• Use HikariCP or similar for connection pooling (AWS SDK for Java 2.x clients are thread-safe)

.NET (Task-Based Concurrency)

Uses a single process with .NET Tasks (same model as ASP.NET Core). The handler object is shared across all Tasks.

Key concerns:

• Use AsyncLocal<T> for request-scoped data
• Inject scoped services via DI container
• Initialize HttpClient and SDK clients as singletons
• Use ConcurrentDictionary<TKey, TValue> and SemaphoreSlim for thread-safe access
• Invocation timeouts are NOT enforced by the runtime — use ILambdaContext.RemainingTime

Common Anti-Patterns

Powertools for AWS Lambda Compatibility

Powertools handles multi-concurrency transparently. No code changes needed.

AWS SDK and X-Ray minimum versions:

LMI Troubleshooting

Common Issues

Debugging Steps

Throttling Issues

1. Check throttles metric for the reason for throttles

Function Not Starting

1. Check capacity provider status: aws lambda get-capacity-provider --capacity-provider-name <name> 2. Verify subnets span 3+ AZs with available IPs 3. Confirm security group allows necessary egress 4. Check operator role has required permissions 5. Check for LMI-managed instances:

   aws ec2 describe-instances --filters "Name=tag-key,Values=aws:lambda:capacity-provider" \
     --query "Reservations[].Instances[].{Id:InstanceId,State:State.Name}"

Performance Issues

1. Check CloudWatch metrics (5-min intervals): CPU utilization, memory, concurrency/env 2. If CPU > 80%: reduce concurrency or add vCPUs (increase memory with appropriate ratio) 3. If throttles > 1%: increase MinExecutionEnvironments 4. If CPU < 20%: increase concurrency — resources are underutilized 5. For Python: verify 4:1 or 8:1 ratio (GIL limits CPU parallelism)

Cost Issues

1. Verify instance count matches actual need (not over-provisioned) 2. Check if Savings Plans or RIs are applied to these instances 3. Compare actual costs against the LMI Pricing Calculator 4. If traffic is lower than expected, consider reducing MaxVCpuCount 5. For dev/test: use ExcludedInstanceTypes to avoid expensive instance families