
Microservices Patterns
- 2 installs
- 28 repo stars
- Updated June 29, 2026
- nickcrew/claude-cortex
This is a copy of microservices-patterns by nickcrew - installs and ranking accrue to the original listing.
Helps with ai & agent building tasks.
About
microservices-patterns is a Claude Code skill for ai & agent building. It helps you ship faster with AI-assisted development.
- microservices-patterns
- AI & Agent Building
- AI-coding skill
Microservices Patterns by the numbers
- 2 all-time installs (skills.sh)
- Data as of Aug 4, 2026 (Skillselion catalog sync)
npx skills add https://github.com/nickcrew/claude-cortex --skill microservices-patternsAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 2 |
|---|---|
| repo stars | ★ 28 |
| Last updated | June 29, 2026 |
| Repository | nickcrew/claude-cortex ↗ |
What it does
Helps with ai & agent building tasks.
Files
Microservices Architecture Patterns
Expert guidance for designing, implementing, and operating microservices architectures.
When to Use This Skill
- Breaking down monolithic applications into services
- Designing distributed systems from scratch
- Implementing service communication patterns (sync/async)
- Managing data consistency across services
- Building resilient distributed systems
- Defining service boundaries and API contracts
Core Principles
1. Single Responsibility - Each service has one reason to change 2. Independent Deployability - No coordination required for deployments 3. Decentralized Data - Each service owns its data exclusively 4. Design for Failure - Embrace failures, build resilience 5. Automate Everything - Deployment, scaling, and recovery
Quick Reference
Load detailed patterns on-demand:
| Task | Load Reference |
|---|---|
| Define service boundaries and decompose monoliths | skills/microservices-patterns/references/service-decomposition.md |
| Implement service communication (sync/async) | skills/microservices-patterns/references/communication-patterns.md |
| Manage data consistency and transactions | skills/microservices-patterns/references/data-management.md |
| Build resilient systems (circuit breakers, retries) | skills/microservices-patterns/references/resilience-patterns.md |
| Add observability (tracing, logging, metrics) | skills/microservices-patterns/references/observability.md |
| Plan deployments and migrations | skills/microservices-patterns/references/deployment-migration.md |
Workflow
1. Understand Requirements
- Map business capabilities and domains
- Assess scalability/resilience needs
- Identify team boundaries
2. Define Service Boundaries
Load references/service-decomposition.md for:
- Business capability decomposition
- DDD bounded contexts
- Service boundary validation
3. Design Communication
Load references/communication-patterns.md for:
- Synchronous: API Gateway, REST, gRPC
- Asynchronous: Message Queue, Pub/Sub, Event Sourcing
4. Manage Data
Load references/data-management.md for:
- Database per service pattern
- Saga distributed transactions
- CQRS read/write optimization
5. Build Resilience
Load references/resilience-patterns.md for:
- Circuit breakers
- Retry with exponential backoff
- Bulkhead isolation
- Rate limiting and timeouts
6. Add Observability
Load references/observability.md for:
- Distributed tracing
- Centralized logging
- Metrics and monitoring
7. Plan Deployment
Load references/deployment-migration.md for:
- Blue-Green, Canary, Rolling deployments
- Strangler Fig migration pattern
Common Mistakes
1. Distributed Monolith - Tightly coupled, must deploy together 2. Shared Database - Multiple services accessing same database 3. Chatty APIs - Excessive synchronous service calls 4. Missing Circuit Breakers - No cascading failure protection 5. No Observability - Deploying without tracing/logging/metrics 6. Ignoring Network Failures - Assuming reliable network 7. No API Versioning - Breaking changes without versioning
Fixes: Load relevant reference files for detailed solutions.
Resources
- Books: "Building Microservices" (Newman), "Microservices Patterns" (Richardson)
- Sites: microservices.io, martinfowler.com/microservices
- Tools: Kubernetes, Istio, Kafka, Kong, Jaeger, Prometheus
Communication Patterns
Patterns for synchronous and asynchronous service-to-service communication.
Synchronous Communication
Pattern: API Gateway
Purpose: Single entry point for all clients, routing to appropriate services.
Client → API Gateway → [Auth, Rate Limiting, Routing] → Microservices
Benefits:
- Simplified client interface
- Centralized cross-cutting concerns
- Protocol translation (REST → gRPC)
- Request aggregation
Implementations: Kong, AWS API Gateway, Nginx, TraefikPattern: Service-to-Service REST
Best Practices:
# Use service discovery (Consul, Eureka, Kubernetes DNS)
GET http://order-service:8080/orders/123
# Include correlation IDs for tracing
X-Correlation-ID: a3f7c9b2-d8e1-4f6g-h9i0
# Use circuit breakers (Hystrix, Resilience4j)
@CircuitBreaker(name = "inventory-service")
public Product getProduct(String id) {
return restTemplate.getForObject(
"http://inventory-service/products/" + id,
Product.class
);
}
# Implement timeouts
connect-timeout: 2000
read-timeout: 5000Pattern: gRPC for Internal Communication
When to Use:
- High performance requirements
- Type-safe contracts (Protocol Buffers)
- Streaming data (server/client/bidirectional)
- Internal service-to-service communication
service OrderService {
rpc GetOrder (GetOrderRequest) returns (Order);
rpc CreateOrder (CreateOrderRequest) returns (Order);
rpc StreamOrders (StreamRequest) returns (stream Order);
}
message Order {
string id = 1;
string customer_id = 2;
repeated OrderItem items = 3;
double total = 4;
}Asynchronous Communication
Pattern: Event-Driven Architecture
Purpose: Services communicate via events, decoupled in time and space.
Event Types:
1. Domain Events (business events):
- OrderCreated
- PaymentProcessed
- InventoryReserved
2. Change Data Capture (CDC):
- OrderStatusChanged
- CustomerUpdated
3. Integration Events (cross-service):
- SendWelcomeEmail
- UpdateRecommendationsEvent Structure:
{
"event_id": "evt_a3f7c9b2",
"event_type": "order.created",
"event_version": "1.0",
"timestamp": "2024-01-15T10:30:00Z",
"source": "order-service",
"correlation_id": "corr_x1y2z3",
"data": {
"order_id": "ord_123",
"customer_id": "cust_456",
"total": 99.99,
"items": [...]
},
"metadata": {
"user_id": "user_789",
"tenant_id": "tenant_abc"
}
}Pattern: Message Queue (Point-to-Point)
Use Case: Work distribution, background jobs, reliable delivery.
Producer → Queue → Consumer(s)
Examples:
- Order placed → Queue → Payment processor
- Email requested → Queue → Email sender
- Image uploaded → Queue → Thumbnail generator
Implementations: RabbitMQ, AWS SQS, Azure Service BusPattern: Publish-Subscribe (Pub/Sub)
Use Case: Broadcasting events to multiple interested services.
Publisher → Topic → Subscriber 1
→ Subscriber 2
→ Subscriber N
Example:
OrderCreated event published to "orders" topic
Subscribers:
- Inventory Service (reserve stock)
- Fulfillment Service (prepare shipment)
- Analytics Service (update metrics)
- Notification Service (send confirmation email)
Implementations: Apache Kafka, AWS SNS, Google Pub/SubPattern: Event Sourcing
Definition: Store all state changes as a sequence of events, not current state.
Traditional (CRUD):
orders table: id, customer_id, status, total
Event Sourcing:
order_events table:
- OrderCreated(order_id, customer_id, items, total)
- PaymentReceived(order_id, amount, payment_method)
- OrderShipped(order_id, tracking_number)
- OrderDelivered(order_id, delivery_time)
Current state = replay all events
Benefits:
- Complete audit trail
- Temporal queries ("what was the state at time T?")
- Event replay for debugging
- Easy to add new projections
Challenges:
- Query complexity
- Event versioning
- Storage growthCommunication Best Practices
When to Use Sync vs Async
Synchronous (REST/gRPC):
- ✅ Real-time queries (get user profile)
- ✅ Request/response workflows
- ✅ Low latency requirements
- ❌ Long-running operations
- ❌ Fire-and-forget actions
Asynchronous (Events/Messages):
- ✅ Fire-and-forget operations
- ✅ Long-running processes
- ✅ Broadcasting to multiple consumers
- ✅ Decoupling services in time
- ❌ Immediate response needed
Service Discovery
Pattern: Services find each other dynamically without hard-coded URLs.
Options:
1. Client-Side Discovery:
Client → Service Registry (Consul/Eureka) → Get service instances → Direct call
2. Server-Side Discovery:
Client → Load Balancer → Service Registry → Route to instance
3. DNS-Based (Kubernetes):
Client → DNS lookup (service-name.namespace.svc.cluster.local) → Service IP
Implementations:
- Consul (HashiCorp)
- Eureka (Netflix)
- Kubernetes DNS
- AWS Cloud MapAPI Versioning
Pattern: Maintain backward compatibility while evolving APIs.
Strategies:
1. URL Versioning:
/api/v1/orders
/api/v2/orders
2. Header Versioning:
Accept: application/vnd.myapi.v2+json
3. Query Parameter:
/api/orders?version=2
4. Content Negotiation:
Accept: application/vnd.myapi+json;version=2
Recommendation: URL versioning (simplest, most explicit)
Version Lifecycle:
- v1: Production (supported)
- v2: Production (current, recommended)
- v3: Beta (early adopters)
- Deprecation policy: 6-12 months noticeError Handling
Pattern: Consistent error responses across services.
Standard error format:
{
"error": {
"code": "INSUFFICIENT_INVENTORY",
"message": "Not enough stock for product SKU-123",
"details": {
"product_id": "SKU-123",
"requested": 10,
"available": 3
},
"timestamp": "2024-01-15T10:30:00Z",
"trace_id": "abc123",
"path": "/api/v1/orders"
}
}
HTTP Status Codes:
- 200: Success
- 201: Created
- 400: Client error (bad request)
- 401: Unauthorized
- 403: Forbidden
- 404: Not found
- 409: Conflict (business rule violation)
- 429: Too many requests (rate limited)
- 500: Server error
- 503: Service unavailable (circuit breaker open)Idempotency
Pattern: Same request can be repeated safely without side effects.
Idempotency Key:
POST /api/orders
Idempotency-Key: a3f7c9b2-d8e1-4f6g
Server stores key + response:
1. First request → Process → Save (key, response) → Return response
2. Duplicate request → Find key → Return cached response
Use for:
- Payment processing
- Order creation
- Any state-changing operation
Implementation:
- Redis/Memcached for key storage
- TTL: 24 hours
- Status: "processing", "completed", "failed"Tools and Technologies
Synchronous Communication
- REST: Spring Boot, Express.js, FastAPI, ASP.NET Core
- gRPC: Protocol Buffers, gRPC-Go, gRPC-Java, gRPC-Web
- API Gateway: Kong, AWS API Gateway, Azure API Management, Apigee
- Service Discovery: Consul, Eureka, Kubernetes DNS, etcd
Asynchronous Communication
- Message Queue: RabbitMQ, AWS SQS, Azure Service Bus
- Pub/Sub: Apache Kafka, AWS SNS+SQS, Google Pub/Sub, NATS
- Event Streaming: Apache Kafka, AWS Kinesis, Azure Event Hubs
- Event Sourcing: Axon Framework, EventStore, Marten
Supporting Tools
- Circuit Breakers: Resilience4j, Hystrix, Polly
- Tracing: Jaeger, Zipkin, AWS X-Ray, DataDog APM
- Load Balancing: Nginx, HAProxy, Envoy, Traefik
Further Reading
- "Enterprise Integration Patterns" by Gregor Hohpe
- "Designing Data-Intensive Applications" by Martin Kleppmann
- microservices.io/patterns/communication-style
- kafka.apache.org/documentation
- grpc.io/docs/what-is-grpc
Data Management Patterns
Pattern: Database per Service
Principle: Each service has its own database, never shared.
Order Service → Orders DB (PostgreSQL)
Inventory Service → Inventory DB (PostgreSQL)
Product Service → Products DB (MongoDB)
Analytics Service → Analytics DB (ClickHouse)
Benefits:
- Independent scaling
- Technology choice flexibility
- Loose coupling
- Clear ownership
Challenges:
- No cross-service joins
- Distributed transactions
- Data consistencyPattern: Saga (Distributed Transactions)
Purpose: Maintain data consistency across services without 2PC.
Orchestration-Based Saga
Order Saga Orchestrator:
1. Create Order (Order Service)
↓ success
2. Reserve Inventory (Inventory Service)
↓ success
3. Process Payment (Payment Service)
↓ success
4. Update Order Status (Order Service)
↓ failure → Compensate
5. Compensating Transactions:
- Release Inventory
- Refund Payment
- Cancel Order
Implementation:
- Orchestrator maintains state machine
- Explicit control flow
- Centralized logicChoreography-Based Saga
Event-Driven Saga:
OrderCreated event
→ Inventory Service reserves stock
→ InventoryReserved event
→ Payment Service processes payment
→ PaymentProcessed event
→ Order Service updates status
If failure at any step:
→ Compensating events cascade backwards
Implementation:
- Decentralized coordination
- Event-driven
- Implicit control flowPattern: CQRS (Command Query Responsibility Segregation)
Definition: Separate read and write models for optimal performance.
Write Model (Commands):
- Optimized for consistency
- Normalized schema
- Transactional
Read Model (Queries):
- Optimized for performance
- Denormalized views
- Eventually consistent
- Materialized views/caching
Sync via events:
Command → Write DB → Event → Read DB(s)
Example:
Write: OrderCreated → Orders DB (PostgreSQL)
Event: OrderCreated published
Read: Update OrderSummary view (Redis)
Update OrderAnalytics (Elasticsearch)Pattern: API Composition
Purpose: Join data from multiple services at the API layer.
GET /customers/123/dashboard
API Gateway:
1. GET /customers/123 (Customer Service)
2. GET /orders?customer_id=123 (Order Service)
3. GET /recommendations/123 (Recommendation Service)
4. Compose response:
{
"customer": {...},
"recent_orders": [...],
"recommendations": [...]
}
Challenges:
- N+1 queries
- Slower response time
- Complex error handling
Optimizations:
- Parallel requests
- GraphQL (client-controlled aggregation)
- Backend for Frontend (BFF) patternData Consistency Strategies
Eventual Consistency
Definition: Data becomes consistent over time, not immediately.
Example: Order placed
1. Order Service: Create order (status: PENDING)
2. Event published: OrderCreated
3. Inventory Service: Reserve stock (async)
4. Payment Service: Process payment (async)
5. Order Service: Update status to CONFIRMED (eventually)
Time window: seconds to minutes
Acceptable for:
- Social media feeds
- Product recommendations
- Analytics dashboards
- Non-critical updates
NOT acceptable for:
- Financial transactions (use Saga)
- Inventory reservations (use Saga)
- Critical business rulesStrong Consistency (within service)
Pattern: ACID transactions within service boundaries.
Order Service Transaction:
BEGIN;
INSERT INTO orders (...);
UPDATE inventory SET reserved = reserved + qty;
INSERT INTO order_items (...);
COMMIT;
Keep related data in same service to maintain ACID.Data Replication Patterns
Change Data Capture (CDC)
Pattern: Capture database changes and publish as events.
Database → Transaction Log → CDC Tool → Event Stream → Consumers
Tools:
- Debezium (Kafka Connect)
- AWS DMS
- Maxwell's Daemon
- Databus (LinkedIn)
Example:
orders table changes → Debezium → Kafka topic → Analytics Service
Benefits:
- No application code changes
- Guaranteed event publication
- Ordered events per entityRead Replicas
Pattern: Replicate data for read scaling.
Write: Client → Primary DB
Read: Client → Read Replica 1/2/3
Lag: Eventually consistent (seconds)
Use for:
- Analytics queries
- Search indexing
- Reporting dashboards
- Read-heavy workloadsData Access Patterns
API per Service
Rule: Only access service data through its API, never directly to database.
❌ WRONG:
Order Service → Inventory DB (direct access)
✅ CORRECT:
Order Service → Inventory Service API → Inventory DB
Why?
- Maintains encapsulation
- Allows service to evolve data model
- Enables security/validation
- Supports versioningShared Data Services
Pattern: Create dedicated service for truly shared data.
Example: Reference Data Service
- Country codes
- Currency rates
- Product categories
- Tax rates
Characteristics:
- Read-mostly data
- Infrequent updates
- Needed by multiple services
- CacheableData Migration Strategies
Dual Writes (Transitional)
Pattern: Write to both old and new data stores during migration.
Migration phases:
1. Old DB only (monolith)
2. Dual write (old + new)
3. Migrate existing data
4. Verify consistency
5. Switch reads to new
6. Remove old writes
7. Decommission old DB
Caution: Not atomic, use for read-mostly dataEvent-Based Migration
Pattern: Publish events from monolith, new services consume.
Monolith → Events → New Microservice
→ Old DB → (gradually deprecated)
Advantages:
- Less risky than dual writes
- Services can evolve independently
- Supports gradual migrationTools and Technologies
Databases
- Relational: PostgreSQL, MySQL, SQL Server
- Document: MongoDB, Couchbase, DynamoDB
- Key-Value: Redis, Memcached
- Column: Cassandra, HBase
- Search: Elasticsearch, Solr
- Time-Series: InfluxDB, TimescaleDB
- Graph: Neo4j, Amazon Neptune
Saga Orchestration
- Frameworks: Axon Framework, Eventuate, Temporal
- Workflow Engines: Camunda, Zeebe, Conductor (Netflix)
- Custom: State machine in code
CDC Tools
- Debezium: Kafka-based CDC for MySQL, PostgreSQL, MongoDB
- Maxwell: MySQL binlog to Kafka
- AWS DMS: Database Migration Service
Best Practices
1. Database per Service - Mandatory, no exceptions 2. Own Your Data - Each service is the source of truth for its data 3. Event-Driven - Use events for cross-service data synchronization 4. Embrace Eventual Consistency - Design for it from the start 5. Saga for Transactions - Use orchestration or choreography patterns 6. CQRS for Complex Queries - Separate read/write models when needed 7. Cache Aggressively - Reduce cross-service calls with caching 8. Monitor Data Lag - Track eventual consistency lag in production 9. Version Your Events - Event schema evolution strategy 10. Test Distributed Scenarios - Chaos engineering for data consistency
Common Pitfalls
❌ Shared Database - Multiple services accessing same database ❌ Distributed Transactions - 2PC across services (avoid at all costs) ❌ Synchronous Saga - Blocking saga calls (use async) ❌ Missing Compensations - Saga without rollback logic ❌ No Idempotency - Duplicate event processing causes issues ❌ Ignoring Data Lag - Not monitoring eventual consistency delays ❌ Direct DB Access - Bypassing service APIs
Further Reading
- "Designing Data-Intensive Applications" by Martin Kleppmann
- "Microservices Patterns" by Chris Richardson (Saga patterns)
- microservices.io/patterns/data
- martinfowler.com/articles/microservices.html#DecentralizedDataManagement
Deployment & Migration Patterns
Strategies for deploying microservices and migrating from monolithic architectures.
Deployment Patterns
Blue-Green Deployment
Load Balancer
→ Blue (current version, 100% traffic)
→ Green (new version, 0% traffic)
Deploy to Green → Test → Switch traffic → Blue becomes standbyCanary Deployment
Load Balancer
→ v1 (95% traffic)
→ v2 (5% traffic - canary)
Monitor metrics → Increase traffic → Full rolloutRolling Deployment
Instances: [v1, v1, v1, v1]
Step 1: [v2, v1, v1, v1]
Step 2: [v2, v2, v1, v1]
Step 3: [v2, v2, v2, v1]
Step 4: [v2, v2, v2, v2]Migration Strategies
Strangler Fig Pattern
Purpose: Gradually migrate from monolith to microservices.
Phase 1: Routing layer intercepts requests
Client → Router → Monolith (all traffic)
Phase 2: Extract first service
Client → Router → Service A (10% traffic)
→ Monolith (90% traffic)
Phase 3: Extract more services
Client → Router → Service A (all orders)
→ Service B (all users)
→ Monolith (remaining)
Phase N: Retire monolith
Client → Router → Services A, B, C, ... (all traffic)Branch by Abstraction
Purpose: Refactor incrementally without feature branches.
1. Create abstraction layer
2. Implement new service behind abstraction
3. Gradually migrate calls to new implementation
4. Remove old implementation
5. Remove abstraction (optional)Continuous Deployment Strategies
Feature Flags
Pattern: Deploy code, enable features gradually via configuration.
Code:
if (featureFlags.isEnabled("new-checkout-flow", userId)) {
return newCheckoutService.process(order);
} else {
return legacyCheckoutService.process(order);
}
Benefits:
- Deploy anytime, release separately
- Gradual rollout (1% → 10% → 100%)
- A/B testing
- Instant rollback (no deploy needed)
- Per-user targeting
Tools:
- LaunchDarkly (commercial)
- Unleash (open source)
- AWS AppConfig
- Custom solution (database + cache)Database Migration Strategies
Pattern: Evolve database schema without downtime.
Phase 1: Expand
- Add new column (nullable)
- Deploy code that writes to both old and new
- Backfill existing data
Phase 2: Migrate
- All code using new column
- Old column no longer written
Phase 3: Contract
- Remove old column
- Deploy code without old column references
Never:
- Rename columns in-place
- Drop columns with active code
- Change types without migrationZero-Downtime Deployment Checklist
✓ **Before Deployment**:
- [ ] Backward-compatible changes only
- [ ] Database migrations deployed separately
- [ ] Feature flags for risky changes
- [ ] Rollback plan documented
- [ ] Health checks configured
✓ **During Deployment**:
- [ ] Rolling deployment (not all-at-once)
- [ ] Monitor error rates
- [ ] Watch response times
- [ ] Check dependency health
✓ **After Deployment**:
- [ ] Verify metrics return to normal
- [ ] Check logs for errors
- [ ] Validate business metrics
- [ ] Document any issuesMigration Best Practices
Strangler Fig Execution Plan
Step 1: Analysis
- Map monolith functionality
- Identify service boundaries
- Prioritize extraction order
Step 2: Setup Infrastructure
- Deploy API gateway/router
- Setup monitoring
- Implement tracing
Step 3: Extract First Service (low-risk)
- Choose independent, low-traffic feature
- Implement as microservice
- Route small % of traffic
- Validate and increase traffic
Step 4: Extract Core Services
- One at a time
- Validate each extraction
- Maintain monolith functioning
Step 5: Retire Monolith
- When all functionality extracted
- Gradual deprecation
- Final decommissionData Migration Strategies
Pattern: Migrate data ownership from monolith to services.
Strategy 1: ETL (Extract-Transform-Load)
- One-time bulk copy
- Use for read-only/archive data
- Simple but downtime risk
Strategy 2: Dual-Write
- Write to both old and new
- Gradually switch reads
- Risk: consistency issues
- Use for transitional period only
Strategy 3: Event-Driven Sync
- Monolith publishes events
- Service consumes and builds own data
- Eventual consistency
- Best for ongoing migration
Strategy 4: Change Data Capture (CDC)
- Capture database changes
- Publish as events
- Services subscribe
- No monolith code changesAPI Gateway Configuration
Pattern: Route requests between monolith and services.
# Nginx example
location /api/orders {
proxy_pass http://order-service:8080;
}
location /api/products {
proxy_pass http://product-service:8080;
}
location / {
proxy_pass http://monolith:8080;
}
# Kong example (declarative config)
services:
- name: order-service
url: http://order-service:8080
routes:
- paths: ["/api/orders"]
- name: monolith
url: http://monolith:8080
routes:
- paths: ["/"]Testing Strategies
Service Testing Pyramid
End-to-End (5%):
- Full system integration tests
- Expensive, slow, fragile
- Use sparingly for critical paths
Integration (20%):
- Service + dependencies (DB, cache)
- Test service in isolation
- Use test containers
Contract (25%):
- API contract validation
- Consumer-driven contracts (Pact)
- Ensure API compatibility
Unit (50%):
- Fast, isolated, deterministic
- Business logic coverage
- Foundation of test suiteContract Testing
Pattern: Verify service compatibility without integration tests.
Producer (Order Service):
- Publishes contract: "POST /orders expects {items, total}"
- Contract tests verify implementation matches
Consumer (UI):
- Defines expectations: "When I POST /orders, I get 201 with order_id"
- Contract tests verify producer meets expectations
Tool: Pact
1. Consumer writes Pact test
2. Publishes contract to broker
3. Producer verifies contract
4. Both can deploy independently if contracts matchDeployment Tools and Platforms
Container Orchestration
Kubernetes:
- Industry standard
- Complex but powerful
- Rich ecosystem
- Automatic scaling, healing
Docker Swarm:
- Simpler than K8s
- Less feature-rich
- Built into Docker
- Good for small-medium deployments
AWS ECS/Fargate:
- AWS-specific
- Simpler than K8s
- Serverless option (Fargate)
- Tight AWS integrationCI/CD Pipelines
Typical Pipeline:
1. Code Push
↓
2. Build & Test
- Unit tests
- Linting
- Security scan
↓
3. Build Container Image
- Docker build
- Push to registry
↓
4. Deploy to Staging
- Run integration tests
- Contract tests
↓
5. Deploy to Production
- Canary deployment
- Monitor metrics
- Gradual rollout
Tools:
- Jenkins (self-hosted)
- GitLab CI (integrated)
- GitHub Actions (cloud)
- CircleCI (cloud)
- AWS CodePipeline (AWS)Infrastructure as Code
Terraform:
resource "kubernetes_deployment" "order_service" {
metadata {
name = "order-service"
}
spec {
replicas = 3
selector {
match_labels = {
app = "order-service"
}
}
template {
metadata {
labels = {
app = "order-service"
}
}
spec {
container {
image = "order-service:1.2.3"
name = "order-service"
}
}
}
}
}
Benefits:
- Version controlled infrastructure
- Reproducible environments
- Automated provisioning
- Documentation as codeAnti-Patterns to Avoid
1. Distributed Monolith - Tightly coupled services, must deploy together 2. Shared Database - Multiple services accessing same database 3. Chatty APIs - Excessive synchronous service calls 4. Mega Services - Services too large, violating single responsibility 5. Missing Circuit Breakers - No cascading failure protection 6. Synchronous Everything - No asynchronous communication 7. God Service - One service orchestrating everything 8. Ignoring Network Failures - Assuming reliable network 9. No Versioning - Breaking changes without versioning 10. Missing Monitoring - Deploying without observability 11. Big Bang Migration - Rewrite everything at once 12. No Rollback Plan - Can't undo deployments 13. Manual Deployments - No automation, error-prone
Migration Checklist
Pre-Migration
- [ ] Business case validated
- [ ] Team has microservices experience
- [ ] Monitoring infrastructure ready
- [ ] CI/CD pipelines established
- [ ] Service boundaries defined
- [ ] Migration roadmap created
During Migration
- [ ] Extract services incrementally
- [ ] Maintain monolith stability
- [ ] Monitor both old and new
- [ ] Document service APIs
- [ ] Implement circuit breakers
- [ ] Add distributed tracing
Post-Migration
- [ ] Decommission monolith
- [ ] Update documentation
- [ ] Conduct retrospective
- [ ] Optimize performance
- [ ] Refine monitoring
- [ ] Plan next services
Tools and Technologies
Deployment Platforms
- Kubernetes: Container orchestration
- AWS ECS/Fargate: Managed containers
- Google Cloud Run: Serverless containers
- Azure Container Instances: Managed containers
Service Mesh
- Istio: Full-featured, complex
- Linkerd: Lightweight, simple
- Consul: Service discovery + mesh
CI/CD
- Jenkins: Self-hosted, flexible
- GitLab CI: Integrated with Git
- GitHub Actions: Cloud-based
- ArgoCD: GitOps for Kubernetes
Feature Flags
- LaunchDarkly: Enterprise feature management
- Unleash: Open source feature flags
- Split.io: Feature delivery platform
Migration Tools
- Debezium: Change data capture
- AWS DMS: Database migration service
- Liquibase/Flyway: Database schema migration
Further Reading
- "Monolith to Microservices" by Sam Newman
- "Accelerate" by Nicole Forsgren (DevOps practices)
- "Continuous Delivery" by Jez Humble
- martinfowler.com/bliki/StranglerFigApplication.html
- kubernetes.io/docs/concepts/workloads
Observability & Cross-Cutting Concerns
The three pillars of observability: distributed tracing, centralized logging, and metrics/monitoring.
Distributed Tracing
Request ID propagation:
Client → API Gateway [trace_id: abc123]
→ Service A [trace_id: abc123, span_id: 001]
→ Service B [trace_id: abc123, span_id: 002]
→ Service C [trace_id: abc123, span_id: 003]
Implementations: Jaeger, Zipkin, AWS X-Ray
Correlation:
X-Correlation-ID: abc123
X-Request-ID: req_xyz789Centralized Logging
Log aggregation pattern:
Services → Log Shipper → Log Aggregator → Search/Analysis
Structure logs (JSON):
{
"timestamp": "2024-01-15T10:30:00Z",
"level": "INFO",
"service": "order-service",
"trace_id": "abc123",
"span_id": "001",
"message": "Order created",
"order_id": "ord_123",
"customer_id": "cust_456"
}
Stack: Filebeat → Logstash → Elasticsearch → Kibana
Fluentd → Kafka → SplunkMetrics & Monitoring
Key Metrics (RED method):
- Rate: requests per second
- Errors: error rate
- Duration: response time (p50, p95, p99)
USE method (infrastructure):
- Utilization: CPU, memory, disk
- Saturation: queue depth
- Errors: error counts
Implementations: Prometheus, Grafana, DataDogService Mesh
Purpose: Infrastructure layer handling service-to-service communication.
Features:
- Traffic management (routing, retries, timeouts)
- Security (mTLS, authentication)
- Observability (metrics, tracing)
- Resilience (circuit breaking, rate limiting)
Architecture:
Service A ←→ Sidecar Proxy (Envoy)
↕
Control Plane (Istio/Linkerd)
↕
Service B ←→ Sidecar Proxy (Envoy)
Implementations: Istio, Linkerd, Consul ConnectObservability Best Practices
Correlation IDs
Pattern: Track requests across services with unique identifiers.
Client Request → Generate trace_id
→ Service A (trace_id, span_id: A1)
→ Service B (trace_id, span_id: B1)
→ Service C (trace_id, span_id: C1)
Headers:
X-Trace-ID: abc123xyz
X-Span-ID: service-a-span-001
X-Parent-Span-ID: gateway-span-001
Benefits:
- End-to-end request tracking
- Debug production issues
- Performance analysis
- Error correlationStructured Logging
Pattern: Use JSON format for machine-readable logs.
{
"timestamp": "2024-01-15T10:30:00.123Z",
"level": "INFO",
"service": "order-service",
"version": "1.2.3",
"environment": "production",
"trace_id": "abc123",
"span_id": "span001",
"user_id": "user456",
"message": "Order created successfully",
"event": "order.created",
"order_id": "ord_789",
"amount": 99.99,
"duration_ms": 145,
"http": {
"method": "POST",
"path": "/api/orders",
"status": 201
}
}
Benefits:
- Easy to parse and query
- Consistent structure
- Rich context
- Aggregation friendlyService Level Objectives (SLOs)
Pattern: Define and monitor service quality targets.
SLI (Service Level Indicator):
- Availability: 99.9% requests successful
- Latency: p95 < 200ms
- Throughput: Handle 1000 req/s
SLO (Service Level Objective):
- 99.9% availability over 30 days
- 99% of requests < 200ms (p99)
- Zero data loss
SLA (Service Level Agreement):
- Customer-facing commitment
- Financial penalties if breached
- Usually lower than internal SLO
Error Budget:
- 100% - 99.9% = 0.1% error budget
- ~43 minutes downtime per month
- When exhausted: freeze features, fix reliabilityAlerting Strategy
Pattern: Alert on symptoms, not causes.
✅ GOOD Alerts (user-impacting):
- Error rate > 5% for 5 minutes
- p99 latency > 1s for 5 minutes
- Availability < 99.9% over 1 hour
❌ BAD Alerts (internal metrics):
- CPU > 80% (might be normal)
- Disk > 90% (not user-facing yet)
- Memory > 70% (symptom, not problem)
Alert Levels:
- Page (critical, wake up on-call)
- Ticket (important, fix next day)
- Log (info, review weekly)
Alert Fatigue Prevention:
- Only alert on user impact
- Require action on every alert
- Group related alerts
- Auto-resolve when recoveredDashboards
Pattern: Visualize system health and performance.
Dashboard Hierarchy:
1. System Overview (executives):
- Overall availability
- Request rate
- Error rate
- Key business metrics
2. Service Dashboard (engineers):
- Service-specific RED metrics
- Dependency health
- Resource utilization
- Recent deployments
3. Detail Dashboard (debugging):
- Per-endpoint metrics
- Database query performance
- Cache hit rates
- Queue depths
Tools:
- Grafana (open source)
- DataDog (commercial)
- New Relic (commercial)
- AWS CloudWatchObservability Stack Examples
Open Source Stack
Metrics:
Prometheus (collection) → Grafana (visualization)
Logging:
Fluentd (collection) → Elasticsearch (storage) → Kibana (visualization)
Tracing:
Jaeger (distributed tracing)
Cost: Free (infrastructure + operational overhead)Commercial Stack
All-in-One:
DataDog, New Relic, Dynatrace
Benefits:
- Integrated metrics, logs, tracing
- Advanced anomaly detection
- Automatic instrumentation
- Better support
Cost: $15-100 per host/monthHybrid Stack
Metrics: Prometheus (self-hosted) → Grafana Cloud (managed)
Logging: Fluentd → Splunk/Sumo Logic (managed)
Tracing: OpenTelemetry → Jaeger (self-hosted)
Benefits:
- Control over critical data
- Reduce operational burden
- Cost optimization
Cost: Mix of free and paidSecurity Observability
Audit Logging
Pattern: Track all security-relevant events.
{
"event_type": "authentication.login",
"timestamp": "2024-01-15T10:30:00Z",
"user_id": "user123",
"ip_address": "203.0.113.42",
"user_agent": "Mozilla/5.0...",
"result": "success",
"mfa_used": true,
"location": {
"country": "US",
"city": "San Francisco"
},
"risk_score": 0.2
}
Log these events:
- Authentication (login, logout, MFA)
- Authorization (permission checks)
- Data access (PII, financial data)
- Configuration changes
- Admin actionsAnomaly Detection
Pattern: Detect unusual patterns automatically.
Examples:
- Sudden spike in 401 errors (attack?)
- Unusual geographic logins
- Abnormal data access patterns
- Unexpected service dependencies
- Traffic pattern changes
Tools:
- DataDog anomaly detection
- AWS GuardDuty
- Elastic ML (machine learning)
- Custom algorithmsTools and Technologies
Metrics Collection
- Prometheus: Open source, pull-based, time-series
- StatsD: Push-based, simple aggregation
- OpenTelemetry: Unified standard
- Cloud Native: AWS CloudWatch, GCP Monitoring, Azure Monitor
Logging
- Elasticsearch: Search and analytics engine
- Loki: Prometheus-inspired log aggregation
- Splunk: Enterprise log management
- CloudWatch Logs: AWS managed logging
Distributed Tracing
- Jaeger: CNCF project, Uber-originated
- Zipkin: Twitter-originated
- Tempo: Grafana-integrated tracing
- AWS X-Ray: Managed tracing for AWS
- OpenTelemetry: Vendor-neutral instrumentation
APM (Application Performance Monitoring)
- DataDog APM: Full-stack observability
- New Relic: Application monitoring
- Dynatrace: AI-powered monitoring
- Elastic APM: Open source APM
Service Mesh
- Istio: Feature-rich, complex
- Linkerd: Lightweight, simple
- Consul Connect: HashiCorp service mesh
- AWS App Mesh: Managed service mesh
Implementation Checklist
✓ Metrics:
- [ ] RED metrics per service (Rate, Errors, Duration)
- [ ] USE metrics per resource (Utilization, Saturation, Errors)
- [ ] Business metrics (orders, revenue, conversions)
- [ ] Prometheus/StatsD instrumentation
- [ ] Grafana dashboards
✓ Logging:
- [ ] Structured JSON logs
- [ ] Centralized log aggregation
- [ ] Correlation IDs in all logs
- [ ] Log levels properly used
- [ ] Log retention policy (30-90 days)
✓ Tracing:
- [ ] Distributed tracing enabled
- [ ] Trace context propagation
- [ ] Sampling strategy defined
- [ ] Trace visualization (Jaeger UI)
- [ ] Performance analysis capability
✓ Alerting:
- [ ] SLO-based alerts defined
- [ ] On-call rotation configured
- [ ] Alert runbooks documented
- [ ] Alert fatigue mitigated
- [ ] Escalation policy defined
✓ Security:
- [ ] Audit logs for sensitive operations
- [ ] Anomaly detection configured
- [ ] Security metrics tracked
- [ ] Compliance requirements met
Further Reading
- "Distributed Systems Observability" by Cindy Sridharan
- "Site Reliability Engineering" by Google (Chapter 6: Monitoring)
- opentelemetry.io (unified observability standard)
- prometheus.io/docs/practices
- grafana.com/docs/grafana-cloud
Resilience Patterns
Pattern: Circuit Breaker
Purpose: Prevent cascading failures by failing fast.
States:
Closed → Normal operation, requests pass through
Open → Failure threshold reached, fail fast
Half-Open → Test if service recovered
Configuration:
failure_threshold: 5 failures in 10s
timeout: 30s
half_open_max_calls: 3
@CircuitBreaker(name = "payment-service")
public PaymentResult processPayment(Payment payment) {
return paymentClient.process(payment);
}
Libraries: Resilience4j, Hystrix, PollyPattern: Retry with Exponential Backoff
Purpose: Retry failed requests with increasing delays.
@Retry(
maxAttempts = 3,
backoff = @Backoff(
delay = 1000, // 1s initial
multiplier = 2, // 1s, 2s, 4s
maxDelay = 10000
)
)
public Order getOrder(String id) {
return orderClient.getOrder(id);
}
Add jitter to prevent thundering herd:
delay = base_delay * (2 ^ attempt) + random(0, 1000)Pattern: Bulkhead
Purpose: Isolate resources to prevent one failure affecting others.
Thread Pool Isolation:
payment-service:
thread_pool_size: 10
queue_size: 20
inventory-service:
thread_pool_size: 20
queue_size: 50
If payment-service threads exhaust, inventory-service unaffected.
@Bulkhead(
name = "payment-service",
type = Bulkhead.Type.THREADPOOL,
maxThreadPoolSize = 10
)Pattern: Rate Limiting
Purpose: Protect services from overload.
Strategies:
1. Token Bucket:
- Tokens refill at fixed rate
- Request consumes token
- Burst capacity allowed
2. Leaky Bucket:
- Requests queued
- Processed at fixed rate
- Queue overflow rejected
3. Fixed Window:
- 100 requests per minute
- Counter resets each minute
4. Sliding Window:
- More accurate
- Prevents burst at window boundary
Implementation:
@RateLimiter(
name = "api",
limitForPeriod = 100,
limitRefreshPeriod = "1m"
)Pattern: Timeout
Purpose: Prevent indefinite waiting.
Timeout Hierarchy:
Client → (5s) → API Gateway → (3s) → Service A → (1s) → Service B
Each layer has shorter timeout than caller.
RestTemplate:
.setConnectTimeout(2000)
.setReadTimeout(5000)
HTTP Client:
HttpClient.newBuilder()
.connectTimeout(Duration.ofSeconds(2))
.build()Pattern: Fallback
Purpose: Provide alternative response when primary fails.
Strategies:
1. Cached Response:
try {
return productService.getProduct(id);
} catch (Exception e) {
return cache.get(id); // Return stale data
}
2. Default Value:
try {
return recommendationService.getRecommendations(userId);
} catch (Exception e) {
return DEFAULT_RECOMMENDATIONS; // Popular products
}
3. Degraded Functionality:
try {
return fullUserProfile(id);
} catch (Exception e) {
return basicUserProfile(id); // Partial data
}
4. Fail Silent:
try {
analyticsService.trackEvent(event);
} catch (Exception e) {
log.error("Analytics unavailable", e);
// Continue without tracking
}Pattern: Health Check
Purpose: Monitor service availability and readiness.
Endpoints:
1. Liveness Probe:
GET /health/live
Returns: 200 if service is running
K8s action: Restart pod if failing
2. Readiness Probe:
GET /health/ready
Returns: 200 if service can handle traffic
K8s action: Remove from load balancer if failing
Example:
{
"status": "UP",
"checks": [
{
"name": "database",
"status": "UP",
"responseTime": "15ms"
},
{
"name": "redis",
"status": "UP",
"responseTime": "3ms"
},
{
"name": "payment-service",
"status": "DOWN",
"error": "Connection timeout"
}
]
}Resilience Patterns Combination
Recommended Stack (use together):
Request flow with resilience:
1. Timeout (prevent hanging)
↓
2. Circuit Breaker (fail fast if service down)
↓
3. Retry with Backoff (handle transient failures)
↓
4. Bulkhead (isolate resources)
↓
5. Rate Limiter (protect from overload)
↓
6. Fallback (graceful degradation)
Configuration example (Resilience4j):
@CircuitBreaker(name = "payment-service", fallbackMethod = "paymentFallback")
@Retry(name = "payment-service")
@RateLimiter(name = "payment-service")
@Bulkhead(name = "payment-service")
@TimeLimiter(name = "payment-service")
public PaymentResult processPayment(Payment payment) {
return paymentClient.process(payment);
}
public PaymentResult paymentFallback(Payment payment, Exception e) {
// Queue for async processing
return PaymentResult.queued(payment.getId());
}Failure Mode Analysis
Cascading Failures
Problem: Failure in one service spreads to others.
Scenario:
1. Database slow query (10s)
2. Service A threads blocked waiting
3. Service A stops responding
4. Service B calls timeout
5. Service B threads blocked
6. Service B stops responding
7. Entire system down
Prevention:
- Timeouts at every layer
- Circuit breakers on all external calls
- Bulkhead isolation
- Fast failure detectionThundering Herd
Problem: Many clients retry simultaneously after failure.
Scenario:
1. Service goes down
2. Circuit breakers open
3. Service recovers
4. All circuit breakers try at once
5. Service overwhelmed again
Prevention:
- Jittered retry delays
- Gradual circuit breaker half-open (limited requests)
- Rate limiting
- Backoff multiplierSlow Response = No Response
Problem: Slow responses worse than failures.
Impact:
- Fast failure: 1 thread blocked for 2s = recoverable
- Slow response: 100 threads blocked for 60s = service down
Prevention:
- Aggressive timeouts (2-5s max)
- Monitor p99 latency, not just average
- Fail fast with circuit breakersResilience Testing
Chaos Engineering
Practice: Intentionally inject failures to test resilience.
Scenarios to test:
1. Service Unavailable:
- Kill service instances
- Verify circuit breakers open
- Verify fallbacks work
2. Latency Injection:
- Add 5s delay to responses
- Verify timeouts trigger
- Verify no thread exhaustion
3. Network Partition:
- Block network between services
- Verify graceful degradation
4. Resource Exhaustion:
- Spike traffic 10x
- Verify rate limiting works
- Verify bulkheads isolate
Tools:
- Chaos Monkey (Netflix)
- Gremlin
- Chaos Mesh (Kubernetes)
- Litmus (Kubernetes)Resilience Metrics
Monitor these metrics:
Circuit Breaker:
- circuit_breaker_state{service="payment"} → CLOSED/OPEN/HALF_OPEN
- circuit_breaker_failures_total{service="payment"}
- circuit_breaker_calls_total{service="payment",result="success/failure"}
Retry:
- retry_attempts_total{service="payment"}
- retry_success_rate{service="payment"}
Timeout:
- timeout_total{service="payment"}
- request_duration_seconds{service="payment",quantile="0.99"}
Bulkhead:
- bulkhead_available_concurrent_calls{service="payment"}
- bulkhead_max_concurrent_calls{service="payment"}
Rate Limiter:
- rate_limiter_allowed_total{service="payment"}
- rate_limiter_rejected_total{service="payment"}Best Practices
1. Defense in Depth - Use multiple resilience patterns together 2. Fail Fast - Don't wait for timeouts, use circuit breakers 3. Graceful Degradation - Provide fallbacks, not errors 4. Monitor Everything - Track circuit breaker states, retry counts, timeouts 5. Test Failures - Chaos engineering in staging/production 6. Tune Thresholds - Based on SLOs and actual traffic patterns 7. Document Behavior - What happens when dependencies fail? 8. Timeouts Everywhere - Every network call must have timeout 9. Idempotent Operations - Safe to retry without side effects 10. Circuit Breaker per Dependency - Isolate failures
Common Mistakes
❌ No Timeouts - Threads blocked indefinitely ❌ No Circuit Breakers - Cascading failures spread ❌ Synchronous Retries - Blocking caller during retry ❌ No Jitter - Thundering herd on retry ❌ Shared Thread Pools - One slow dependency affects all ❌ Ignoring Partial Failures - Treat as complete failures ❌ No Fallbacks - Errors propagate to users ❌ Testing Only Happy Path - Never tested failure scenarios
Tools and Libraries
Java
- Resilience4j: Circuit breaker, retry, rate limiter, bulkhead, timeout
- Hystrix: Circuit breaker (deprecated, use Resilience4j)
- Spring Retry: Retry with backoff
.NET
- Polly: Circuit breaker, retry, timeout, fallback, bulkhead
Go
- go-resiliency: Circuit breaker, retry, timeout
- gobreaker: Circuit breaker
JavaScript/TypeScript
- opossum: Circuit breaker
- cockatiel: Retry, circuit breaker, timeout
Platform-Level
- Istio/Linkerd: Service mesh with built-in resilience
- Envoy: Proxy with circuit breaking, retries, timeouts
- Kong: API gateway with rate limiting, circuit breaking
Further Reading
- "Release It!" by Michael Nygard
- "Site Reliability Engineering" by Google
- netflix.github.io/Hystrix (concepts still valuable)
- resilience4j.readme.io
- principlesofchaos.org
Service Decomposition Patterns
Strategies for breaking down monoliths and defining service boundaries.
Pattern 1: Decompose by Business Capability
Definition: Organize services around business capabilities, not technical layers.
Example:
Business Capabilities → Services
Order Management:
- Order Service (create, track, cancel orders)
- Fulfillment Service (pick, pack, ship)
Customer Management:
- Customer Profile Service
- Customer Preferences Service
Inventory:
- Stock Management Service
- Warehouse ServiceBenefits:
- Services aligned with business domains
- Clear ownership boundaries
- Easier to understand and maintain
- Teams organized around business capabilities
- Stable over time (business capabilities change slowly)
Trade-offs:
- Requires deep business understanding
- May need refactoring as business evolves
- Service boundaries can be subjective
Process: 1. Identify business capabilities (what the business does) 2. Group related capabilities 3. Define service per capability 4. Validate with domain experts
Pattern 2: Decompose by Subdomain (DDD)
Definition: Use Domain-Driven Design to identify bounded contexts as service boundaries.
Example:
E-commerce Domain:
Core Subdomains (competitive advantage):
- Product Catalog Service
- Order Processing Service
- Pricing Engine Service
Supporting Subdomains:
- Customer Service
- Notification Service
Generic Subdomains (buy vs. build):
- Payment Gateway (integrate Stripe)
- Shipping (integrate FedEx/UPS)DDD Concepts:
- Bounded Context: Clear boundary for a model's applicability
- Ubiquitous Language: Shared vocabulary within a context
- Context Map: Relationships between bounded contexts
- Aggregates: Consistency boundaries within a service
Bounded Context Indicators:
- Different language/terminology
- Different business rules
- Independent rate of change
- Different data models
Process: 1. Perform domain analysis (Event Storming workshop) 2. Identify bounded contexts 3. Map context relationships 4. Create service per bounded context
Pattern 3: Decompose by Transaction
Definition: Group operations that need to be ACID transactions into a service.
Example:
Order Service includes:
- Create Order
- Reserve Inventory
- Calculate Total
- Apply Discount
Why? These operations need to be atomic and consistent.When to Use:
- Operations require strong consistency
- Complex business rules span multiple entities
- Avoid distributed transactions
Trade-offs:
- ✅ Strong consistency within service
- ❌ May create larger services
- ❌ Can conflict with business capability alignment
Service Boundary Validation
Checklist for Good Service Boundaries
✓ Single Responsibility
- Service has one reason to change
- Clear, focused purpose
- No overlapping concerns
✓ Independent Deployability
- Can deploy without coordinating with other teams
- Breaking changes don't affect other services
- Versioned APIs for backward compatibility
✓ Data Ownership
- Service owns its data exclusively
- No shared databases
- Clear data access patterns
✓ Team Ownership
- One team owns the service
- Team can make decisions independently
- Clear accountability
✓ Minimal Coupling
- Few dependencies on other services
- Async communication preferred
- Well-defined contracts
Anti-patterns to Avoid
❌ Distributed Monolith
- Services must deploy together
- Tight coupling through shared data
- Synchronous call chains
❌ Anemic Services
- Service is just a CRUD wrapper
- No business logic
- No clear responsibility
❌ God Service
- Service does too much
- Multiple unrelated responsibilities
- Becomes a bottleneck
Service Sizing Guidelines
Micro vs Macro Services
Microservices (small, focused):
- Single business capability
- Small team ownership (2-pizza team)
- Quick to understand and modify
- May require more orchestration
Macroservices (larger, self-contained):
- Multiple related capabilities
- Reduced inter-service communication
- Simpler operational overhead
- May be harder to understand
Right Size:
"A service should be as small as possible but as large as necessary"
- Focus on clear boundaries, not arbitrary size
Size Indicators
Too Small (consider merging):
- Excessive inter-service communication
- Always deploy together
- Shared data models
- No independent value
Too Large (consider splitting):
- Multiple teams working on same service
- Frequent merge conflicts
- Unrelated features bundled together
- Performance bottlenecks
Tools and Techniques
Event Storming
Workshop technique to discover domain events and boundaries:
- Gather domain experts
- Identify domain events (past tense verbs)
- Group events into bounded contexts
- Define service boundaries
Context Mapping
Visualize relationships between services:
- Upstream/Downstream dependencies
- Customer/Supplier relationships
- Shared Kernel (shared code/data)
- Anti-Corruption Layer (translation between contexts)
Service Blueprint
Document service architecture:
- Service responsibilities
- Dependencies (upstream/downstream)
- Data ownership
- API contracts
Decision Framework
Questions to Ask
1. What business capability does this serve?
- Aligns service with business organization
2. Who owns this domain?
- Defines team boundaries
3. What data does this need exclusive access to?
- Determines data ownership
4. What must be strongly consistent?
- Groups transactions appropriately
5. What changes together?
- Identifies coupling
6. What scales independently?
- Separates different scalability needs
Further Reading
- "Domain-Driven Design" by Eric Evans
- "Implementing Domain-Driven Design" by Vaughn Vernon
- "Building Microservices" by Sam Newman
- microservices.io/patterns/decomposition
# Microservices Patterns Skill Quality Rubric
version: "1.0.0"
skill_name: microservices-patterns
evaluated_date: "2026-01-05"
dimensions:
clarity:
weight: 25
description: "Clear explanation of distributed system concepts"
criteria:
- "Service decomposition strategies are clear"
- "Communication patterns explained with diagrams"
- "Saga patterns have step-by-step examples"
- "Circuit breaker logic is visual"
completeness:
weight: 25
description: "Comprehensive microservices coverage"
criteria:
- "Covers all communication patterns (sync/async)"
- "Includes service discovery"
- "Has data management strategies"
- "Covers observability (logging, tracing, metrics)"
accuracy:
weight: 30
description: "Correct distributed systems guidance"
criteria:
- "CAP theorem correctly applied"
- "Event sourcing patterns are accurate"
- "Saga compensation logic is correct"
- "Security between services is sound"
usefulness:
weight: 20
description: "Practical microservices implementation"
criteria:
- "Framework-specific examples (Spring, Go, Node)"
- "Docker/Kubernetes integration"
- "Testing distributed systems"
- "Migration from monolith covered"
passing_criteria:
minimum_score: 3.5
target_score: 4.0
exceptional_score: 4.5
required_dimensions:
- accuracy
blocking_issues:
- "Incorrect CAP theorem application"
- "Missing compensation in saga examples"
- "Insecure service-to-service auth"
scoring_guide:
clarity:
"1": "Distributed concepts confusing"
"2": "Hard to follow service interactions"
"3": "Understandable with effort"
"4": "Clear with good diagrams"
"5": "Exceptional visual explanations"
accuracy:
"1": "Fundamentally wrong distributed advice"
"2": "CAP/consistency issues"
"3": "Mostly correct patterns"
"4": "Accurate, battle-tested patterns"
"5": "Production-proven, comprehensive"