Cicd Pipelines

CI/CD Pipelines

Production CI/CD with GitHub Actions, testing automation, and deployment strategies.

Quick Start

# .github/workflows/ci.yml
name: CI Pipeline

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'
          cache: 'pip'

      - name: Install dependencies
        run: |
          pip install -r requirements.txt
          pip install -r requirements-dev.txt

      - name: Run linting
        run: ruff check .

      - name: Run type checking
        run: mypy src/

      - name: Run tests
        run: pytest tests/ --cov=src --cov-report=xml

      - name: Upload coverage
        uses: codecov/codecov-action@v3
        with:
          file: coverage.xml

Core Concepts

1. Complete CI/CD Pipeline

# .github/workflows/deploy.yml
name: Deploy Pipeline

on:
  push:
    branches: [main]

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: pip install -r requirements.txt && pytest

  build:
    needs: test
    runs-on: ubuntu-latest
    outputs:
      image_tag: ${{ steps.meta.outputs.tags }}
    steps:
      - uses: actions/checkout@v4

      - name: Log in to registry
        uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Extract metadata
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
          tags: |
            type=sha,prefix=
            type=ref,event=branch

      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

  deploy-staging:
    needs: build
    runs-on: ubuntu-latest
    environment: staging
    steps:
      - name: Deploy to staging
        run: |
          kubectl set image deployment/app \
            app=${{ needs.build.outputs.image_tag }}

  deploy-production:
    needs: [build, deploy-staging]
    runs-on: ubuntu-latest
    environment: production
    steps:
      - name: Deploy to production
        run: |
          kubectl set image deployment/app \
            app=${{ needs.build.outputs.image_tag }}

2. Matrix Testing

jobs:
  test:
    strategy:
      matrix:
        python-version: ['3.10', '3.11', '3.12']
        os: [ubuntu-latest, macos-latest]
        database: [postgres, mysql]
        exclude:
          - os: macos-latest
            database: mysql
    runs-on: ${{ matrix.os }}
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: test
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - run: pytest --db=${{ matrix.database }}

3. Reusable Workflows

# .github/workflows/python-ci.yml (reusable)
name: Python CI

on:
  workflow_call:
    inputs:
      python-version:
        required: false
        type: string
        default: '3.12'

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ inputs.python-version }}
      - run: pip install -r requirements.txt && pytest

# Usage in another workflow
jobs:
  ci:
    uses: ./.github/workflows/python-ci.yml
    with:
      python-version: '3.11'

4. Deployment Strategies

# Blue-Green Deployment
deploy:
  steps:
    - name: Deploy to green
      run: |
        kubectl apply -f k8s/deployment-green.yaml
        kubectl rollout status deployment/app-green

    - name: Run smoke tests
      run: ./scripts/smoke-test.sh $GREEN_URL

    - name: Switch traffic
      run: |
        kubectl patch service app \
          -p '{"spec":{"selector":{"version":"green"}}}'

    - name: Cleanup blue
      run: kubectl delete deployment app-blue

# Canary Deployment
deploy-canary:
  steps:
    - name: Deploy canary (10%)
      run: |
        kubectl apply -f k8s/deployment-canary.yaml
        kubectl scale deployment/app-canary --replicas=1
        kubectl scale deployment/app-stable --replicas=9

    - name: Monitor canary
      run: ./scripts/monitor-canary.sh --duration=30m

    - name: Promote or rollback
      run: |
        if [ "$CANARY_SUCCESS" == "true" ]; then
          kubectl scale deployment/app-canary --replicas=10
          kubectl scale deployment/app-stable --replicas=0
        else
          kubectl delete deployment/app-canary
        fi

Tools & Technologies

Troubleshooting Guide

Best Practices

# ✅ DO: Cache dependencies
- uses: actions/cache@v4
  with:
    path: ~/.cache/pip
    key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }}

# ✅ DO: Use environments for deployments
environment: production

# ✅ DO: Pin action versions
- uses: actions/checkout@v4  # Not @main

# ✅ DO: Add timeouts
jobs:
  test:
    timeout-minutes: 10

# ❌ DON'T: Store secrets in code
# ❌ DON'T: Skip tests for faster deployments

Resources

• GitHub Actions Docs
• GitLab CI Docs
• ArgoCD Docs

---

Skill Certification Checklist:

• [ ] Can create CI pipelines with testing
• [ ] Can build and push Docker images
• [ ] Can deploy to Kubernetes
• [ ] Can implement deployment strategies
• [ ] Can create reusable workflows

Cicd Pipelines Guide

Overview

This guide provides comprehensive documentation for the cicd-pipelines skill in the custom-plugin-data-engineer plugin.

Category: Devops

Quick Start

Prerequisites

• Familiarity with devops concepts
• Development environment set up
• Plugin installed and configured

Basic Usage

# Invoke the skill
claude "cicd-pipelines - [your task description]"

# Example
claude "cicd-pipelines - analyze the current implementation"

Core Concepts

Key Principles

1. Consistency - Follow established patterns 2. Clarity - Write readable, maintainable code 3. Quality - Validate before deployment

Best Practices

• Always validate input data
• Handle edge cases explicitly
• Document your decisions
• Write tests for critical paths

Common Tasks

Task 1: Basic Implementation

# Example implementation pattern
def implement_cicd_pipelines(input_data):
    """
    Implement cicd-pipelines functionality.

    Args:
        input_data: Input to process

    Returns:
        Processed result
    """
    # Validate input
    if not input_data:
        raise ValueError("Input required")

    # Process
    result = process(input_data)

    # Return
    return result

Task 2: Advanced Usage

For advanced scenarios, consider:

• Configuration customization via assets/config.yaml
• Validation using scripts/validate.py
• Integration with other skills

Troubleshooting

Common Issues

Related Resources

• SKILL.md - Skill specification
• config.yaml - Configuration options
• validate.py - Validation script

---

Last updated: 2025-12-30

Cicd Pipelines Patterns

Design Patterns

Pattern 1: Input Validation

Always validate input before processing:

def validate_input(data):
    if data is None:
        raise ValueError("Data cannot be None")
    if not isinstance(data, dict):
        raise TypeError("Data must be a dictionary")
    return True

Pattern 2: Error Handling

Use consistent error handling:

try:
    result = risky_operation()
except SpecificError as e:
    logger.error(f"Operation failed: {e}")
    handle_error(e)
except Exception as e:
    logger.exception("Unexpected error")
    raise

Pattern 3: Configuration Loading

Load and validate configuration:

import yaml

def load_config(config_path):
    with open(config_path) as f:
        config = yaml.safe_load(f)
    validate_config(config)
    return config

Anti-Patterns to Avoid

❌ Don't: Swallow Exceptions

# BAD
try:
    do_something()
except:
    pass

✅ Do: Handle Explicitly

# GOOD
try:
    do_something()
except SpecificError as e:
    logger.warning(f"Expected error: {e}")
    return default_value

Category-Specific Patterns: Devops

Recommended Approach

1. Start with the simplest implementation 2. Add complexity only when needed 3. Test each addition 4. Document decisions

Common Integration Points

• Configuration: assets/config.yaml
• Validation: scripts/validate.py
• Documentation: references/GUIDE.md

---

Pattern library for cicd-pipelines skill