Cicd Pipeline Generator

CI/CD Pipeline Generator

Overview

Generate production-ready CI/CD pipeline configuration files for various platforms (GitHub Actions, GitLab CI, CircleCI, Jenkins). This skill provides templates and guidance for setting up automated workflows that handle linting, testing, building, and deployment for modern web applications, particularly Node.js/Next.js projects.

Core Capabilities

1. Platform Selection

Choose the appropriate CI/CD platform based on project requirements:

• GitHub Actions: Best for GitHub-hosted projects with native integration
• GitLab CI/CD: Ideal for GitLab repositories with complex pipeline needs
• CircleCI: Optimized for Docker workflows and fast build times
• Jenkins: Suitable for self-hosted, highly customizable environments

Refer to references/platform-comparison.md for detailed platform comparisons, pros/cons, and use case recommendations.

2. Pipeline Configuration Generation

Generate pipeline configs following these principles:

Pipeline Stages

Structure pipelines with these standard stages:

1. Install Dependencies

• Checkout code from repository
• Setup runtime environment (Node.js version)
• Restore cached dependencies
• Install dependencies with npm ci
• Cache dependencies for future runs

2. Lint

• Run ESLint for code quality
• Run TypeScript type checking
• Fail fast on linting errors

3. Test

• Execute unit tests
• Execute integration tests
• Generate code coverage reports
• Upload coverage to reporting services (Codecov, Coveralls)

4. Build

• Create production build
• Verify build succeeds
• Store build artifacts

5. Deploy

• Deploy to staging (develop branch)
• Deploy to production (main branch)
• Run post-deployment smoke tests

Caching Strategy

Implement effective caching to speed up builds:

# Cache node_modules based on package-lock.json
cache:
  key: ${{ hashFiles('package-lock.json') }}
  paths:
    - node_modules/
    - .npm/

Environment Variables

Configure necessary environment variables:

• NODE_ENV: Set to production for builds
• Platform-specific tokens: Store as secrets
• Build-time variables: Pass to build process

3. Template Usage

Use provided templates from assets/ directory:

GitHub Actions Template (assets/github-actions-nodejs.yml):

• Multi-job workflow with lint, test, build, deploy
• Matrix builds for multiple Node.js versions (optional)
• Vercel deployment integration
• Artifact uploading
• Code coverage reporting

GitLab CI Template (assets/gitlab-ci-nodejs.yml):

• Multi-stage pipeline
• Dependency caching
• Manual production deployment
• Automatic staging deployment
• Coverage reporting

To use a template: 1. Copy the appropriate template file 2. Place in the correct location:

• GitHub Actions: .github/workflows/ci.yml
• GitLab CI: .gitlab-ci.yml

3. Customize deployment targets, environment variables, and branch names 4. Add required secrets to platform settings

4. Deployment Configuration

Vercel Deployment

For GitHub Actions:

- uses: amondnet/vercel-action@v25
  with:
    vercel-token: ${{ secrets.VERCEL_TOKEN }}
    vercel-org-id: ${{ secrets.VERCEL_ORG_ID }}
    vercel-project-id: ${{ secrets.VERCEL_PROJECT_ID }}
    vercel-args: '--prod'

Required Secrets:

• VERCEL_TOKEN: Get from Vercel account settings
• VERCEL_ORG_ID: From Vercel project settings
• VERCEL_PROJECT_ID: From Vercel project settings

Netlify Deployment

- run: |
    npm install -g netlify-cli
    netlify deploy --prod --dir=.next
  env:
    NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
    NETLIFY_SITE_ID: ${{ secrets.NETLIFY_SITE_ID }}

AWS S3 + CloudFront

- uses: aws-actions/configure-aws-credentials@v4
  with:
    aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
    aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
    aws-region: us-east-1

- run: |
    aws s3 sync .next/static s3://${{ secrets.S3_BUCKET }}/static
    aws cloudfront create-invalidation --distribution-id ${{ secrets.CF_DIST_ID }} --paths "/*"

5. Testing Integration

Configure test execution with proper reporting:

Jest Configuration:

- name: Run tests with coverage
  run: npm test -- --coverage --coverageReporters=text --coverageReporters=lcov

- name: Upload coverage
  uses: codecov/codecov-action@v4
  with:
    files: ./coverage/lcov.info
    flags: unittests

Fail Fast Strategy:

# Run quick tests first
jobs:
  lint:  # Fails in ~30 seconds
  test:  # Fails in ~2 minutes
  build: # Fails in ~5 minutes
    needs: [lint, test]
  deploy:
    needs: [build]

6. Branch-Based Workflows

Implement different behaviors per branch:

Feature Branches / PRs:

• Run lint + test only
• No deployment
• Add PR comments with test results

Develop Branch:

• Run lint + test + build
• Deploy to staging environment
• Automatic deployment

Main Branch:

• Run lint + test + build
• Deploy to production
• Manual approval (optional)
• Create release tags

Example:

deploy_staging:
  if: github.ref == 'refs/heads/develop'
  # Deploy to staging

deploy_production:
  if: github.ref == 'refs/heads/main'
  environment: production  # Requires manual approval
  # Deploy to production

Workflow Decision Tree

Follow this decision tree to generate the appropriate pipeline:

1. Which platform?

• GitHub → Use assets/github-actions-nodejs.yml
• GitLab → Use assets/gitlab-ci-nodejs.yml
• CircleCI/Jenkins → Adapt GitHub Actions template
• Unsure → Consult references/platform-comparison.md

2. What stages are needed?

• Always include: Lint, Test, Build
• Optional: Security scanning, E2E tests, performance tests
• Add deployment stage if deploying from CI

3. Which deployment platform?

• Vercel → Use Vercel deployment examples
• Netlify → Use Netlify CLI approach
• AWS → Use AWS Actions/CLI
• Custom → Implement custom deployment script

4. What triggers?

• On push to main/develop
• On pull request
• On tag creation
• Manual workflow dispatch

5. What environment variables needed?

• Platform tokens (Vercel, Netlify, AWS)
• API keys for external services
• Build-time environment variables
• Feature flags

Best Practices

Security

• Store all secrets in platform secret management (never in code)
• Use least-privilege tokens (read-only when possible)
• Rotate secrets regularly
• Audit secret access permissions
• Never log secrets (use *** masking)

Performance

• Cache dependencies aggressively
• Parallelize independent jobs
• Use matrix builds for multi-version testing
• Fail fast: Run quick checks before slow ones
• Optimize Docker layer caching

Reliability

• Pin exact Node.js versions (18.x not just 18)
• Commit lockfiles (package-lock.json)
• Add retry logic for flaky external services
• Set reasonable timeouts (10-15 minutes max)
• Use continue-on-error for non-critical steps

Maintainability

• Add comments explaining complex logic
• Use reusable workflows/templates
• Keep configs DRY (Don't Repeat Yourself)
• Version control all pipeline changes
• Document required secrets in README

Common Patterns

Multi-Environment Deployment

deploy_staging:
  environment: staging
  if: github.ref == 'refs/heads/develop'

deploy_production:
  environment: production
  if: github.ref == 'refs/heads/main'
  needs: [deploy_staging]

Matrix Testing

strategy:
  matrix:
    node-version: [16.x, 18.x, 20.x]
    os: [ubuntu-latest, windows-latest]

Conditional Steps

- name: Deploy
  if: github.event_name == 'push' && github.ref == 'refs/heads/main'
  run: npm run deploy

Artifact Management

- name: Upload build
  uses: actions/upload-artifact@v4
  with:
    name: build-output
    path: .next/
    retention-days: 7

- name: Download build
  uses: actions/download-artifact@v4
  with:
    name: build-output

Troubleshooting

Pipeline Failures

1. Check action/job logs for error messages 2. Verify environment variables and secrets are set 3. Test commands locally before adding to pipeline 4. Check for platform-specific issues in documentation

Slow Builds

1. Verify cache is working (check cache hit/miss logs) 2. Parallelize independent jobs 3. Use faster runners if available 4. Optimize dependency installation

Deployment Failures

1. Verify deployment tokens are valid 2. Check platform status pages 3. Review deployment logs 4. Test deployment commands locally

Resources

Templates (assets/)

• github-actions-nodejs.yml: Complete GitHub Actions workflow
• gitlab-ci-nodejs.yml: Complete GitLab CI pipeline

Reference Documentation (references/)

• platform-comparison.md: Detailed comparison of CI/CD platforms, deployment targets, best practices, and common patterns

Example Usage

User Request: "Create a GitHub Actions workflow that runs tests and deploys to Vercel"

Steps: 1. Copy assets/github-actions-nodejs.yml template 2. Create .github/workflows/ directory if it doesn't exist 3. Save as .github/workflows/ci.yml 4. Update deployment section with Vercel credentials 5. Add secrets to GitHub repository settings:

• VERCEL_TOKEN
• VERCEL_ORG_ID
• VERCEL_PROJECT_ID

6. Commit and push to trigger workflow

User Request: "Set up GitLab CI with staging and production environments"

Steps: 1. Copy assets/gitlab-ci-nodejs.yml template 2. Save as .gitlab-ci.yml in repository root 3. Configure GitLab CI/CD variables:

• VERCEL_TOKEN
• Other deployment credentials

4. Review manual approval settings for production 5. Commit to trigger pipeline

Advanced Configuration

Monorepo Support

paths:
  - 'apps/frontend/**'
  - 'packages/**'

Scheduled Runs

on:
  schedule:
    - cron: '0 2 * * *'  # Daily at 2 AM

External Service Integration

- name: Notify Slack
  uses: 8398a7/action-slack@v3
  with:
    status: ${{ job.status }}
    webhook_url: ${{ secrets.SLACK_WEBHOOK }}

Security Scanning

- name: Run security audit
  run: npm audit --audit-level=moderate

- name: Check for vulnerabilities
  uses: snyk/actions/node@master
  env:
    SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}

CI/CD Platform Comparison

This reference provides detailed information about different CI/CD platforms to help choose the right configuration.

Platform Overview

GitHub Actions

Best For: Projects hosted on GitHub, simple to complex workflows Pricing: Free for public repos, 2000 mins/month for private repos Configuration File: .github/workflows/*.yml

Pros:

• Native GitHub integration
• Large marketplace of actions
• Matrix builds for multiple environments
• Excellent caching support
• Built-in secrets management

Cons:

• Can be expensive for heavy CI usage
• Limited to GitHub-hosted projects

Common Use Cases:

• Automated testing on PR
• Deploy to Vercel/Netlify
• Publish npm packages
• Release automation

GitLab CI/CD

Best For: GitLab-hosted projects, complex pipelines Pricing: Free tier available, paid tiers for advanced features Configuration File: .gitlab-ci.yml

Pros:

• Built into GitLab
• Powerful pipeline visualization
• Extensive deployment options
• Auto DevOps features
• Kubernetes integration

Cons:

• GitLab-specific
• Steeper learning curve
• Resource limits on free tier

Common Use Cases:

• Multi-stage deployments
• Kubernetes deployments
• Container registry integration
• Advanced testing strategies

CircleCI

Best For: Fast builds, Docker-first workflows Pricing: Free tier with limits, paid plans available Configuration File: .circleci/config.yml

Pros:

• Very fast build times
• Excellent Docker support
• SSH debugging
• Orbs for reusable configs
• Works with GitHub, Bitbucket, GitLab

Cons:

• Free tier limitations
• Can be complex for simple projects

Common Use Cases:

• Docker-based applications
• High-frequency builds
• Microservices
• Fast feedback loops

Jenkins

Best For: Self-hosted, highly customizable pipelines Pricing: Free (self-hosted) Configuration File: Jenkinsfile

Pros:

• Completely free
• Highly customizable
• Massive plugin ecosystem
• Full control over infrastructure

Cons:

• Requires maintenance
• Infrastructure costs
• Setup complexity
• UI feels dated

Common Use Cases:

• Enterprise environments
• On-premise deployments
• Complex custom workflows
• Legacy system integration

Configuration Patterns

Node.js/Next.js Applications

Key Steps: 1. Install Dependencies: npm ci (faster, more reliable than npm install) 2. Lint: npm run lint (catch code quality issues) 3. Test: npm test (run unit/integration tests) 4. Build: npm run build (create production build) 5. Deploy: Platform-specific deployment commands

Caching Strategy:

• Cache node_modules/ directory
• Cache npm cache directory
• Use lockfile for cache key

Environment Variables:

• NODE_ENV=production
• API keys/tokens via secrets
• Build-time environment variables

Common Pipeline Stages

1. Install Stage

- Checkout code
- Setup Node.js
- Restore cache (if exists)
- Run npm ci
- Save cache

2. Lint Stage

- Restore dependencies from cache
- Run ESLint
- Run TypeScript type checking

3. Test Stage

- Restore dependencies from cache
- Run unit tests
- Run integration tests
- Generate coverage report
- Upload coverage to reporting service

4. Build Stage

- Restore dependencies from cache
- Run production build
- Store build artifacts

5. Deploy Stage

- Download build artifacts
- Deploy to hosting platform
- Run smoke tests
- Notify team

Deployment Targets

Vercel

Best For: Next.js, React, static sites Setup:

• Install Vercel CLI or use GitHub integration
• Set VERCEL_TOKEN, VERCEL_ORG_ID, VERCEL_PROJECT_ID
• Deploy: vercel --prod or use GitHub Action

Features:

• Zero-config for Next.js
• Preview deployments for PRs
• Automatic HTTPS
• Edge functions support

Netlify

Best For: Static sites, JAMstack apps Setup:

• Install Netlify CLI
• Set NETLIFY_AUTH_TOKEN, NETLIFY_SITE_ID
• Deploy: netlify deploy --prod

Features:

• Built-in forms
• Split testing
• Branch previews
• Serverless functions

AWS (S3 + CloudFront)

Best For: Scalable static hosting Setup:

• Configure AWS credentials
• Build application
• Sync to S3: aws s3 sync ./build s3://bucket-name
• Invalidate CloudFront cache

Features:

• Unlimited scalability
• Full AWS integration
• Cost-effective at scale
• Global CDN

Docker Registry

Best For: Containerized applications Setup:

• Build Docker image
• Tag image with version
• Push to registry (Docker Hub, ECR, GCR)

Commands:

docker build -t app:$VERSION .
docker tag app:$VERSION registry/app:$VERSION
docker push registry/app:$VERSION

Best Practices

Security

1. Never commit secrets: Use platform secret management 2. Limit secret access: Only expose secrets to necessary jobs 3. Use read-only tokens: When possible, use minimal permissions 4. Rotate secrets regularly: Especially for long-lived projects 5. Audit access: Review who has access to secrets

Performance

1. Cache dependencies: Dramatically speeds up builds 2. Parallelize jobs: Run independent jobs concurrently 3. Fail fast: Run quick jobs first (lint before build) 4. Use matrix builds: Test multiple versions in parallel 5. Optimize Docker layers: Cache expensive operations

Reliability

1. Pin versions: Specify exact Node.js versions 2. Use lockfiles: Commit package-lock.json 3. Handle failures: Use continue-on-error or retry logic 4. Set timeouts: Prevent hanging builds 5. Monitor pipelines: Alert on failures

Maintainability

1. Keep configs DRY: Use reusable workflows/templates 2. Document decisions: Add comments explaining complex logic 3. Version control: Track changes to pipeline configs 4. Test changes: Use separate branches for pipeline updates 5. Review regularly: Remove unused jobs and optimize

Common Patterns

Feature Branch Workflow

PR opened → Lint + Test → Build
PR merged → Lint + Test + Build + Deploy to staging
Push to main → Deploy to production

Gitflow Workflow

develop branch → Deploy to staging
main branch → Deploy to production
hotfix/* → Deploy to hotfix environment
release/* → Deploy to UAT

Trunk-Based Development

All commits to main → Test + Build
Tag created → Deploy to production

Troubleshooting

Slow Builds

• Check cache configuration
• Parallelize independent jobs
• Use faster runners
• Optimize dependencies

Flaky Tests

• Increase timeouts
• Add retry logic
• Mock external dependencies
• Use deterministic test data

Failed Deployments

• Check environment variables
• Verify credentials/tokens
• Review deployment logs
• Test locally first

Cache Issues

• Verify cache key configuration
• Check cache size limits
• Clear and rebuild cache
• Use more specific cache keys