
Gitlab Ci Validator
- 416 installs
- 286 repo stars
- Updated July 26, 2026
- akin-ozer/cc-devops-skills
gitlab-ci-validator is a Claude Code skill that validates GitLab CI YAML for schema, job graph, best practices, and security for developers who need to stop broken pipelines before they waste runner time.
About
gitlab-ci-validator is a DevOps skill in akin-ozer/cc-devops-skills that orchestrates a 6-gate validation workflow on .gitlab-ci.yml through validate_gitlab_ci.sh. Required gates cover syntax and security; recommended gates add best practices; optional --test-only runs gitlab-ci-local when Docker is available. Three Python validators check syntax, best practices, and security with severity levels from critical through suggestion. Developers reach for gitlab-ci-validator when editing stages, includes, or job dependencies and need pre-merge CI review on GitLab.
- Validate GitLab CI YAML before push
- Detect job dependency and syntax issues
- Prevent broken pipelines from blocking releases
- Save runner minutes on bad configs
- Improve CI maintainability across repos
Gitlab Ci Validator by the numbers
- 416 all-time installs (skills.sh)
- Ranked #286 of 1,435 DevOps & CI/CD skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/akin-ozer/cc-devops-skills --skill gitlab-ci-validatorAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 416 |
|---|---|
| repo stars | ★ 286 |
| Last updated | July 26, 2026 |
| Repository | akin-ozer/cc-devops-skills ↗ |
How do you validate GitLab CI YAML before pipeline runs?
Validate GitLab CI YAML for schema, job graph, and common misconfigurations before pipeline runs waste runner time or block releases.
Who is it for?
Developers editing GitLab CI configs who want a structured 6-gate validation workflow before merge.
Skip if: Repositories on GitHub Actions or Azure Pipelines without a .gitlab-ci.yml file.
When should I use this skill?
A developer changes .gitlab-ci.yml or asks to validate, lint, or security-review a GitLab pipeline.
What you get
Reviewed .gitlab-ci.yml with syntax, security, and best-practice findings ranked by critical through suggestion severity.
- Pipeline YAML issue report
- Severity-ranked security and best-practice findings
By the numbers
- 6-gate deterministic validation workflow from syntax through strict mode
- 3 dedicated Python validators: validate_syntax.py, check_best_practices.py, check_security.py
Files
GitLab CI/CD Validator
Comprehensive toolkit for validating, linting, testing, and securing .gitlab-ci.yml configurations.
Trigger Phrases
Use this skill when requests include intent like:
- "Validate this
.gitlab-ci.yml" - "Why is this GitLab pipeline failing?"
- "Run a security review for our GitLab CI"
- "Check pipeline best practices"
- "Lint GitLab CI config before merge"
Setup And Prerequisites (Run First)
All commands below assume repository root as current working directory.
# Ensure validator scripts are executable
chmod +x devops-skills-plugin/skills/gitlab-ci-validator/scripts/*.sh \
devops-skills-plugin/skills/gitlab-ci-validator/scripts/*.py
# Required runtime
python3 --versionUse one canonical command path for orchestration:
VALIDATOR="bash devops-skills-plugin/skills/gitlab-ci-validator/scripts/validate_gitlab_ci.sh"Optional local execution tooling (for --test-only):
bash devops-skills-plugin/skills/gitlab-ci-validator/scripts/install_tools.shQuick Start Commands
# 1) Full validation (syntax + best practices + security)
$VALIDATOR .gitlab-ci.yml
# 2) Syntax and schema only (required first gate)
$VALIDATOR .gitlab-ci.yml --syntax-only
# 3) Best-practices only (recommended)
$VALIDATOR .gitlab-ci.yml --best-practices
# 4) Security only (required before merge)
$VALIDATOR .gitlab-ci.yml --security-only
# 5) Optional local pipeline structure test (needs gitlab-ci-local + Docker)
$VALIDATOR .gitlab-ci.yml --test-only
# 6) Strict mode (treat best-practice warnings as failure)
$VALIDATOR .gitlab-ci.yml --strictDeterministic Validation Workflow
Follow these gates in order:
1. Run Quick Start command 2 (--syntax-only). 2. If syntax fails, stop and fix errors before continuing. 3. Run Quick Start command 3 (--best-practices) and apply relevant improvements. 4. Run Quick Start command 4 (--security-only) and fix all critical/high findings before merge. 5. Optionally run Quick Start command 5 (--test-only) for local execution checks. 6. Run Quick Start command 6 (--strict) for final merge gate.
Required gates: syntax + security. Recommended gate: best practices. Optional gate: local execution test.
Rule Severity Rationale And Documentation Links
Severity Model
critical: Direct credential/secret exposure or high-confidence compromise path. Block merge.high: Exploitable unsafe behavior or strong security regression. Fix before merge.medium: Security hardening gap with realistic risk. Track and fix soon.low/suggestion: Optimization or maintainability improvement.
Rule Classes And Why They Matter
- Syntax rules (
yaml-syntax,job-stage-undefined,dependencies-undefined-job): prevent pipeline parse and dependency failures. - Best-practice rules (
cache-missing,artifact-no-expiration,dag-optimization): reduce runtime cost and improve pipeline throughput. - Security rules (
hardcoded-password,curl-pipe-bash,include-remote-unverified): reduce credential leaks and supply-chain risk.
References
- Local syntax reference:
devops-skills-plugin/skills/gitlab-ci-validator/docs/gitlab-ci-reference.md - Local best practices:
devops-skills-plugin/skills/gitlab-ci-validator/docs/best-practices.md - Local common issues:
devops-skills-plugin/skills/gitlab-ci-validator/docs/common-issues.md - GitLab CI YAML reference: https://docs.gitlab.com/ee/ci/yaml/
- GitLab CI/CD components: https://docs.gitlab.com/ee/ci/components/
- GitLab pipeline security guidance: https://docs.gitlab.com/ee/ci/pipelines/settings.html
Fallbacks For Tool Or Environment Constraints
- Missing
python3: - Behavior: validator cannot run.
- Fallback: install Python 3 and rerun.
- Missing
PyYAML: - Behavior:
python_wrapper.shauto-creates.venvand installspyyamlwhen possible. - Fallback in restricted/offline environments: pre-install
pyyamlfrom an internal mirror, then rerun. - Missing
gitlab-ci-local,node, ordocker: - Behavior:
--test-onlyreports warning/failure. - Fallback: skip local execution testing and continue with syntax/best-practice/security gates.
- No execute permission on scripts:
- Behavior: shell permission errors.
- Fallback: rerun the setup
chmodcommand from the Setup section.
Examples
Example 1: New Pipeline Validation
$VALIDATOR examples/basic-pipeline.gitlab-ci.yml --syntax-only
$VALIDATOR examples/basic-pipeline.gitlab-ci.yml --security-onlyExample 2: Pre-Merge Hard Gate
$VALIDATOR .gitlab-ci.yml --strictExample 3: CI Integration
stages:
- validate
validate_gitlab_ci:
stage: validate
script:
- chmod +x devops-skills-plugin/skills/gitlab-ci-validator/scripts/*.sh devops-skills-plugin/skills/gitlab-ci-validator/scripts/*.py
- bash devops-skills-plugin/skills/gitlab-ci-validator/scripts/validate_gitlab_ci.sh .gitlab-ci.yml --strictIndividual Validators (Advanced)
# Syntax validator (via wrapper for PyYAML fallback)
bash devops-skills-plugin/skills/gitlab-ci-validator/scripts/python_wrapper.sh \
devops-skills-plugin/skills/gitlab-ci-validator/scripts/validate_syntax.py .gitlab-ci.yml
# Best-practices validator
bash devops-skills-plugin/skills/gitlab-ci-validator/scripts/python_wrapper.sh \
devops-skills-plugin/skills/gitlab-ci-validator/scripts/check_best_practices.py .gitlab-ci.yml
# Security validator
bash devops-skills-plugin/skills/gitlab-ci-validator/scripts/python_wrapper.sh \
devops-skills-plugin/skills/gitlab-ci-validator/scripts/check_security.py .gitlab-ci.ymlDone Criteria
- Frontmatter
nameanddescriptionunchanged. - One canonical orchestrator path is used consistently.
- Setup and
chmodprerequisites appear before workflow/use examples. - Quick-start and workflow are non-duplicative (workflow references quick-start gates).
- Severity rationale and rule-to-doc references are explicit.
- Fallback behavior is documented for missing tools and constrained environments.
- Examples are executable from repository root.
Notes
- This skill validates configuration and static patterns; it does not execute production pipelines.
- Use
gitlab-ci-localor GitLab CI Lint for runtime behavior confirmation.
# Tools directory (created by install_tools.sh)
.tools/
# Node modules (if gitlab-ci-local installed locally)
node_modules/
package-lock.json
GitLab CI/CD Best Practices
Pipeline Design
Use Stages Effectively
Organize jobs into logical stages that represent your development workflow:
stages:
- .pre # Setup and validation
- build # Compilation and asset generation
- test # Testing and quality checks
- scan # Security scanning
- deploy # Deployment
- .post # Cleanup and notificationsLeverage DAG with needs
Create directed acyclic graphs to run jobs as soon as their dependencies complete:
stages:
- build
- test
- deploy
build_frontend:
stage: build
script: npm run build:frontend
build_backend:
stage: build
script: go build ./cmd/server
test_frontend:
stage: test
needs: [build_frontend] # Starts immediately after build_frontend
script: npm test
test_backend:
stage: test
needs: [build_backend] # Runs in parallel with test_frontend
script: go test ./...
deploy:
stage: deploy
needs:
- test_frontend
- test_backend
script: ./deploy.shBenefits:
- Faster pipeline execution
- Parallel job execution
- Reduced waiting time
Use rules Instead of only/except
The rules keyword is more powerful and flexible:
# ❌ Deprecated approach
deploy_job:
script: ./deploy.sh
only:
- main
- tags
except:
- schedules
# ✅ Modern approach
deploy_job:
script: ./deploy.sh
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
- if: '$CI_COMMIT_TAG'
- if: '$CI_PIPELINE_SOURCE == "schedule"'
when: neverPerformance Optimization
Implement Effective Caching
Cache dependencies to avoid repeated downloads:
variables:
CACHE_VERSION: "v1"
.npm_cache:
cache:
key: ${CACHE_VERSION}-${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
- .npm/
policy: pull-push
install_deps:
extends: .npm_cache
script:
- npm ci --cache .npm
test_job:
extends: .npm_cache
cache:
policy: pull # Only download, don't upload
needs: [install_deps]
script:
- npm testBest practices:
- Use version prefixes in cache keys for invalidation
- Use
pullpolicy for read-only jobs - Cache package manager files (.npm, .pip, .gem)
- Don't cache artifacts (use
artifactsinstead)
Optimize Artifact Usage
Only save what you need and set appropriate expiration:
build_job:
script:
- npm run build
artifacts:
paths:
- dist/
- public/
exclude:
- dist/**/*.map # Exclude source maps if not needed
expire_in: 1 week # Clean up old artifacts
test_job:
script:
- npm test
artifacts:
paths:
- coverage/
expire_in: 2 days
when: always # Save even on failure
reports:
coverage_report:
coverage_format: cobertura
path: coverage/cobertura.xmlUse Parallel Execution
Speed up testing with parallel jobs:
test_job:
script:
- npm test -- --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL
parallel: 5
# Or with matrix for multiple configurations
test_matrix:
script:
- npm test
parallel:
matrix:
- NODE_VERSION: ['16', '18', '20']
OS: ['ubuntu-latest', 'macos-latest']Make Jobs Interruptible
Allow automatic cancellation of redundant jobs:
test_job:
script:
- npm test
interruptible: true # Cancel if newer pipeline starts
deploy_production:
script:
- ./deploy.sh
interruptible: false # Never cancel production deploymentsSecurity Best Practices
Never Hardcode Secrets
# ❌ NEVER do this
deploy_job:
script:
- export AWS_SECRET_KEY="AKIAIOSFODNN7EXAMPLE"
- ./deploy.sh
# ✅ Use CI/CD variables or secrets managers
deploy_job:
script:
- ./deploy.sh
variables:
AWS_REGION: "us-east-1"
secrets:
AWS_SECRET_KEY:
vault: production/aws/credentials@opsPin Docker Image Versions
Always use specific versions or SHA digests:
# ❌ Avoid using latest tags
build_job:
image: node:latest
script: npm build
# ✅ Pin to specific versions
build_job:
image: node:18.17.0-alpine
script: npm build
# ✅ Even better: Use SHA digest
build_job:
image: node@sha256:a6385a6bb2fdcb7c48fc871e35e32af8daaa82c518f934fcd0e5a42c0dd6ed71
script: npm buildMask Sensitive Variables
Protect sensitive information in logs:
variables:
PUBLIC_API_URL: "https://api.example.com"
# In GitLab UI, mark these as:
# - Protected (only available on protected branches)
# - Masked (hidden in logs)
# - Hidden (not visible in settings)Validate External Inputs
When using pipeline variables or inputs, validate them:
validate_input:
stage: .pre
script:
- |
if [[ ! "$DEPLOY_ENV" =~ ^(staging|production)$ ]]; then
echo "Invalid DEPLOY_ENV: $DEPLOY_ENV"
exit 1
fiLock Dependencies
Pin exact versions to avoid supply chain attacks:
# For npm
install_deps:
script:
- npm ci # Uses package-lock.json
# For Python with hash verification
install_deps:
script:
- pip install -r requirements.txt --require-hashes
# For Go
verify_deps:
script:
- go mod verifyPin Include References
Reference specific commits or protected tags:
# ❌ Avoid branch references
include:
- project: 'my-group/templates'
file: '/templates/.gitlab-ci.yml'
ref: main
# ✅ Use specific commit SHAs
include:
- project: 'my-group/templates'
file: '/templates/.gitlab-ci.yml'
ref: 'a1b2c3d4e5f6'
# ✅ Or protected tags
include:
- project: 'my-group/templates'
file: '/templates/.gitlab-ci.yml'
ref: 'v1.2.3'Code Organization
Use Templates and Extends
Create reusable job templates:
.base_deploy:
stage: deploy
script:
- ./deploy.sh
retry:
max: 2
when:
- runner_system_failure
- stuck_or_timeout_failure
before_script:
- echo "Deploying to $ENVIRONMENT"
deploy_staging:
extends: .base_deploy
variables:
ENVIRONMENT: staging
environment:
name: staging
url: https://staging.example.com
deploy_production:
extends: .base_deploy
variables:
ENVIRONMENT: production
environment:
name: production
url: https://example.com
when: manual
only:
- mainUse YAML Anchors
Reduce repetition with YAML anchors:
.default_retry: &default_retry
retry:
max: 2
when:
- runner_system_failure
- stuck_or_timeout_failure
.node_cache: &node_cache
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
- .npm/
build_job:
<<: *default_retry
<<: *node_cache
script:
- npm run buildOrganize with Include
Split large configurations into multiple files:
# .gitlab-ci.yml
include:
- local: '.gitlab/ci/build.yml'
- local: '.gitlab/ci/test.yml'
- local: '.gitlab/ci/deploy.yml'
- local: '.gitlab/ci/security.yml'
stages:
- build
- test
- deployResource Management
Set Appropriate Timeouts
Prevent jobs from hanging indefinitely:
# Project default: Set in UI under Settings > CI/CD
# Job-specific timeout
long_running_job:
script:
- ./long_process.sh
timeout: 3h
quick_job:
script:
- ./quick_check.sh
timeout: 5mUse Resource Groups
Prevent concurrent deployments:
deploy_production:
stage: deploy
script:
- ./deploy.sh
resource_group: production
environment:
name: productionControl Runner Selection
Use tags to select appropriate runners:
build_job:
tags:
- docker
- high-cpu
script:
- make build
deploy_job:
tags:
- deployment
- protected
script:
- ./deploy.shError Handling
Use allow_failure Strategically
# Experimental features that shouldn't block pipeline
experimental_test:
script:
- ./experimental_feature_test.sh
allow_failure: true
# Allow specific exit codes
integration_test:
script:
- ./integration_tests.sh
allow_failure:
exit_codes:
- 137 # OOM killed
- 143 # SIGTERMImplement Retry Logic
Retry on transient failures:
flaky_test:
script:
- npm run e2e
retry:
max: 2
when:
- runner_system_failure
- stuck_or_timeout_failure
- unknown_failureUse after_script for Cleanup
Ensure cleanup happens regardless of job status:
integration_test:
services:
- postgres:14
script:
- ./run_tests.sh
after_script:
- ./cleanup_test_data.shEnvironment Management
Use Dynamic Environments
Create review apps for merge requests:
deploy_review:
stage: deploy
script:
- ./deploy_review_app.sh
environment:
name: review/$CI_COMMIT_REF_SLUG
url: https://$CI_ENVIRONMENT_SLUG.example.com
on_stop: stop_review
auto_stop_in: 3 days
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
stop_review:
stage: deploy
script:
- ./stop_review_app.sh
environment:
name: review/$CI_COMMIT_REF_SLUG
action: stop
when: manual
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'Use Environment-Specific Variables
variables:
GLOBAL_VAR: "value"
deploy_staging:
stage: deploy
variables:
DEPLOY_URL: "https://staging.example.com"
DEBUG_MODE: "true"
script:
- ./deploy.sh
environment:
name: staging
deploy_production:
stage: deploy
variables:
DEPLOY_URL: "https://example.com"
DEBUG_MODE: "false"
script:
- ./deploy.sh
environment:
name: productionTesting Best Practices
Separate Test Types
Organize tests by scope and speed:
unit_test:
stage: test
script:
- npm run test:unit
artifacts:
reports:
junit: junit.xml
coverage_report:
coverage_format: cobertura
path: coverage/cobertura.xml
integration_test:
stage: test
script:
- npm run test:integration
needs: [build_job]
e2e_test:
stage: test
script:
- npm run test:e2e
needs: [deploy_staging]
allow_failure: true # E2E tests can be flakyUse Test Reports
Leverage GitLab's test report features:
test_job:
script:
- npm test
artifacts:
when: always
reports:
junit: test-results.xml
coverage_report:
coverage_format: cobertura
path: coverage/cobertura.xmlDocumentation
Comment Your Pipeline
Add clear comments explaining complex logic:
# This job deploys to production only on the main branch
# and requires manual approval for safety
deploy_production:
stage: deploy
script:
- ./deploy.sh
environment:
name: production
when: manual
rules:
# Only run on main branch
- if: '$CI_COMMIT_BRANCH == "main"'
# Skip on scheduled pipelines
- if: '$CI_PIPELINE_SOURCE == "schedule"'
when: neverUse Meaningful Job Names
Choose descriptive names that explain purpose:
# ❌ Unclear names
job1:
script: npm test
job2:
script: ./script.sh
# ✅ Clear names
unit_tests:
script: npm run test:unit
deploy_to_staging:
script: ./deploy.sh stagingWorkflow Optimization
Control Pipeline Creation
Prevent unnecessary pipelines:
workflow:
rules:
# Run for merge requests
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
# Run for main branch
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
# Run for tags
- if: '$CI_COMMIT_TAG'
# Don't run otherwise
- when: neverUse Changes Detection
Run jobs only when relevant files change:
frontend_test:
script:
- npm run test:frontend
rules:
- changes:
- frontend/**/*
- package.json
- package-lock.json
backend_test:
script:
- go test ./...
rules:
- changes:
- backend/**/*.go
- go.mod
- go.sumMaintenance
Version Your Pipeline
Track pipeline configuration changes:
# Add version comments
# Pipeline version: 2.1.0
# Last updated: 2025-01-15
# Changelog: Added security scanning job
variables:
PIPELINE_VERSION: "2.1.0"Regular Audits
Periodically review and optimize:
1. Remove unused jobs and stages 2. Update Docker image versions 3. Review cache effectiveness 4. Check artifact storage usage 5. Update deprecated keywords 6. Review and update security practices
Monitor Pipeline Performance
Track key metrics:
- Pipeline duration
- Success rate
- Cache hit rate
- Artifact storage usage
- Runner queue times
- Job failure rates
Use GitLab's analytics features to identify bottlenecks and optimization opportunities.
Common GitLab CI/CD Issues and Solutions
Syntax Errors
Invalid YAML Syntax
Problem: YAML formatting errors prevent pipeline execution.
Common causes:
- Inconsistent indentation (mixing tabs and spaces)
- Missing colons after keys
- Incorrect list formatting
- Unquoted special characters
Examples:
# ❌ Wrong indentation
job_name:
script:
- echo "test"
# ✅ Correct indentation
job_name:
script:
- echo "test"
# ❌ Missing colon
job_name
script:
- echo "test"
# ✅ Correct syntax
job_name:
script:
- echo "test"
# ❌ Unquoted special characters
job_name:
script:
- echo $VAR: value
# ✅ Quoted special characters
job_name:
script:
- echo "$VAR: value"Solution: Use a YAML linter or GitLab's CI Lint tool to validate syntax.
Reserved Keyword as Job Name
Problem: Using reserved keywords as job names causes validation errors.
Reserved keywords:
image,services,stages,typesbefore_script,after_script,variablescache,include,pages,default,workflow
# ❌ Using reserved keyword
image:
stage: build
script:
- echo "build"
# ✅ Use a different name
build_image:
stage: build
script:
- echo "build"Job Configuration Issues
Missing script Keyword
Problem: Every job must have a script section (except some special jobs like trigger).
# ❌ Missing script
test_job:
stage: test
# ✅ With script
test_job:
stage: test
script:
- npm testUndefined Stage Reference
Problem: Job references a stage that doesn't exist in stages definition.
# ❌ Undefined stage
stages:
- build
- test
deploy_job:
stage: deploy # 'deploy' stage not defined
script:
- ./deploy.sh
# ✅ Stage defined
stages:
- build
- test
- deploy
deploy_job:
stage: deploy
script:
- ./deploy.shInvalid Job Dependencies
Problem: Referencing non-existent jobs in dependencies or needs.
# ❌ References non-existent job
test_job:
stage: test
dependencies:
- build_job # This job doesn't exist
script:
- npm test
# ✅ Valid dependency
build_job:
stage: build
script:
- npm run build
artifacts:
paths:
- dist/
test_job:
stage: test
dependencies:
- build_job
script:
- npm testCircular Dependencies with needs
Problem: Creating circular dependencies with needs keyword.
# ❌ Circular dependency
job_a:
needs: [job_b]
script: echo "A"
job_b:
needs: [job_a]
script: echo "B"
# ✅ Valid DAG
job_a:
script: echo "A"
job_b:
needs: [job_a]
script: echo "B"Variable Issues
Undefined Variable Reference
Problem: Referencing variables that don't exist.
# ❌ Undefined variable
deploy_job:
script:
- echo "Deploying to $UNDEFINED_ENV"
# ✅ Define variable
variables:
DEPLOY_ENV: "staging"
deploy_job:
script:
- echo "Deploying to $DEPLOY_ENV"Variable Scope Issues
Problem: Variables not available in expected scope.
# ❌ Job variable not available globally
job_a:
variables:
MY_VAR: "value"
script:
- echo $MY_VAR
job_b:
script:
- echo $MY_VAR # Not available here
# ✅ Use global variable
variables:
MY_VAR: "value"
job_a:
script:
- echo $MY_VAR
job_b:
script:
- echo $MY_VARHardcoded Secrets
Problem: Sensitive data exposed in pipeline configuration.
# ❌ Hardcoded credentials
deploy_job:
script:
- export API_KEY="sk_live_1234567890"
- ./deploy.sh
# ✅ Use CI/CD variables or secrets
deploy_job:
script:
- ./deploy.sh # API_KEY from CI/CD variables
secrets:
API_KEY:
vault: production/api/key@opsArtifact and Cache Issues
Artifacts Not Passed Between Jobs
Problem: Jobs can't access files from previous jobs.
# ❌ No artifacts defined
build_job:
stage: build
script:
- npm run build
test_job:
stage: test
script:
- ls dist/ # Directory doesn't exist
# ✅ With artifacts
build_job:
stage: build
script:
- npm run build
artifacts:
paths:
- dist/
expire_in: 1 hour
test_job:
stage: test
needs: [build_job]
script:
- ls dist/ # Now availableCache Not Working
Problem: Dependencies downloaded on every job run.
Common causes:
- Wrong cache paths
- Incorrect cache key
- Cache policy misconfiguration
- Runner doesn't support caching
# ❌ Wrong cache configuration
test_job:
cache:
paths:
- node_modules/ # Wrong path or not created
script:
- npm ci
- npm test
# ✅ Correct cache configuration
variables:
npm_config_cache: "$CI_PROJECT_DIR/.npm"
test_job:
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- .npm/
- node_modules/
script:
- npm ci --cache .npm --prefer-offline
- npm testCache vs Artifacts Confusion
Problem: Using cache for job outputs instead of artifacts.
# ❌ Using cache for build outputs
build_job:
cache:
paths:
- dist/ # Should be artifacts
script:
- npm run build
# ✅ Correct usage
build_job:
cache:
paths:
- node_modules/ # Dependencies (cache)
artifacts:
paths:
- dist/ # Build outputs (artifacts)
script:
- npm ci
- npm run buildRules and Conditions Issues
Conflicting Rules
Problem: Multiple rules that contradict each other.
# ❌ Conflicting rules
deploy_job:
script:
- ./deploy.sh
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
when: always
- if: '$CI_COMMIT_BRANCH == "main"'
when: never # Conflicts with above
# ✅ Clear rules
deploy_job:
script:
- ./deploy.sh
rules:
- if: '$CI_COMMIT_BRANCH == "main" && $CI_PIPELINE_SOURCE != "schedule"'
when: on_success
- when: neverMixing rules with only/except
Problem: Cannot use rules with only/except in the same job.
# ❌ Mixing rules and only/except
deploy_job:
script:
- ./deploy.sh
only:
- main
rules:
- if: '$CI_COMMIT_TAG'
# ✅ Use rules only
deploy_job:
script:
- ./deploy.sh
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
- if: '$CI_COMMIT_TAG'Incorrect changes Usage
Problem: changes not working as expected.
# ❌ Changes with wrong pipeline source
test_job:
script:
- npm test
rules:
- changes:
- src/**/*.js
# Won't work on branch pipelines without if condition
# ✅ Correct changes usage
test_job:
script:
- npm test
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
changes:
- src/**/*.js
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'Docker and Service Issues
Image Pull Failures
Problem: Cannot pull Docker images.
Common causes:
- Image doesn't exist
- Authentication required
- Network issues
- Using
:latestwithout recent pull
# ❌ Non-existent or inaccessible image
test_job:
image: mycompany/nonexistent:latest
script:
- npm test
# ✅ Valid, accessible image
test_job:
image: node:18-alpine
script:
- npm test
# ✅ Private registry with authentication
test_job:
image: registry.gitlab.com/mygroup/myimage:v1.0
before_script:
- echo $CI_REGISTRY_PASSWORD | docker login -u $CI_REGISTRY_USER --password-stdin $CI_REGISTRY
script:
- npm testService Connection Issues
Problem: Cannot connect to services (databases, etc.).
# ❌ Wrong service alias or missing variables
test_job:
image: node:18
services:
- postgres:14
script:
- npm run test:integration # Connection fails
# ✅ Proper service configuration
test_job:
image: node:18
services:
- name: postgres:14
alias: postgres
variables:
POSTGRES_DB: testdb
POSTGRES_USER: test
POSTGRES_PASSWORD: test
DATABASE_URL: "postgres://test:test@postgres:5432/testdb"
script:
- npm run test:integrationUsing :latest Tag
Problem: Unpredictable behavior with :latest tags.
# ❌ Using latest tag
build_job:
image: node:latest # Could change unexpectedly
script:
- npm run build
# ✅ Pin specific version
build_job:
image: node:18.17.0-alpine
script:
- npm run build
# ✅ Even better: Use SHA digest
build_job:
image: node@sha256:a6385a6bb2fdcb7c48fc871e35e32af8daaa82c518f934fcd0e5a42c0dd6ed71
script:
- npm run buildPerformance Issues
Slow Pipeline Execution
Problem: Pipelines take too long to complete.
Solutions:
1. Use caching:
.node_cache:
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
- .npm/
build_job:
extends: .node_cache
script:
- npm ci --cache .npm
- npm run build2. Use `needs` for parallel execution:
# Instead of sequential stages
build_a:
stage: build
script: make build_a
build_b:
stage: build # Runs in parallel with build_a
script: make build_b
test_a:
stage: test
needs: [build_a] # Starts immediately after build_a
script: make test_a3. Use parallel jobs:
test_job:
script:
- npm test -- --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL
parallel: 5Cache Miss Rate High
Problem: Cache frequently invalidated or not used.
# ❌ Cache key changes too often
test_job:
cache:
key: $CI_COMMIT_SHA # Different for every commit
paths:
- node_modules/
# ✅ Stable cache key
test_job:
cache:
key: ${CI_COMMIT_REF_SLUG}-${CI_PROJECT_DIR}/package-lock.json
paths:
- node_modules/Downloading Same Artifacts Multiple Times
Problem: Multiple jobs downloading same artifacts unnecessarily.
# ❌ All artifacts downloaded
build_a:
artifacts:
paths:
- dist_a/
build_b:
artifacts:
paths:
- dist_b/
test_job:
needs: [build_a, build_b]
script:
- test dist_a/ # Only needs dist_a
# ✅ Use dependencies to control downloads
test_job:
needs:
- build_a
- build_b
dependencies:
- build_a # Only download from build_a
script:
- test dist_a/Security Issues
Secrets in Logs
Problem: Sensitive data visible in job logs.
# ❌ Secrets printed to logs
deploy_job:
script:
- echo "API Key: $API_KEY" # Visible in logs
- ./deploy.sh
# ✅ Mask variables and avoid printing
deploy_job:
script:
- ./deploy.sh
# Mark API_KEY as masked in CI/CD settingsUnpinned Dependencies
Problem: Vulnerable or malicious dependencies could be installed.
# ❌ Unpinned dependencies
install_job:
script:
- npm install # Could install different versions
# ✅ Locked dependencies
install_job:
script:
- npm ci # Uses package-lock.json
# ✅ With hash verification (Python)
install_job:
script:
- pip install -r requirements.txt --require-hashesInsecure Script Patterns
Problem: Scripts vulnerable to injection or other attacks.
# ❌ Command injection risk
deploy_job:
script:
- curl $EXTERNAL_URL | bash # Dangerous
# ✅ Verify and validate
deploy_job:
script:
- curl -o script.sh $EXTERNAL_URL
- sha256sum -c script.sh.sha256
- bash script.shEnvironment and Deployment Issues
Environment Not Created
Problem: Deployment environments not showing in GitLab UI.
# ❌ Missing environment keyword
deploy_job:
script:
- ./deploy.sh staging
# ✅ With environment
deploy_job:
script:
- ./deploy.sh staging
environment:
name: staging
url: https://staging.example.comManual Jobs Not Stopping Pipeline
Problem: Pipeline continues without waiting for manual job.
# ❌ Pipeline continues
deploy_staging:
script:
- ./deploy.sh
when: manual
deploy_production:
needs: [deploy_staging] # Starts immediately
script:
- ./deploy.sh
# ✅ Use allow_failure: false
deploy_staging:
script:
- ./deploy.sh
when: manual
allow_failure: false # Pipeline waits
deploy_production:
needs: [deploy_staging]
script:
- ./deploy.shReview App Cleanup Issues
Problem: Review apps not automatically stopped.
# ❌ No cleanup
deploy_review:
script:
- ./deploy_review.sh
environment:
name: review/$CI_COMMIT_REF_SLUG
url: https://$CI_ENVIRONMENT_SLUG.example.com
# ✅ With auto-stop and stop job
deploy_review:
script:
- ./deploy_review.sh
environment:
name: review/$CI_COMMIT_REF_SLUG
url: https://$CI_ENVIRONMENT_SLUG.example.com
on_stop: stop_review
auto_stop_in: 3 days
stop_review:
script:
- ./stop_review.sh
environment:
name: review/$CI_COMMIT_REF_SLUG
action: stop
when: manualRunner Issues
No Runners Available
Problem: Jobs stuck in "pending" state.
Solutions:
- Check runner tags match job tags
- Verify runners are online and not paused
- Check runner capacity and queue
- Review runner permissions for project
# If job requires specific tags
build_job:
tags:
- docker
- linux
script:
- make build
# Ensure runners with these tags are available and activeRunner Timeout
Problem: Jobs fail due to timeout.
# ❌ Default timeout too short
long_running_job:
script:
- ./long_process.sh # Takes > 1 hour
# ✅ Increase timeout
long_running_job:
script:
- ./long_process.sh
timeout: 3hInclude and Template Issues
Include File Not Found
Problem: Cannot find included file.
# ❌ Wrong path
include:
- local: 'templates/ci.yml' # Missing leading slash
# ✅ Correct path
include:
- local: '/templates/ci.yml' # Absolute path from repo rootCircular Includes
Problem: Files include each other creating a loop.
# File A includes File B
# File B includes File A
# Results in: "Maximum includes depth reached"
# Solution: Restructure includes to avoid circular referencesInclude with Wrong Project Path
Problem: Cannot access files from other projects.
# ❌ Wrong project path or no access
include:
- project: 'wrong-group/wrong-project'
file: '/templates/ci.yml'
# ✅ Correct project path with access
include:
- project: 'my-group/templates'
ref: 'v1.2.3'
file: '/templates/ci.yml'Debugging Tips
Enable Debug Logging
Add debug variables to get more information:
variables:
CI_DEBUG_TRACE: "true" # Enable debug mode
CI_DEBUG_SERVICES: "true" # Debug service connectionsUse echo for Debugging
Print variable values and script execution:
debug_job:
script:
- echo "Branch: $CI_COMMIT_BRANCH"
- echo "Ref: $CI_COMMIT_REF_NAME"
- env | sort # Print all environment variables
- set -x # Enable command tracing
- ./my_script.shTest Locally
Use tools to test pipelines locally:
# Using gitlab-ci-local
npm install -g gitlab-ci-local
gitlab-ci-localUse CI Lint Tool
Validate configuration before committing:
1. Navigate to: CI/CD > Pipeline editor > Validate tab 2. Paste your .gitlab-ci.yml content 3. Review validation results and errors 4. Or use API: POST /api/v4/ci/lint
GitLab CI/CD YAML Reference
Overview
GitLab CI/CD pipelines are defined in .gitlab-ci.yml files using YAML syntax. The file must be located at the root of your repository. The order of keywords is not important unless otherwise specified.
File Structure
# Global configuration
default:
# Default settings for all jobs
include:
# Import external configurations
stages:
# Define pipeline stages
variables:
# Global variables
workflow:
# Pipeline execution rules
# Job definitions
job_name:
stage: stage_name
script:
- command1
- command2Global Keywords
default
Establishes custom default values that are copied to jobs lacking specific keyword definitions.
Supported keywords:
image,services,before_script,after_scriptcache,artifacts,retry,timeout,interruptibletags,hooks
Example:
default:
image: ruby:3.0
retry: 2
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- vendor/include
Imports configuration from external YAML files.
Types:
local: Files in same repositoryproject: Files from other GitLab projectsremote: Files from external URLstemplate: GitLab-provided templatescomponent: CI/CD catalog components
Example:
include:
- local: '/templates/.gitlab-ci-template.yml'
- template: 'Auto-DevOps.gitlab-ci.yml'
- project: 'my-group/my-project'
file: '/templates/.gitlab-ci.yml'
- remote: 'https://example.com/.gitlab-ci.yml'
- component: $CI_SERVER_FQDN/my-org/security/secret-detection@1.0stages
Defines the names and order of pipeline stages. Jobs in the same stage run in parallel.
Default stages (if not defined): 1. .pre 2. build 3. test 4. deploy 5. .post
Example:
stages:
- build
- test
- deploy
- cleanupvariables
Sets CI/CD variables available to all jobs or specific jobs.
Example:
variables:
DATABASE_URL: "postgres://postgres@postgres/db"
DEPLOY_NOTE:
description: "The deployment note"
value: "Default deployment"
job_name:
variables:
DEPLOY_ENV: "production"workflow
Controls pipeline behavior and creation rules.
Example:
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'Job Keywords
Required Keywords
script
The only required keyword. Defines shell commands executed by the runner.
Syntax:
job_name:
script: "single command"
# OR multi-line
job_name:
script:
- command1
- command2
- |
multi-line
command blockExecution Control
before_script
Commands running before the script section.
Example:
job_name:
before_script:
- echo "Preparing environment"
- bundle install
script:
- bundle exec rspecafter_script
Commands running after script completion. Executes in a separate shell context.
Example:
job_name:
script:
- ./deploy.sh
after_script:
- ./cleanup.shstage
Assigns the job to a specific pipeline stage.
Example:
build_job:
stage: build
script:
- make buildwhen
Controls when jobs run.
Values:
on_success(default): Run when all previous jobs succeedon_failure: Run when at least one previous job failsalways: Always runmanual: Require manual actiondelayed: Delay job executionnever: Never run
Example:
cleanup_job:
stage: cleanup
script:
- ./cleanup.sh
when: always
deploy_job:
stage: deploy
script:
- ./deploy.sh
when: manualArtifact Management
artifacts
Specifies files and directories to save after job completion.
Sub-keywords:
paths: File locations to includeexclude: Patterns to excludeexpire_in: Retention duration (default: 30 days)name: Archive namewhen: Upload condition (on_success, on_failure, always)reports: Collect test/coverage/security reports
Example:
test_job:
script:
- npm test
artifacts:
paths:
- coverage/
- dist/
exclude:
- coverage/**/*.tmp
expire_in: 1 week
reports:
junit: test-results.xml
coverage_report:
coverage_format: cobertura
path: coverage/cobertura.xmlcache
Defines files cached between job runs for faster execution.
Sub-keywords:
paths: Items to cachekey: Cache identifierpolicy: Download/upload behavior (pull, push, pull-push)when: Cache condition (on_success, on_failure, always)untracked: Cache untracked files
Example:
build_job:
script:
- npm install
- npm run build
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
- .npm/
policy: pull-push
test_job:
script:
- npm test
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
policy: pullJob Dependencies
dependencies
Restricts artifact downloads to specified jobs only.
Example:
build_job:
stage: build
script:
- make build
artifacts:
paths:
- binaries/
test_job:
stage: test
script:
- ./test.sh
dependencies:
- build_jobneeds
Executes jobs earlier than stage ordering permits, creating a directed acyclic graph (DAG).
Example:
build_job:
stage: build
script:
- make build
test_job:
stage: test
script:
- make test
needs:
- build_job
deploy_job:
stage: deploy
script:
- make deploy
needs:
- test_jobConditional Execution
rules
Determines job creation based on conditions. Replaces only/except.
Example:
deploy_job:
script:
- echo "Deploy to production"
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
when: manual
- if: '$CI_COMMIT_BRANCH == "staging"'
when: on_success
- when: never
test_job:
script:
- npm test
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
- changes:
- src/**/*.js
- test/**/*.jsallow_failure
Permits job failure without stopping the pipeline.
Example:
experimental_test:
script:
- experimental_command
allow_failure: true
# With exit codes
integration_test:
script:
- ./integration_tests.sh
allow_failure:
exit_codes: [137, 255]Environment & Deployment
environment
Specifies deployment target environment.
Sub-keywords:
name: Environment nameurl: Environment URLon_stop: Job to stop environmentauto_stop_in: Auto-stop durationaction: Deployment action (start, prepare, stop)
Example:
deploy_staging:
stage: deploy
script:
- ./deploy.sh staging
environment:
name: staging
url: https://staging.example.com
on_stop: stop_staging
auto_stop_in: 1 day
deploy_review:
stage: deploy
script:
- ./deploy.sh review
environment:
name: review/$CI_COMMIT_REF_SLUG
url: https://$CI_ENVIRONMENT_SLUG.example.com
on_stop: stop_review
auto_stop_in: 1 week
stop_review:
stage: deploy
script:
- ./stop_review.sh
environment:
name: review/$CI_COMMIT_REF_SLUG
action: stop
when: manualContainer Configuration
image
Specifies Docker container image for job execution.
Example:
test_job:
image: node:18-alpine
script:
- npm test
# With specific digest (recommended for security)
secure_job:
image: node@sha256:abc123...
script:
- npm run secure-testservices
Defines Docker service images (databases, cache servers, etc.).
Example:
integration_test:
image: node:18
services:
- name: postgres:14
alias: postgres
- name: redis:7-alpine
alias: cache
variables:
POSTGRES_DB: testdb
POSTGRES_USER: test
POSTGRES_PASSWORD: test
script:
- npm run integration-testResource Management
tags
Selects runners by labels.
Example:
build_job:
tags:
- docker
- linux
deploy_job:
tags:
- kubernetes
- productiontimeout
Sets job-level timeout, overriding project settings.
Example:
long_running_job:
script:
- ./long_process.sh
timeout: 3hresource_group
Limits job concurrency within a resource group.
Example:
deploy_production:
script:
- ./deploy.sh
resource_group: productionparallel
Runs multiple job instances in parallel.
Example:
test_job:
script:
- npm test
parallel: 5
# With matrix
test_matrix:
script:
- bundle exec rspec
parallel:
matrix:
- RUBY_VERSION: ['2.7', '3.0', '3.1']
DATABASE: ['postgres', 'mysql']interruptible
Allows job cancellation when made redundant by newer runs.
Example:
test_job:
script:
- npm test
interruptible: trueAdvanced Features
extends
Inherits configuration from other jobs or templates.
Example:
.default_retry:
retry:
max: 2
when:
- runner_system_failure
- stuck_or_timeout_failure
test_job:
extends: .default_retry
script:
- npm testretry
Auto-retry configuration on failure.
Example:
test_job:
script:
- flaky_test.sh
retry:
max: 2
when:
- runner_system_failure
- stuck_or_timeout_failure
- unknown_failurecoverage
Extracts code coverage metrics via regex.
Example:
test_job:
script:
- npm test
coverage: '/Coverage: \d+\.\d+/'secrets
Specifies required CI/CD secrets from external providers.
Example:
deploy_job:
script:
- ./deploy.sh
secrets:
DATABASE_PASSWORD:
vault: production/db/password@ops
file: false
API_KEY:
vault: production/api/key@opstrigger
Defines downstream pipeline triggers.
Example:
trigger_downstream:
stage: deploy
trigger:
project: my-group/downstream-project
branch: mainrelease
Generates release objects.
Example:
release_job:
stage: release
script:
- echo "Creating release"
release:
tag_name: '$CI_COMMIT_TAG'
name: 'Release $CI_COMMIT_TAG'
description: 'Release notes here'Predefined Variables
Common CI/CD variables available in all pipelines:
CI_COMMIT_BRANCH: Current branch nameCI_COMMIT_SHA: Current commit SHACI_COMMIT_REF_NAME: Branch or tag nameCI_COMMIT_REF_SLUG: Lowercased, shortened to 63 bytesCI_COMMIT_TAG: Commit tag nameCI_DEFAULT_BRANCH: Default branch nameCI_ENVIRONMENT_NAME: Environment nameCI_ENVIRONMENT_SLUG: Simplified environment nameCI_JOB_ID: Job IDCI_JOB_NAME: Job nameCI_JOB_STAGE: Job stageCI_PIPELINE_ID: Pipeline IDCI_PIPELINE_SOURCE: Pipeline trigger sourceCI_PROJECT_DIR: Repository clone directoryCI_PROJECT_ID: Project IDCI_PROJECT_NAME: Project nameCI_PROJECT_PATH: Project pathCI_PROJECT_URL: Project URLCI_REGISTRY: GitLab container registry URLCI_REGISTRY_IMAGE: Container registry image pathCI_RUNNER_ID: Runner IDCI_SERVER_URL: GitLab instance URL
Reserved Keywords
The following keywords cannot be used as job names:
imageservicesstagestypesbefore_scriptafter_scriptvariablescacheincludepagesdefaultworkflow
Validation
Use the GitLab CI Lint tool to validate your configuration:
- Web UI: Navigate to Build > Pipeline editor > Validate tab
- API: POST to
/api/v4/ci/lint - VS Code: Use GitLab Workflow extension
Common Patterns
Anchors and References
.default_cache: &default_cache
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
build_job:
<<: *default_cache
script:
- npm install
- npm run buildHidden Jobs (Templates)
.deploy_template:
script:
- ./deploy.sh
only:
- main
deploy_staging:
extends: .deploy_template
environment:
name: staging
deploy_production:
extends: .deploy_template
environment:
name: production
when: manualBest Practices
1. Use `rules` instead of `only`/`except`: More flexible and powerful 2. Leverage caching: Cache dependencies between jobs 3. Use `needs` for DAG pipelines: Faster execution 4. Pin Docker images: Use specific versions or SHA digests 5. Set artifact expiration: Avoid storage bloat 6. Use templates and extends: DRY principle 7. Define meaningful stage names: Clear pipeline flow 8. Use `interruptible`: Save resources on redundant jobs 9. Implement proper error handling: Use allow_failure appropriately 10. Document your pipeline: Use comments in YAML
stages:
- .pre
- build
- test
- security
- deploy
- .post
variables:
CACHE_VERSION: v1
NODE_VERSION: '18'
validate_config:
stage: .pre
image:
name: alpine:latest
script:
- echo "Validating configuration"
- test -f package.json || exit 1
- test -f Dockerfile || exit 1
tags:
- docker
build_frontend:
stage: build
image:
name: node:${NODE_VERSION}-alpine
script:
- echo "Building frontend"
- cd frontend
- npm ci
- npm run build
artifacts:
paths:
- frontend/dist/
expire_in: 1 day
cache:
- key: ${CACHE_VERSION}-frontend-${CI_COMMIT_REF_SLUG}
paths:
- frontend/node_modules/
- frontend/.npm/
policy: pull-push
when: on_success
tags:
- docker
build_backend:
stage: build
image:
name: golang:1.21-alpine
script:
- echo "Building backend"
- cd backend
- go mod download
- go build -o bin/server ./cmd/server
artifacts:
paths:
- backend/bin/
expire_in: 1 day
cache:
- key: ${CACHE_VERSION}-backend-${CI_COMMIT_REF_SLUG}
paths:
- /go/pkg/mod/
policy: pull-push
when: on_success
tags:
- docker
build_docs:
stage: build
image:
name: python:3.11-alpine
script:
- echo "Building documentation"
- pip install mkdocs mkdocs-material
- mkdocs build
artifacts:
paths:
- site/
expire_in: 1 week
tags:
- docker
test_frontend_unit:
stage: test
image:
name: node:${NODE_VERSION}-alpine
needs:
- job: build_frontend
artifacts: true
optional: false
script:
- echo "Running frontend unit tests"
- cd frontend
- npm ci
- npm run test:unit
artifacts:
reports:
junit: frontend/junit.xml
coverage_report:
coverage_format: cobertura
path: frontend/coverage/cobertura.xml
coverage: /Lines\s*:\s*(\d+\.\d+)%/
tags:
- docker
test_frontend_e2e:
stage: test
image:
name: cypress/included:13.6.0
needs:
- job: build_frontend
artifacts: true
optional: false
services:
- name: nginx:alpine
alias: web
script:
- echo "Running E2E tests"
- cd frontend
- npm ci
- npm run test:e2e
artifacts:
when: always
paths:
- frontend/cypress/screenshots/
- frontend/cypress/videos/
expire_in: 1 week
allow_failure: true
tags:
- docker
test_backend_unit:
stage: test
image:
name: golang:1.21-alpine
needs:
- job: build_backend
artifacts: true
optional: false
script:
- echo "Running backend unit tests"
- cd backend
- go test -v -coverprofile=coverage.out ./...
- go tool cover -func=coverage.out
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: backend/coverage.xml
coverage: /total:.*?(\d+\.\d+)%/
tags:
- docker
test_backend_integration:
stage: test
image:
name: golang:1.21-alpine
needs:
- job: build_backend
artifacts: true
optional: false
services:
- name: postgres:15-alpine
alias: postgres
- name: redis:7-alpine
alias: redis
variables:
POSTGRES_DB: testdb
POSTGRES_USER: test
POSTGRES_PASSWORD: test
DATABASE_URL: postgres://test:test@postgres:5432/testdb?sslmode=disable
REDIS_URL: redis://redis:6379
script:
- echo "Running backend integration tests"
- cd backend
- go test -v -tags=integration ./...
tags:
- docker
lint_frontend:
stage: test
image:
name: node:${NODE_VERSION}-alpine
needs:
- job: build_frontend
artifacts: false
optional: false
script:
- echo "Linting frontend code"
- cd frontend
- npm ci
- npm run lint
allow_failure: true
tags:
- docker
lint_backend:
stage: test
image:
name: golangci/golangci-lint:v1.55-alpine
needs:
- job: build_backend
artifacts: false
optional: false
script:
- echo "Linting backend code"
- cd backend
- golangci-lint run
allow_failure: true
tags:
- docker
sast_scan:
stage: security
image:
name: returntocorp/semgrep:latest
needs: []
script:
- semgrep --config=auto --json --output=sast-report.json .
artifacts:
reports:
sast: sast-report.json
allow_failure: true
tags:
- docker
dependency_scan:
stage: security
image:
name: aquasec/trivy:latest
needs: []
script:
- trivy fs --format json --output dependency-report.json .
artifacts:
paths:
- dependency-report.json
expire_in: 1 week
allow_failure: true
tags:
- docker
secret_scan:
stage: security
image:
name: trufflesecurity/trufflehog:latest
needs: []
script:
- trufflehog filesystem --directory=. --json > secrets-report.json
artifacts:
paths:
- secrets-report.json
expire_in: 1 week
allow_failure: true
tags:
- docker
deploy_staging:
stage: deploy
image:
name: alpine:latest
needs:
- job: test_frontend_unit
artifacts: true
optional: false
- job: test_backend_unit
artifacts: true
optional: false
- job: test_backend_integration
artifacts: true
optional: false
before_script:
- apk add --no-cache curl
script:
- echo "Deploying to staging"
- curl -X POST "$STAGING_DEPLOY_WEBHOOK"
environment:
name: staging
url: https://staging.example.com
on_stop: stop_staging
auto_stop_in: 7 days
rules:
- if: $CI_COMMIT_BRANCH == "develop"
tags:
- docker
stop_staging:
stage: deploy
image:
name: alpine:latest
before_script:
- apk add --no-cache curl
script:
- echo "Stopping staging environment"
- curl -X DELETE "$STAGING_DEPLOY_WEBHOOK"
environment:
name: staging
action: stop
when: manual
rules:
- if: $CI_COMMIT_BRANCH == "develop"
tags:
- docker
deploy_review:
stage: deploy
image:
name: alpine:latest
needs:
- job: test_frontend_unit
artifacts: true
optional: false
- job: test_backend_unit
artifacts: true
optional: false
before_script:
- apk add --no-cache curl
script:
- echo "Deploying review app"
- echo "URL https://$CI_ENVIRONMENT_SLUG.review.example.com"
- curl -X POST "$REVIEW_DEPLOY_WEBHOOK"
environment:
name: review/main
url: https://review-ci-commit-ref-sl.review.example.com
on_stop: stop_review
auto_stop_in: 3 days
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
tags:
- docker
stop_review:
stage: deploy
image:
name: alpine:latest
before_script:
- apk add --no-cache curl
script:
- echo "Stopping review app"
- curl -X DELETE "$REVIEW_DEPLOY_WEBHOOK"
environment:
name: review/main
action: stop
when: manual
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
tags:
- docker
deploy_production:
stage: deploy
image:
name: alpine:latest
needs:
- job: test_frontend_unit
artifacts: true
optional: false
- job: test_backend_unit
artifacts: true
optional: false
- job: test_backend_integration
artifacts: true
optional: false
- job: sast_scan
artifacts: true
optional: false
- job: dependency_scan
artifacts: true
optional: false
before_script:
- apk add --no-cache curl
script:
- echo "Deploying to production"
- curl -X POST "$PRODUCTION_DEPLOY_WEBHOOK"
environment:
name: production
url: https://example.com
rules:
- if: $CI_COMMIT_BRANCH == "main"
when: manual
allow_failure: false
tags:
- docker
notify_deployment:
stage: .post
image:
name: alpine:latest
before_script:
- apk add --no-cache curl
script:
- echo "Sending deployment notification"
- |
curl -X POST "$SLACK_WEBHOOK" \
-H 'Content-Type: application/json' \
-d "{\"text\":\"Pipeline completed: $CI_PIPELINE_URL\"}"
when: always
rules:
- if: $CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH == "develop"
tags:
- docker
default: {}
# To contribute improvements to CI/CD templates, please follow the Development guide at:
# https://docs.gitlab.com/development/cicd/templates/
# This specific template is located at:
# https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Dependency-Scanning.gitlab-ci.yml
# Read more about this feature here: https://docs.gitlab.com/ee/user/application_security/dependency_scanning/
#
# Configure dependency scanning with CI/CD variables (https://docs.gitlab.com/ee/ci/variables/).
# List of available variables: https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#available-cicd-variables
variables:
# Setting this variable affects all Security templates
# (SAST, Dependency Scanning, ...)
SECURE_ANALYZERS_PREFIX: "$CI_TEMPLATE_REGISTRY_HOST/security-products"
#
DS_EXCLUDED_ANALYZERS: ""
DS_EXCLUDED_PATHS: "spec, test, tests, tmp, node_modules"
DS_MAJOR_VERSION: 6
DS_SCHEMA_MODEL: 15
dependency_scanning:
stage: test
script:
- echo "$CI_JOB_NAME is used for configuration only, and its script should not be executed"
- exit 1
artifacts:
access: 'developer'
reports:
dependency_scanning: gl-dependency-scanning-report.json
dependencies: []
rules:
- when: never
.ds-analyzer:
extends: dependency_scanning
allow_failure: true
variables:
# DS_ANALYZER_IMAGE is an undocumented variable used internally to allow QA to
# override the analyzer image with a custom value. This may be subject to change or
# breakage across GitLab releases.
DS_ANALYZER_IMAGE: "$SECURE_ANALYZERS_PREFIX/$DS_ANALYZER_NAME:$DS_MAJOR_VERSION"
# DS_ANALYZER_NAME is an undocumented variable used in job definitions
# to inject the analyzer name in the image name.
DS_ANALYZER_NAME: ""
image:
name: "$DS_ANALYZER_IMAGE$DS_IMAGE_SUFFIX"
# `rules` must be overridden explicitly by each child job
# see https://gitlab.com/gitlab-org/gitlab/-/issues/218444
script:
- /analyzer run
.cyclonedx-reports:
artifacts:
access: 'developer'
paths:
- "**/gl-sbom-*.cdx.json"
reports:
cyclonedx: "**/gl-sbom-*.cdx.json"
.gemnasium-shared-rule:
exists:
- '**/{Gemfile.lock,composer.lock,gems.locked,go.sum,npm-shrinkwrap.json,package-lock.json,yarn.lock,pnpm-lock.yaml,packages.lock.json,conan.lock}'
gemnasium-dependency_scanning:
extends:
- .ds-analyzer
- .cyclonedx-reports
variables:
DS_ANALYZER_NAME: "gemnasium"
GEMNASIUM_LIBRARY_SCAN_ENABLED: "true"
rules:
- if: $DEPENDENCY_SCANNING_DISABLED == 'true' || $DEPENDENCY_SCANNING_DISABLED == '1'
when: never
- if: $DS_EXCLUDED_ANALYZERS =~ /gemnasium([^-]|$)/
when: never
# The following 3 blocks of rules define whether the job runs in a an *MR pipeline* or a *branch pipeline*
# when an MR exists. If the job has additional rules to observe they should be added in the blocks 1 and 3
# to cover both the *MR pipeline* and the *branch pipeline* workflows.
# 1. Run the job in an *MR pipeline* if MR pipelines for AST are enabled and there's an open merge request.
## When FIPS mode is enabled, use the FIPS compatible image
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_PIPELINE_SOURCE == "merge_request_event" &&
$GITLAB_FEATURES =~ /\bdependency_scanning\b/ &&
$CI_GITLAB_FIPS_MODE == "true"
exists: !reference [.gemnasium-shared-rule, exists]
variables:
DS_IMAGE_SUFFIX: "-fips"
DS_REMEDIATE: "false"
## When FIPS mode is not enabled, use the regular image
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_PIPELINE_SOURCE == "merge_request_event" &&
$GITLAB_FEATURES =~ /\bdependency_scanning\b/
exists: !reference [.gemnasium-shared-rule, exists]
# 2. Don't run the job in a *branch pipeline* if *MR pipelines* for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_OPEN_MERGE_REQUESTS
when: never
# 3. Finally, run the job in a *branch pipeline* (When MR pipelines are disabled for AST, or it is enabled but no open MRs exist for the branch).
## When FIPS mode is enabled, use the FIPS compatible image
- if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bdependency_scanning\b/ &&
$CI_GITLAB_FIPS_MODE == "true"
exists: !reference [.gemnasium-shared-rule, exists]
variables:
DS_IMAGE_SUFFIX: "-fips"
DS_REMEDIATE: "false"
## When FIPS mode is not enabled, use the regular image
- if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bdependency_scanning\b/
exists: !reference [.gemnasium-shared-rule, exists]
.gemnasium-maven-shared-rule:
exists:
- '**/{build.gradle,build.gradle.kts,build.sbt,pom.xml}'
gemnasium-maven-dependency_scanning:
extends:
- .ds-analyzer
- .cyclonedx-reports
variables:
DS_ANALYZER_NAME: "gemnasium-maven"
rules:
- if: $DEPENDENCY_SCANNING_DISABLED == 'true' || $DEPENDENCY_SCANNING_DISABLED == '1'
when: never
- if: $DS_EXCLUDED_ANALYZERS =~ /gemnasium-maven/
when: never
# The following 3 blocks of rules define whether the job runs in a an *MR pipeline* or a *branch pipeline*
# when an MR exists. If the job has additional rules to observe they should be added in the blocks 1 and 3
# to cover both the *MR pipeline* and the *branch pipeline* workflows.
# 1. Run the job in an *MR pipeline* if MR pipelines for AST are enabled and there's an open merge request.
## When FIPS mode is enabled, use the FIPS compatible image
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_PIPELINE_SOURCE == "merge_request_event" &&
$GITLAB_FEATURES =~ /\bdependency_scanning\b/ &&
$CI_GITLAB_FIPS_MODE == "true"
exists: !reference [.gemnasium-maven-shared-rule, exists]
variables:
DS_IMAGE_SUFFIX: "-fips"
DS_REMEDIATE: "false"
## When FIPS mode is not enabled, use the regular image
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_PIPELINE_SOURCE == "merge_request_event" &&
$GITLAB_FEATURES =~ /\bdependency_scanning\b/
exists: !reference [.gemnasium-maven-shared-rule, exists]
# 2. Don't run the job in a *branch pipeline* if *MR pipelines* for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_OPEN_MERGE_REQUESTS
when: never
# 3. Finally, run the job in a *branch pipeline* (When MR pipelines are disabled for AST, or it is enabled but no open MRs exist for the branch).
## When FIPS mode is enabled, use the FIPS compatible image
- if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bdependency_scanning\b/ &&
$CI_GITLAB_FIPS_MODE == "true"
exists: !reference [.gemnasium-maven-shared-rule, exists]
variables:
DS_IMAGE_SUFFIX: "-fips"
## When FIPS mode is not enabled, use the regular image
- if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bdependency_scanning\b/
exists: !reference [.gemnasium-maven-shared-rule, exists]
.gemnasium-python-shared-rule:
exists:
- '**/{requirements.txt,requirements.pip,Pipfile,Pipfile.lock,requires.txt,setup.py,poetry.lock,uv.lock}'
gemnasium-python-dependency_scanning:
extends:
- .ds-analyzer
- .cyclonedx-reports
variables:
DS_ANALYZER_NAME: "gemnasium-python"
rules:
- if: $DEPENDENCY_SCANNING_DISABLED == 'true' || $DEPENDENCY_SCANNING_DISABLED == '1'
when: never
- if: $DS_EXCLUDED_ANALYZERS =~ /gemnasium-python/
when: never
# The following 3 blocks of rules define whether the job runs in a an *MR pipeline* or a *branch pipeline*
# when an MR exists. If the job has additional rules to observe they should be added in the blocks 1 and 3
# to cover both the *MR pipeline* and the *branch pipeline* workflows.
# 1. Run the job in an *MR pipeline* if MR pipelines for AST are enabled and there's an open merge request.
## When FIPS mode is enabled, use the FIPS compatible image
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_PIPELINE_SOURCE == "merge_request_event" &&
$GITLAB_FEATURES =~ /\bdependency_scanning\b/ &&
$CI_GITLAB_FIPS_MODE == "true"
exists: !reference [.gemnasium-python-shared-rule, exists]
variables:
DS_IMAGE_SUFFIX: "-fips"
## When FIPS mode is not enabled, use the regular image
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_PIPELINE_SOURCE == "merge_request_event" &&
$GITLAB_FEATURES =~ /\bdependency_scanning\b/
exists: !reference [.gemnasium-python-shared-rule, exists]
# Support passing of $PIP_REQUIREMENTS_FILE
# See https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#configuring-specific-analyzers-used-by-dependency-scanning
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_PIPELINE_SOURCE == "merge_request_event" &&
$GITLAB_FEATURES =~ /\bdependency_scanning\b/ &&
$PIP_REQUIREMENTS_FILE &&
$CI_GITLAB_FIPS_MODE == "true"
variables:
DS_IMAGE_SUFFIX: "-fips"
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_PIPELINE_SOURCE == "merge_request_event" &&
$GITLAB_FEATURES =~ /\bdependency_scanning\b/ &&
$PIP_REQUIREMENTS_FILE
# 2. Don't run the job in a *branch pipeline* if *MR pipelines* for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_OPEN_MERGE_REQUESTS
when: never
# 3. Finally, run the job in a *branch pipeline* (When MR pipelines are disabled for AST, or it is enabled but no open MRs exist for the branch).
## When FIPS mode is enabled, use the FIPS compatible image
- if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bdependency_scanning\b/ &&
$CI_GITLAB_FIPS_MODE == "true"
exists: !reference [.gemnasium-python-shared-rule, exists]
variables:
DS_IMAGE_SUFFIX: "-fips"
## When FIPS mode is not enabled, use the regular image
- if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bdependency_scanning\b/
exists: !reference [.gemnasium-python-shared-rule, exists]
# Support passing of $PIP_REQUIREMENTS_FILE
# See https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#configuring-specific-analyzers-used-by-dependency-scanning
- if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bdependency_scanning\b/ &&
$PIP_REQUIREMENTS_FILE &&
$CI_GITLAB_FIPS_MODE == "true"
variables:
DS_IMAGE_SUFFIX: "-fips"
- if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bdependency_scanning\b/ &&
$PIP_REQUIREMENTS_FILE
# Read more about this feature here: https://docs.gitlab.com/ee/user/application_security/sast/
#
# Configure SAST with CI/CD variables (https://docs.gitlab.com/ee/ci/variables/).
# List of available variables: https://docs.gitlab.com/ee/user/application_security/sast/#available-cicd-variables
variables:
# Setting this variable affects all Security templates
# (SAST, Dependency Scanning, ...)
SECURE_ANALYZERS_PREFIX: "$CI_TEMPLATE_REGISTRY_HOST/security-products"
#
SAST_IMAGE_SUFFIX: ""
SAST_EXCLUDED_ANALYZERS: ""
DEFAULT_SAST_EXCLUDED_PATHS: "spec, test, tests, tmp"
SAST_EXCLUDED_PATHS: "$DEFAULT_SAST_EXCLUDED_PATHS"
SCAN_KUBERNETES_MANIFESTS: "false"
sast:
stage: test
artifacts:
access: 'developer'
reports:
sast: gl-sast-report.json
paths: [gl-sast-report.json]
rules:
- when: never
variables:
SEARCH_MAX_DEPTH: 4
script:
- echo "$CI_JOB_NAME is used for configuration only, and its script should not be executed"
- exit 1
.sast-analyzer:
extends: sast
allow_failure: true
# `rules` must be overridden explicitly by each child job
# see https://gitlab.com/gitlab-org/gitlab/-/issues/218444
script:
- /analyzer run
.deprecated-16.8:
extends: .sast-analyzer
script:
- echo "This job was deprecated in GitLab 16.8 and removed in GitLab 17.0"
- echo "For more information see https://docs.gitlab.com/update/deprecations/#sast-analyzer-coverage-changing-in-gitlab-170"
- exit 1
rules:
- when: never
# list of extensions that are supported by gitlab-advanced-sast. Some of these are also supported by semgrep-sast
# NOTE: When adding a new extension here, make sure to remove any matching extensions from `.semgrep-with-advanced-sast-exist-rules`
.gitlab-advanced-sast-exist-rules:
exists:
- '**/{*.py,*.go,*.java,*.jsp,*.js,*.jsx,*.ts,*.tsx,*.cjs,*.mjs,*.cs,*.rb,*.php}'
gitlab-advanced-sast:
extends: .sast-analyzer
image:
name: "$SAST_ANALYZER_IMAGE"
variables:
SEARCH_MAX_DEPTH: 20
SAST_ANALYZER_IMAGE_TAG: '2'
SAST_ANALYZER_IMAGE: "$SECURE_ANALYZERS_PREFIX/gitlab-advanced-sast:$SAST_ANALYZER_IMAGE_TAG$SAST_IMAGE_SUFFIX"
cache:
key: "scan-metrics-$CI_COMMIT_REF_SLUG"
fallback_keys:
- "scan-metrics-$CI_DEFAULT_BRANCH"
paths:
- "scan_metrics.csv"
rules:
- if: $SAST_DISABLED == 'true' || $SAST_DISABLED == '1'
when: never
- if: $SAST_EXCLUDED_ANALYZERS =~ /(^|[,[:space:]])gitlab-advanced-sast([,[:space:]]|$)/
when: never
- if: $GITLAB_ADVANCED_SAST_ENABLED != 'true' && $GITLAB_ADVANCED_SAST_ENABLED != '1'
when: never
# The following 3 blocks of rules define whether the job runs in a an *MR pipeline* or a *branch pipeline*
# when an MR exists. If the job has additional rules to observe they should be added in the blocks 1 and 3
# to cover both the *MR pipeline* and the *branch pipeline* workflows.
# 1. Run the job in an *MR* pipeline if MR pipelines for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_PIPELINE_SOURCE == "merge_request_event" &&
$GITLAB_FEATURES =~ /\bsast_advanced\b/
exists: !reference [.gitlab-advanced-sast-exist-rules, exists]
# 2. Don't run the job in a *branch pipeline* if *MR pipelines* for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_OPEN_MERGE_REQUESTS
when: never
# 3. Finally, run the job in a *branch pipeline* (When MR pipelines are disabled for AST, or it is enabled but no open MRs exist for the branch).
- if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bsast_advanced\b/
exists: !reference [.gitlab-advanced-sast-exist-rules, exists]
bandit-sast:
extends: .sast-analyzer
script:
- echo "This job was deprecated in GitLab 14.8 and removed in GitLab 15.4"
- echo "For more information see https://gitlab.com/gitlab-org/gitlab/-/issues/352554"
- exit 1
rules:
- when: never
brakeman-sast:
extends: .deprecated-16.8
eslint-sast:
extends: .sast-analyzer
script:
- echo "This job was deprecated in GitLab 14.8 and removed in GitLab 15.4"
- echo "For more information see https://gitlab.com/gitlab-org/gitlab/-/issues/352554"
- exit 1
rules:
- when: never
flawfinder-sast:
extends: .deprecated-16.8
kubesec-sast:
extends: .sast-analyzer
image:
name: "$SAST_ANALYZER_IMAGE"
variables:
SAST_ANALYZER_IMAGE_TAG: 6
SAST_ANALYZER_IMAGE: "$SECURE_ANALYZERS_PREFIX/kubesec:$SAST_ANALYZER_IMAGE_TAG"
rules:
- if: $SAST_DISABLED == 'true' || $SAST_DISABLED == '1'
when: never
- if: $SAST_EXCLUDED_ANALYZERS =~ /kubesec/
when: never
# The following 3 blocks of rules define whether the job runs in a an *MR pipeline* or a *branch pipeline*
# when an MR exists. If the job has additional rules to observe they should be added in the blocks 1 and 3
# to cover both the *MR pipeline* and the *branch pipeline* workflows.
# 1. Run the job in an *MR* pipeline if MR pipelines for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_PIPELINE_SOURCE == "merge_request_event" &&
$SCAN_KUBERNETES_MANIFESTS == 'true'
# 2. Don't run the job in a *branch pipeline* if *MR pipelines* for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_OPEN_MERGE_REQUESTS
when: never
# 3. Finally, run the job in a *branch pipeline* (When MR pipelines are disabled for AST, or it is enabled but no open MRs exist for the branch).
- if: $CI_COMMIT_BRANCH &&
$SCAN_KUBERNETES_MANIFESTS == 'true'
gosec-sast:
extends: .sast-analyzer
script:
- echo "This job was deprecated in GitLab 14.8 and removed in GitLab 15.4"
- echo "For more information see https://gitlab.com/gitlab-org/gitlab/-/issues/352554"
- exit 1
rules:
- when: never
mobsf-android-sast:
extends: .deprecated-16.8
mobsf-ios-sast:
extends: .deprecated-16.8
nodejs-scan-sast:
extends: .deprecated-16.8
phpcs-security-audit-sast:
extends: .deprecated-16.8
.pmd-apex-exist-rules:
exists:
- '**/*.cls'
pmd-apex-sast:
extends: .sast-analyzer
image:
name: "$SAST_ANALYZER_IMAGE"
variables:
SAST_ANALYZER_IMAGE_TAG: 6
SAST_ANALYZER_IMAGE: "$SECURE_ANALYZERS_PREFIX/pmd-apex:$SAST_ANALYZER_IMAGE_TAG"
rules:
- if: $SAST_DISABLED == 'true' || $SAST_DISABLED == '1'
when: never
- if: $SAST_EXCLUDED_ANALYZERS =~ /pmd-apex/
when: never
# The following 3 blocks of rules define whether the job runs in a an *MR pipeline* or a *branch pipeline*
# when an MR exists. If the job has additional rules to observe they should be added in the blocks 1 and 3
# to cover both the *MR pipeline* and the *branch pipeline* workflows.
# 1. Run the job in an *MR* pipeline if MR pipelines for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_PIPELINE_SOURCE == "merge_request_event"
exists: !reference [.pmd-apex-exist-rules, exists]
# 2. Don't run the job in a *branch pipeline* if *MR pipelines* for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_OPEN_MERGE_REQUESTS
when: never
# 3. Finally, run the job in a *branch pipeline* (When MR pipelines are disabled for AST, or it is enabled but no open MRs exist for the branch).
- if: $CI_COMMIT_BRANCH
exists: !reference [.pmd-apex-exist-rules, exists]
# list of extensions that are supported by semgrep-sast. Some of these are also supported by gitlab-advanced-sast
.semgrep-exist-rules:
exists:
- '**/{*.py,*.js,*.jsx,*.ts,*.tsx,*.cjs,*.mjs,*.c,*.cc,*.cpp,*.c++,*.cp,*.cxx,*.h,*.hpp,*.go,*.java,*.cs,*.scala,*.sc,*.php,*.swift,*.m,*.rb,*.kt,*.properties,application*.yml,bootstrap*.yml,application*.yaml,bootstrap*.yaml}'
# list of extensions that are only supported by semgrep, and not gitlab-advanced-sast
# NOTE: this list MUST NOT contain any extensions that are present in `.gitlab-advanced-sast-exist-rules`
.semgrep-with-advanced-sast-exist-rules:
exists:
- '**/{*.c,*.cc,*.cpp,*.c++,*.cp,*.cxx,*.h,*.hpp,*.scala,*.sc,*.swift,*.m,*.kt,*.properties,application*.yml,bootstrap*.yml,application*.yaml,bootstrap*.yaml}'
security-code-scan-sast:
extends: .sast-analyzer
script:
- echo "This job was deprecated in GitLab 15.9 and removed in GitLab 16.0"
- echo "For more information see https://gitlab.com/gitlab-org/gitlab/-/issues/390416"
- exit 1
rules:
- when: never
semgrep-sast:
extends: .sast-analyzer
image:
name: "$SAST_ANALYZER_IMAGE"
variables:
SEARCH_MAX_DEPTH: 20
SAST_ANALYZER_IMAGE_TAG: 6
SAST_ANALYZER_IMAGE: "$SECURE_ANALYZERS_PREFIX/semgrep:$SAST_ANALYZER_IMAGE_TAG$SAST_IMAGE_SUFFIX"
rules:
- if: $SAST_DISABLED == 'true' || $SAST_DISABLED == '1'
when: never
- if: $SAST_EXCLUDED_ANALYZERS =~ /semgrep/
when: never
# The following 3 blocks of rules define whether the job runs in a an *MR pipeline* or a *branch pipeline*
# when an MR exists. If the job has additional rules to observe they should be added in the blocks 1 and 3
# to cover both the *MR pipeline* and the *branch pipeline* workflows.
# 1. Run the job in an *MR* pipeline if MR pipelines for AST are enabled and there's an open merge request.
## In case gitlab-advanced-sast also runs, exclude files already scanned by gitlab-advanced-sast
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_PIPELINE_SOURCE == "merge_request_event" &&
$GITLAB_FEATURES =~ /\bsast_advanced\b/ &&
$SAST_EXCLUDED_ANALYZERS !~ /(^|[,[:space:]])gitlab-advanced-sast([,[:space:]]|$)/ &&
($GITLAB_ADVANCED_SAST_ENABLED == 'true' || $GITLAB_ADVANCED_SAST_ENABLED == '1')
variables:
# Customization of SAST_EXCLUDED_PATHS to be removed in 19.0, more details https://gitlab.com/gitlab-org/gitlab/-/issues/562940
SAST_EXCLUDED_PATHS: "$DEFAULT_SAST_EXCLUDED_PATHS, **/*.py, **/*.go, **/*.java, **/*.js, **/*.jsx, **/*.ts, **/*.tsx, **/*.cjs, **/*.mjs, **/*.cs, **/*.rb, **/*.php"
SAST_SEMGREP_EXCLUDED_PATHS: "**/*.py, **/*.go, **/*.java, **/*.js, **/*.jsx, **/*.ts, **/*.tsx, **/*.cjs, **/*.mjs, **/*.cs, **/*.rb, **/*.php"
exists: !reference [.semgrep-with-advanced-sast-exist-rules, exists]
## In case gitlab-advanced-sast already covers all the files that semgrep-sast would have scanned (i.e the previous rule did not match) skip this job
- if: $CI_PIPELINE_SOURCE == "merge_request_event" &&
$GITLAB_FEATURES =~ /\bsast_advanced\b/ &&
$SAST_EXCLUDED_ANALYZERS !~ /(^|[,[:space:]])gitlab-advanced-sast([,[:space:]]|$)/ &&
($GITLAB_ADVANCED_SAST_ENABLED == 'true' || $GITLAB_ADVANCED_SAST_ENABLED == '1')
when: never
## In case gitlab-advanced-sast doesn't run, scan all files supported by semgrep
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_PIPELINE_SOURCE == "merge_request_event"
exists: !reference [.semgrep-exist-rules, exists]
# 2. Don't run the job in a *branch pipeline* if *MR pipelines* for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_OPEN_MERGE_REQUESTS
when: never
# 3. Finally, run the job in a *branch pipeline* (When MR pipelines are disabled for AST, or it is enabled but no open MRs exist for the branch).
## In case gitlab-advanced-sast also runs, exclude files already scanned by gitlab-advanced-sast
- if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bsast_advanced\b/ &&
$SAST_EXCLUDED_ANALYZERS !~ /(^|[,[:space:]])gitlab-advanced-sast([,[:space:]]|$)/ &&
($GITLAB_ADVANCED_SAST_ENABLED == 'true' || $GITLAB_ADVANCED_SAST_ENABLED == '1')
variables:
# Customization of SAST_EXCLUDED_PATHS to be removed in 19.0, more details https://gitlab.com/gitlab-org/gitlab/-/issues/562940
SAST_EXCLUDED_PATHS: "$DEFAULT_SAST_EXCLUDED_PATHS, **/*.py, **/*.go, **/*.java, **/*.js, **/*.jsx, **/*.ts, **/*.tsx, **/*.cjs, **/*.mjs, **/*.cs, **/*.rb, **/*.php"
SAST_SEMGREP_EXCLUDED_PATHS: "**/*.py, **/*.go, **/*.java, **/*.js, **/*.jsx, **/*.ts, **/*.tsx, **/*.cjs, **/*.mjs, **/*.cs, **/*.rb, **/*.php"
exists: !reference [.semgrep-with-advanced-sast-exist-rules, exists]
## In case gitlab-advanced-sast already covers all the files that semgrep-sast would have scanned
- if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bsast_advanced\b/ &&
$SAST_EXCLUDED_ANALYZERS !~ /(^|[,[:space:]])gitlab-advanced-sast([,[:space:]]|$)/ &&
($GITLAB_ADVANCED_SAST_ENABLED == 'true' || $GITLAB_ADVANCED_SAST_ENABLED == '1')
when: never
## In case gitlab-advanced-sast doesn't run, scan all files supported by semgrep
- if: $CI_COMMIT_BRANCH
exists: !reference [.semgrep-exist-rules, exists]
.sobelow-exist-rules:
exists:
- '**/mix.exs'
sobelow-sast:
extends: .sast-analyzer
image:
name: "$SAST_ANALYZER_IMAGE"
variables:
SAST_ANALYZER_IMAGE_TAG: 6
SAST_ANALYZER_IMAGE: "$SECURE_ANALYZERS_PREFIX/sobelow:$SAST_ANALYZER_IMAGE_TAG"
rules:
- if: $SAST_DISABLED == 'true' || $SAST_DISABLED == '1'
when: never
- if: $SAST_EXCLUDED_ANALYZERS =~ /sobelow/
when: never
# The following 3 blocks of rules define whether the job runs in a an *MR pipeline* or a *branch pipeline*
# when an MR exists. If the job has additional rules to observe they should be added in the blocks 1 and 3
# to cover both the *MR pipeline* and the *branch pipeline* workflows.
# 1. Run the job in an *MR* pipeline if MR pipelines for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_PIPELINE_SOURCE == "merge_request_event"
exists: !reference [.sobelow-exist-rules, exists]
# 2. Don't run the job in a *branch pipeline* if *MR pipelines* for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_OPEN_MERGE_REQUESTS
when: never
# 3. Finally, run the job in a *branch pipeline* (When MR pipelines are disabled for AST, or it is enabled but no open MRs exist for the branch).
- if: $CI_COMMIT_BRANCH
exists: !reference [.sobelow-exist-rules, exists]
.spotbugs-exist-rules:
exists:
- '**/*.groovy'
spotbugs-sast:
extends: .sast-analyzer
image:
name: "$SAST_ANALYZER_IMAGE"
variables:
SAST_ANALYZER_IMAGE_TAG: 5
SAST_ANALYZER_IMAGE: "$SECURE_ANALYZERS_PREFIX/spotbugs:$SAST_ANALYZER_IMAGE_TAG"
rules:
- if: $SAST_EXCLUDED_ANALYZERS =~ /spotbugs/
when: never
- if: $SAST_DISABLED == 'true' || $SAST_DISABLED == '1'
when: never
# The following 3 blocks of rules define whether the job runs in a an *MR pipeline* or a *branch pipeline*
# when an MR exists. If the job has additional rules to observe they should be added in the blocks 1 and 3
# to cover both the *MR pipeline* and the *branch pipeline* workflows.
# 1. Run the job in an *MR* pipeline if MR pipelines for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_PIPELINE_SOURCE == "merge_request_event"
exists: !reference [.spotbugs-exist-rules, exists]
# 2. Don't run the job in a *branch pipeline* if *MR pipelines* for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_OPEN_MERGE_REQUESTS
when: never
# 3. Finally, run the job in a *branch pipeline* (When MR pipelines are disabled for AST, or it is enabled but no open MRs exist for the branch).
- if: $CI_COMMIT_BRANCH
exists: !reference [.spotbugs-exist-rules, exists]
.gitlab-advanced-sast-cpp-exist-rules:
exists:
- '**/{*.c,*.cc,*.cpp,*.c++,*.cp,*.cxx,*.h,*.hpp}'
gitlab-advanced-sast-cpp:
extends: .sast-analyzer
image:
name: "$SAST_ANALYZER_IMAGE"
variables:
SAST_ANALYZER_IMAGE_TAG: 1
SAST_ANALYZER_IMAGE: "$SECURE_ANALYZERS_PREFIX/clangsa:$SAST_ANALYZER_IMAGE_TAG$SAST_IMAGE_SUFFIX"
rules:
- if: $SAST_DISABLED == 'true' || $SAST_DISABLED == '1'
when: never
- if: $SAST_EXCLUDED_ANALYZERS =~ /gitlab-advanced-sast-cpp/
when: never
- if: $GITLAB_ADVANCED_SAST_CPP_ENABLED != 'true' && $GITLAB_ADVANCED_SAST_CPP_ENABLED != '1'
when: never
# The following 3 blocks of rules define whether the job runs in a an *MR pipeline* or a *branch pipeline*
# when an MR exists. If the job has additional rules to observe they should be added in the blocks 1 and 3
# to cover both the *MR pipeline* and the *branch pipeline* workflows.
# 1. Run the job in an *MR* pipeline if MR pipelines for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_PIPELINE_SOURCE == "merge_request_event" &&
$GITLAB_FEATURES =~ /\bsast_advanced\b/
exists: !reference [.gitlab-advanced-sast-cpp-exist-rules, exists]
# 2. Don't run the job in a *branch pipeline* if *MR pipelines* for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_OPEN_MERGE_REQUESTS
when: never
# 3. Finally, run the job in a *branch pipeline* (When MR pipelines are disabled for AST, or it is enabled but no open MRs exist for the branch).
- if: $CI_COMMIT_BRANCH &&
$GITLAB_FEATURES =~ /\bsast_advanced\b/
exists: !reference [.gitlab-advanced-sast-cpp-exist-rules, exists]
# Read more about this feature here: https://docs.gitlab.com/ee/user/application_security/secret_detection
#
# Configure the scanning tool through the environment variables.
# List of the variables: https://docs.gitlab.com/ee/user/application_security/secret_detection/#available-variables
# How to set: https://docs.gitlab.com/ee/ci/yaml/#variables
variables:
# Setting this variable affects all Security templates
# (SAST, Dependency Scanning, ...)
SECURE_ANALYZERS_PREFIX: "$CI_TEMPLATE_REGISTRY_HOST/security-products"
#
SECRET_DETECTION_IMAGE_SUFFIX: ""
SECRETS_ANALYZER_VERSION: "7"
SECRET_DETECTION_EXCLUDED_PATHS: ""
.secret-analyzer:
stage: test
image: "$SECURE_ANALYZERS_PREFIX/secrets:$SECRETS_ANALYZER_VERSION$SECRET_DETECTION_IMAGE_SUFFIX"
services: []
allow_failure: true
variables:
GIT_DEPTH: "50"
# `rules` must be overridden explicitly by each child job
# see https://gitlab.com/gitlab-org/gitlab/-/issues/218444
artifacts:
access: 'developer'
reports:
secret_detection: gl-secret-detection-report.json
paths: [gl-secret-detection-report.json]
secret_detection:
extends: .secret-analyzer
rules:
- if: $SECRET_DETECTION_DISABLED == 'true' || $SECRET_DETECTION_DISABLED == '1'
when: never
# The following 3 blocks of rules define whether the job runs in a an *MR pipeline* or a *branch pipeline*
# when an MR exists. If the job has additional rules to observe they should be added in the blocks 1 and 3
# to cover both the *MR pipeline* and the *branch pipeline* workflows.
# 1. Run the job in an *MR* pipeline if MR pipelines for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_PIPELINE_SOURCE == "merge_request_event"
# 2. Don't run the job in a *branch pipeline* if *MR pipelines* for AST are enabled and there's an open merge request.
- if: $AST_ENABLE_MR_PIPELINES == "true" &&
$CI_OPEN_MERGE_REQUESTS
when: never
# 3. Finally, run the job in a *branch pipeline* (When MR pipelines are disabled for AST, or it is enabled but no open MRs exist for the branch).
- if: $CI_COMMIT_BRANCH
script:
- /analyzer run
# Basic GitLab CI/CD Pipeline Example
# This example demonstrates a simple three-stage pipeline with best practices
stages:
- build
- test
- deploy
variables:
APP_NAME: "my-application"
APP_VERSION: "1.0.0"
# Default settings for all jobs
default:
retry:
max: 2
when:
- runner_system_failure
- stuck_or_timeout_failure
interruptible: true
tags:
- docker
# Build job
build_app:
stage: build
image: node:18-alpine
timeout: 15m
script:
- echo "Building $APP_NAME version $APP_VERSION"
- npm ci
- npm run build
artifacts:
paths:
- dist/
- node_modules/
expire_in: 1 hour
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- .npm/
# Unit tests
unit_tests:
stage: test
image: node:18-alpine
timeout: 10m
needs:
- build_app
dependencies:
- build_app
script:
- echo "Running unit tests"
- npm run test:unit
artifacts:
when: always
expire_in: 1 week
reports:
junit: junit.xml
coverage_report:
coverage_format: cobertura
path: coverage/cobertura.xml
coverage: '/Lines\s*:\s*(\d+\.\d+)%/'
# Linting job
lint_code:
stage: test
image: node:18-alpine
timeout: 5m
needs:
- build_app
dependencies:
- build_app
script:
- echo "Linting code"
- npm run lint
allow_failure: true
# Deploy to staging
deploy_staging:
stage: deploy
image: alpine:3.19
timeout: 10m
interruptible: false
needs:
- unit_tests
- lint_code
before_script:
- apk add --no-cache curl
script:
- echo "Deploying to staging environment"
- echo "Version $APP_VERSION"
- curl -X POST $STAGING_WEBHOOK_URL
environment:
name: staging
url: https://staging.example.com
resource_group: staging
rules:
- if: '$CI_COMMIT_BRANCH == "develop"'
# Deploy to production
deploy_production:
stage: deploy
image: alpine:3.19
timeout: 10m
interruptible: false
needs:
- unit_tests
- lint_code
before_script:
- apk add --no-cache curl
script:
- echo "Deploying to production environment"
- echo "Version $APP_VERSION"
- curl -X POST $PRODUCTION_WEBHOOK_URL
environment:
name: production
url: https://example.com
resource_group: production
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
when: manual
allow_failure: false
# Complex Workflow Example
# Demonstrates advanced GitLab CI/CD features including:
# - Multiple includes and templates
# - Matrix builds
# - Conditional workflows
# - Release automation
# - Performance testing
# - Infrastructure as Code
include:
# Use Jobs templates instead of deprecated Security templates
- template: Jobs/SAST.gitlab-ci.yml
- template: Jobs/Secret-Detection.gitlab-ci.yml
- template: Jobs/Dependency-Scanning.gitlab-ci.yml
stages:
- .pre
- validate
- build
- test
- performance
- security
- package
- deploy
- verify
- .post
variables:
CACHE_VERSION: "v2"
DOCKER_BUILDKIT: 1
FF_USE_FASTZIP: "true"
ARTIFACT_COMPRESSION_LEVEL: "fast"
CACHE_COMPRESSION_LEVEL: "fast"
# Workflow rules to control when pipelines run
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
- if: '$CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/'
- if: '$CI_PIPELINE_SOURCE == "schedule"'
- when: never
# Default settings for all jobs
default:
retry:
max: 2
when:
- runner_system_failure
- stuck_or_timeout_failure
interruptible: true
tags:
- docker
# Hidden job templates
.base_node_job:
image: node:18-alpine
cache:
key: ${CACHE_VERSION}-node-${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
- .npm/
before_script:
- npm ci --cache .npm --prefer-offline
.base_go_job:
image: golang:1.21-alpine
cache:
key: ${CACHE_VERSION}-go-${CI_COMMIT_REF_SLUG}
paths:
- /go/pkg/mod/
before_script:
- go mod download
.deploy_template:
image: google/cloud-sdk:486.0.0-alpine
before_script:
- echo $GCP_SERVICE_KEY | base64 -d > ${HOME}/gcp-key.json
- gcloud auth activate-service-account --key-file ${HOME}/gcp-key.json
- gcloud config set project $GCP_PROJECT_ID
# Validation stage
validate_yaml:
stage: .pre
image: alpine:3.19
before_script:
- apk add --no-cache yamllint
script:
- yamllint -c .yamllint.yml .
rules:
- changes:
- "**/*.yml"
- "**/*.yaml"
validate_terraform:
stage: validate
image: hashicorp/terraform:1.9
script:
- cd infrastructure/
- terraform fmt -check -recursive
- terraform init -backend=false
- terraform validate
rules:
- changes:
- "infrastructure/**/*.tf"
# Build stage with matrix
build_frontend:
extends: .base_node_job
stage: build
parallel:
matrix:
- NODE_VERSION: ["16", "18", "20"]
image: node:${NODE_VERSION}-alpine
script:
- echo "Building with Node ${NODE_VERSION}"
- npm run build
- npm run build:storybook
artifacts:
paths:
- dist/
- storybook-static/
expire_in: 1 day
name: "frontend-node${NODE_VERSION}-${CI_COMMIT_SHORT_SHA}"
build_backend:
extends: .base_go_job
stage: build
script:
- CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-w -s" -o bin/server-linux-amd64 ./cmd/server
- CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build -ldflags="-w -s" -o bin/server-darwin-arm64 ./cmd/server
artifacts:
paths:
- bin/
expire_in: 1 day
build_docker_images:
stage: build
image: docker:24-cli
services:
- docker:24-dind
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- |
docker build \
--cache-from $CI_REGISTRY_IMAGE:buildcache \
--build-arg BUILDKIT_INLINE_CACHE=1 \
--build-arg VERSION=$CI_COMMIT_TAG \
--build-arg COMMIT=$CI_COMMIT_SHORT_SHA \
--tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA \
--tag $CI_REGISTRY_IMAGE:latest \
.
- docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
- if [ -n "$CI_COMMIT_TAG" ]; then docker push $CI_REGISTRY_IMAGE:latest; fi
needs:
- build_backend
# Test stage
test_unit:
extends: .base_node_job
stage: test
script:
- npm run test:unit -- --coverage
artifacts:
when: always
reports:
junit: junit.xml
coverage_report:
coverage_format: cobertura
path: coverage/cobertura.xml
coverage: '/Lines\s*:\s*(\d+\.\d+)%/'
test_integration:
extends: .base_go_job
stage: test
services:
- name: postgres:15-alpine
alias: db
- name: redis:7-alpine
alias: cache
- name: elasticsearch:8.11.0
alias: search
variables:
discovery.type: single-node
xpack.security.enabled: "false"
variables:
DATABASE_URL: $TEST_DATABASE_URL # Set in CI/CD variables
REDIS_URL: $TEST_REDIS_URL
ELASTICSEARCH_URL: $TEST_ELASTICSEARCH_URL
script:
- go test -v -tags=integration -coverprofile=coverage.out ./...
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: coverage.xml
test_e2e:
image: mcr.microsoft.com/playwright:v1.40.0-jammy
stage: test
needs:
- build_frontend
services:
- name: postgres:15-alpine
alias: db
variables:
DATABASE_URL: $TEST_DATABASE_URL # Set in CI/CD variables
script:
- npm ci
- npm run test:e2e
artifacts:
when: always
paths:
- playwright-report/
- test-results/
expire_in: 7 days
allow_failure: true
test_accessibility:
extends: .base_node_job
stage: test
needs:
- build_frontend
script:
- npm run test:a11y
artifacts:
when: always
paths:
- a11y-report/
expire_in: 7 days
allow_failure: true
# Performance testing
performance_lighthouse:
image: cypress/browsers:node18.12.0-chrome106-ff106
stage: performance
needs:
- build_frontend
script:
- npm install -g @lhci/cli
- lhci autorun --collect.url=http://localhost:3000
artifacts:
paths:
- .lighthouseci/
expire_in: 7 days
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
allow_failure: true
performance_k6:
image: grafana/k6:0.52.0
stage: performance
needs:
- deploy_staging
script:
- k6 run --out json=k6-results.json tests/k6/load-test.js
artifacts:
paths:
- k6-results.json
expire_in: 7 days
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
allow_failure: true
# Security stage (in addition to included templates)
security_container_scan:
image: aquasec/trivy:0.55.0
stage: security
needs:
- build_docker_images
script:
- trivy image --format json --output container-scan.json $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
- trivy image --severity HIGH,CRITICAL --exit-code 1 $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
artifacts:
paths:
- container-scan.json
expire_in: 7 days
allow_failure: false
security_license_scan:
extends: .base_node_job
stage: security
script:
- npm install -g license-checker
- license-checker --json --out licenses.json
- license-checker --failOn 'GPL;AGPL'
artifacts:
paths:
- licenses.json
expire_in: 30 days
allow_failure: true
# Package stage
package_helm_chart:
image: alpine/helm:3.15.3
stage: package
needs:
- build_docker_images
script:
- helm package helm/app --version $CI_COMMIT_TAG --app-version $CI_COMMIT_TAG
- helm push app-$CI_COMMIT_TAG.tgz oci://$CI_REGISTRY/$CI_PROJECT_PATH/charts
rules:
- if: '$CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/'
create_release:
image: registry.gitlab.com/gitlab-org/release-cli:v0.17.0
stage: package
needs:
- build_backend
- build_frontend
script:
- echo "Creating release for $CI_COMMIT_TAG"
release:
tag_name: '$CI_COMMIT_TAG'
name: 'Release $CI_COMMIT_TAG'
description: './CHANGELOG.md'
assets:
links:
- name: 'Linux Binary'
url: '${CI_PROJECT_URL}/-/jobs/${CI_JOB_ID}/artifacts/file/bin/server-linux-amd64'
- name: 'macOS Binary'
url: '${CI_PROJECT_URL}/-/jobs/${CI_JOB_ID}/artifacts/file/bin/server-darwin-arm64'
rules:
- if: '$CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/'
# Deploy stage
deploy_staging:
extends: .deploy_template
stage: deploy
needs:
- test_unit
- test_integration
- security_container_scan
script:
- gcloud run deploy $SERVICE_NAME-staging
--image $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
--platform managed
--region $GCP_REGION
--allow-unauthenticated
environment:
name: staging
url: https://staging.example.com
on_stop: stop_staging
auto_stop_in: 7 days
deployment_tier: staging
rules:
- if: '$CI_COMMIT_BRANCH == "develop"'
stop_staging:
extends: .deploy_template
stage: deploy
script:
- gcloud run services delete $SERVICE_NAME-staging
--platform managed
--region $GCP_REGION
--quiet
environment:
name: staging
action: stop
when: manual
rules:
- if: '$CI_COMMIT_BRANCH == "develop"'
deploy_production:
extends: .deploy_template
stage: deploy
needs:
- test_unit
- test_integration
- test_e2e
- security_container_scan
script:
- gcloud run deploy $SERVICE_NAME
--image $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
--platform managed
--region $GCP_REGION
--allow-unauthenticated
environment:
name: production
url: https://example.com
deployment_tier: production
rules:
- if: '$CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/'
when: manual
allow_failure: false
interruptible: false
resource_group: production
deploy_canary:
extends: .deploy_template
stage: deploy
needs:
- deploy_production
script:
- echo "Deploying canary with 10% traffic"
- gcloud run services update-traffic $SERVICE_NAME
--to-revisions=$CI_COMMIT_SHORT_SHA=10,LATEST=90
--platform managed
--region $GCP_REGION
environment:
name: production/canary
deployment_tier: production
rules:
- if: '$CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/'
when: manual
interruptible: false
# Verify stage
verify_deployment:
image: curlimages/curl:8.9.0
stage: verify
needs:
- deploy_production
script:
- |
response=$(curl -s -o /dev/null -w "%{http_code}" https://example.com/health)
if [ $response -eq 200 ]; then
echo "Health check passed"
else
echo "Health check failed with status $response"
exit 1
fi
rules:
- if: '$CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/'
smoke_tests:
extends: .base_node_job
stage: verify
needs:
- deploy_production
script:
- npm run test:smoke
rules:
- if: '$CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/'
# Post-deployment actions
notify_success:
image: alpine:3.19
stage: .post
before_script:
- apk add --no-cache curl
script:
- |
curl -X POST $SLACK_WEBHOOK \
-H 'Content-Type: application/json' \
-d "{
\"text\": \"✅ Pipeline successful!\",
\"attachments\": [{
\"color\": \"good\",
\"fields\": [
{\"title\": \"Project\", \"value\": \"$CI_PROJECT_NAME\", \"short\": true},
{\"title\": \"Branch\", \"value\": \"$CI_COMMIT_BRANCH\", \"short\": true},
{\"title\": \"Commit\", \"value\": \"$CI_COMMIT_SHORT_SHA\", \"short\": true},
{\"title\": \"Author\", \"value\": \"$GITLAB_USER_NAME\", \"short\": true}
],
\"actions\": [{
\"type\": \"button\",
\"text\": \"View Pipeline\",
\"url\": \"$CI_PIPELINE_URL\"
}]
}]
}"
when: on_success
rules:
- if: '$CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/'
notify_failure:
image: alpine:3.19
stage: .post
before_script:
- apk add --no-cache curl
script:
- |
curl -X POST $SLACK_WEBHOOK \
-H 'Content-Type: application/json' \
-d "{
\"text\": \"❌ Pipeline failed!\",
\"attachments\": [{
\"color\": \"danger\",
\"fields\": [
{\"title\": \"Project\", \"value\": \"$CI_PROJECT_NAME\", \"short\": true},
{\"title\": \"Branch\", \"value\": \"$CI_COMMIT_BRANCH\", \"short\": true},
{\"title\": \"Failed Job\", \"value\": \"$CI_JOB_NAME\", \"short\": true}
],
\"actions\": [{
\"type\": \"button\",
\"text\": \"View Pipeline\",
\"url\": \"$CI_PIPELINE_URL\"
}]
}]
}"
when: on_failure
rules:
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
- if: '$CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/'# GitLab CI/CD Pipeline Example with Components (GitLab 17.0+)
#
# This example demonstrates the use of CI/CD components from the GitLab Catalog.
# Components are reusable pipeline configuration units introduced in GitLab 16.x
# and made GA in GitLab 17.0 (May 2024).
#
# Learn more: https://docs.gitlab.com/ci/components/
# Include components from the CI/CD Catalog
include:
# Example 1: Docker build and push component
- component: $CI_SERVER_FQDN/components/docker/build-and-push@1.2.0
inputs:
stage: build
dockerfile: Dockerfile
context: .
docker_image: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
registry_user: $CI_REGISTRY_USER
registry_password: $CI_REGISTRY_PASSWORD
# Example 2: Node.js test component with specific version
- component: gitlab.com/components/nodejs/test@2.1.0
inputs:
stage: test
node_version: "20"
test_command: "npm test"
# Example 3: Security scanning component from official GitLab components
- component: $CI_SERVER_FQDN/components/security/sast@3.0.1
inputs:
stage: security
sast_excluded_paths: "tests/,docs/"
# Example 4: Component with conditional rules
- component: $CI_SERVER_FQDN/components/deploy/kubernetes@1.5.2
inputs:
stage: deploy
k8s_namespace: $CI_ENVIRONMENT_NAME
k8s_cluster: $K8S_CLUSTER_NAME
manifest_path: ./k8s/deployment.yaml
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
# Example 5: Using ~latest for development (not recommended for production)
# Warning: This will use the absolute latest version which may include breaking changes
- component: $CI_SERVER_FQDN/components/quality/code-coverage@~latest
inputs:
stage: report
coverage_format: cobertura
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
# Example 6: Project include (traditional method) with pinned ref for comparison
- project: 'shared-ci/templates'
ref: 'v2.3.0' # Pinned version for security and reproducibility
file: '/templates/notification.gitlab-ci.yml'
# Example 7: Local include for custom jobs
- local: '.gitlab/ci/custom-jobs.yml'
# Define pipeline stages
stages:
- build
- test
- security
- report
- deploy
- .post # Special stage that runs after all other stages
# Global variables
variables:
# Docker configuration
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "/certs"
# Component configuration
CI_DEBUG_COMPONENTS: "false"
# Additional custom job that works alongside components
validate:
stage: test
image: alpine:latest
script:
- echo "Running additional validation..."
- echo "This job runs alongside component-based jobs"
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
# Override or extend component behavior
# Components can be customized using the inputs parameter
# or by defining jobs that extend component-generated jobs
# Example: Production deployment with manual trigger
deploy_production:
stage: deploy
image: bitnami/kubectl:latest
script:
- echo "Deploying to production environment"
- kubectl apply -f k8s/production/
environment:
name: production
url: https://app.example.com
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: manual
needs:
- validate
# Example: Notification job
notify_success:
stage: .post
image: curlimages/curl:latest
script:
- echo "Pipeline completed successfully"
- |
curl -X POST $SLACK_WEBHOOK_URL \
-H 'Content-Type: application/json' \
-d "{\"text\": \"Pipeline $CI_PIPELINE_ID completed for $CI_PROJECT_NAME\"}"
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: on_success
allow_failure: true
# Best Practices for Components:
#
# 1. Version Pinning:
# - Always pin to specific versions (@1.2.3) in production
# - Use ~latest only in development/testing
# - Partial versions (~1.2) match latest patch version
#
# 2. Inputs:
# - Use CI/CD variables for sensitive data (not hardcoded)
# - Validate required inputs are provided
# - Use descriptive input names
#
# 3. Security:
# - Verify component sources (prefer $CI_SERVER_FQDN)
# - Review component code before using in production
# - Monitor for component updates and security advisories
#
# 4. Testing:
# - Test component integration in non-production branches first
# - Use rules to control when components run
# - Monitor pipeline execution times
#
# 5. Documentation:
# - Document why specific components were chosen
# - Keep track of component versions used
# - Document any custom configurations or inputs
# Docker Build and Push Pipeline Example
# Demonstrates building and pushing Docker images to registry
stages:
- test
- build
- scan
- deploy
variables:
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "/certs"
IMAGE_NAME: $CI_REGISTRY_IMAGE
IMAGE_TAG: $CI_COMMIT_SHORT_SHA
# Use Docker-in-Docker service
default:
image: docker:24-cli
services:
- docker:24-dind
before_script:
- docker info
# Run tests before building
test_application:
image: node:18-alpine
stage: test
script:
- npm ci
- npm run test
artifacts:
reports:
junit: junit.xml
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
- .npm/
# Build Docker image
build_image:
stage: build
script:
- echo "Building Docker image"
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build
--build-arg VERSION=$IMAGE_TAG
--build-arg BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
--tag $IMAGE_NAME:$IMAGE_TAG
--tag $IMAGE_NAME:latest
.
- docker push $IMAGE_NAME:$IMAGE_TAG
- docker push $IMAGE_NAME:latest
rules:
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
- if: '$CI_COMMIT_TAG'
tags:
- docker
# Build image for feature branches (no push to latest)
build_feature_image:
stage: build
script:
- echo "Building feature branch image"
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build
--tag $IMAGE_NAME:$CI_COMMIT_REF_SLUG
.
- docker push $IMAGE_NAME:$CI_COMMIT_REF_SLUG
rules:
- if: '$CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH && $CI_COMMIT_TAG == null'
tags:
- docker
# Scan image for vulnerabilities
scan_image:
image: aquasec/trivy:latest
stage: scan
script:
- echo "Scanning image for vulnerabilities"
- trivy image --exit-code 0 --severity LOW,MEDIUM $IMAGE_NAME:$IMAGE_TAG
- trivy image --exit-code 1 --severity HIGH,CRITICAL $IMAGE_NAME:$IMAGE_TAG
needs:
- build_image
allow_failure: true
rules:
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
- if: '$CI_COMMIT_TAG'
tags:
- docker
# Deploy to Kubernetes
deploy_to_k8s:
image: bitnami/kubectl:latest
stage: deploy
script:
- echo "Deploying to Kubernetes"
- kubectl config use-context $KUBE_CONTEXT
- kubectl set image deployment/$APP_NAME $APP_NAME=$IMAGE_NAME:$IMAGE_TAG -n $NAMESPACE
- kubectl rollout status deployment/$APP_NAME -n $NAMESPACE
environment:
name: production
url: https://app.example.com
kubernetes:
namespace: $NAMESPACE
rules:
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
when: manual
needs:
- build_image
- scan_image
tags:
- kubernetes
# Multi-stage build example with BuildKit
build_optimized:
stage: build
variables:
DOCKER_BUILDKIT: 1
script:
- echo "Building with BuildKit"
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build
--cache-from $IMAGE_NAME:buildcache
--build-arg BUILDKIT_INLINE_CACHE=1
--tag $IMAGE_NAME:$IMAGE_TAG-optimized
--file Dockerfile.optimized
.
- docker push $IMAGE_NAME:$IMAGE_TAG-optimized
rules:
- if: '$CI_COMMIT_TAG'
tags:
- docker
# Docker Compose testing
test_with_compose:
stage: test
image: docker/compose:latest
services:
- docker:24-dind
script:
- docker compose up -d
- docker compose ps
- docker compose run --rm test npm run test:integration
- docker compose down
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
tags:
- docker
# Multi-Stage Pipeline with DAG (Directed Acyclic Graph)
# Demonstrates advanced pipeline structure with parallel execution
stages:
- .pre
- build
- test
- security
- deploy
- .post
variables:
CACHE_VERSION: "v1"
NODE_VERSION: "18"
# Pre-flight checks
validate_config:
stage: .pre
image: alpine:latest
script:
- echo "Validating configuration"
- test -f package.json || exit 1
- test -f Dockerfile || exit 1
tags:
- docker
# Build frontend
build_frontend:
stage: build
image: node:${NODE_VERSION}-alpine
script:
- echo "Building frontend"
- cd frontend
- npm ci
- npm run build
artifacts:
paths:
- frontend/dist/
expire_in: 1 day
cache:
key: ${CACHE_VERSION}-frontend-${CI_COMMIT_REF_SLUG}
paths:
- frontend/node_modules/
- frontend/.npm/
tags:
- docker
# Build backend
build_backend:
stage: build
image: golang:1.21-alpine
script:
- echo "Building backend"
- cd backend
- go mod download
- go build -o bin/server ./cmd/server
artifacts:
paths:
- backend/bin/
expire_in: 1 day
cache:
key: ${CACHE_VERSION}-backend-${CI_COMMIT_REF_SLUG}
paths:
- /go/pkg/mod/
tags:
- docker
# Build documentation
build_docs:
stage: build
image: python:3.11-alpine
script:
- echo "Building documentation"
- pip install mkdocs mkdocs-material
- mkdocs build
artifacts:
paths:
- site/
expire_in: 1 week
tags:
- docker
# Frontend unit tests - runs immediately after build_frontend
test_frontend_unit:
stage: test
image: node:${NODE_VERSION}-alpine
needs:
- job: build_frontend
artifacts: true
script:
- echo "Running frontend unit tests"
- cd frontend
- npm ci
- npm run test:unit
artifacts:
reports:
junit: frontend/junit.xml
coverage_report:
coverage_format: cobertura
path: frontend/coverage/cobertura.xml
coverage: '/Lines\s*:\s*(\d+\.\d+)%/'
tags:
- docker
# Frontend E2E tests
test_frontend_e2e:
stage: test
image: cypress/included:13.6.0
needs:
- job: build_frontend
artifacts: true
services:
- name: nginx:alpine
alias: web
script:
- echo "Running E2E tests"
- cd frontend
- npm ci
- npm run test:e2e
artifacts:
when: always
paths:
- frontend/cypress/screenshots/
- frontend/cypress/videos/
expire_in: 1 week
allow_failure: true
tags:
- docker
# Backend unit tests
test_backend_unit:
stage: test
image: golang:1.21-alpine
needs:
- job: build_backend
artifacts: true
script:
- echo "Running backend unit tests"
- cd backend
- go test -v -coverprofile=coverage.out ./...
- go tool cover -func=coverage.out
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: backend/coverage.xml
coverage: '/total:.*?(\d+\.\d+)%/'
tags:
- docker
# Backend integration tests
test_backend_integration:
stage: test
image: golang:1.21-alpine
needs:
- job: build_backend
artifacts: true
services:
- name: postgres:15-alpine
alias: postgres
- name: redis:7-alpine
alias: redis
variables:
POSTGRES_DB: testdb
POSTGRES_USER: test
POSTGRES_PASSWORD: test
DATABASE_URL: "postgres://test:test@postgres:5432/testdb?sslmode=disable"
REDIS_URL: "redis://redis:6379"
script:
- echo "Running backend integration tests"
- cd backend
- go test -v -tags=integration ./...
tags:
- docker
# Lint frontend code
lint_frontend:
stage: test
image: node:${NODE_VERSION}-alpine
needs:
- job: build_frontend
artifacts: false
script:
- echo "Linting frontend code"
- cd frontend
- npm ci
- npm run lint
allow_failure: true
tags:
- docker
# Lint backend code
lint_backend:
stage: test
image: golangci/golangci-lint:v1.55-alpine
needs:
- job: build_backend
artifacts: false
script:
- echo "Linting backend code"
- cd backend
- golangci-lint run
allow_failure: true
tags:
- docker
# SAST scanning
sast_scan:
stage: security
image: returntocorp/semgrep:latest
needs: []
script:
- semgrep --config=auto --json --output=sast-report.json .
artifacts:
reports:
sast: sast-report.json
allow_failure: true
tags:
- docker
# Dependency scanning
dependency_scan:
stage: security
image: aquasec/trivy:latest
needs: []
script:
- trivy fs --format json --output dependency-report.json .
artifacts:
paths:
- dependency-report.json
expire_in: 1 week
allow_failure: true
tags:
- docker
# Secret scanning
secret_scan:
stage: security
image: trufflesecurity/trufflehog:latest
needs: []
script:
- trufflehog filesystem --directory=. --json > secrets-report.json
artifacts:
paths:
- secrets-report.json
expire_in: 1 week
allow_failure: true
tags:
- docker
# Deploy to staging
deploy_staging:
stage: deploy
image: alpine:latest
needs:
- test_frontend_unit
- test_backend_unit
- test_backend_integration
before_script:
- apk add --no-cache curl
script:
- echo "Deploying to staging"
- curl -X POST "$STAGING_DEPLOY_WEBHOOK"
environment:
name: staging
url: https://staging.example.com
on_stop: stop_staging
auto_stop_in: 7 days
rules:
- if: '$CI_COMMIT_BRANCH == "develop"'
tags:
- docker
# Stop staging environment
stop_staging:
stage: deploy
image: alpine:latest
before_script:
- apk add --no-cache curl
script:
- echo "Stopping staging environment"
- curl -X DELETE "$STAGING_DEPLOY_WEBHOOK"
environment:
name: staging
action: stop
when: manual
rules:
- if: '$CI_COMMIT_BRANCH == "develop"'
tags:
- docker
# Deploy review app
deploy_review:
stage: deploy
image: alpine:latest
needs:
- test_frontend_unit
- test_backend_unit
before_script:
- apk add --no-cache curl
script:
- echo "Deploying review app"
- echo "URL https://$CI_ENVIRONMENT_SLUG.review.example.com"
- curl -X POST "$REVIEW_DEPLOY_WEBHOOK"
environment:
name: review/$CI_COMMIT_REF_SLUG
url: https://$CI_ENVIRONMENT_SLUG.review.example.com
on_stop: stop_review
auto_stop_in: 3 days
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
tags:
- docker
# Stop review app
stop_review:
stage: deploy
image: alpine:latest
before_script:
- apk add --no-cache curl
script:
- echo "Stopping review app"
- curl -X DELETE "$REVIEW_DEPLOY_WEBHOOK"
environment:
name: review/$CI_COMMIT_REF_SLUG
action: stop
when: manual
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
tags:
- docker
# Deploy to production
deploy_production:
stage: deploy
image: alpine:latest
needs:
- test_frontend_unit
- test_backend_unit
- test_backend_integration
- sast_scan
- dependency_scan
before_script:
- apk add --no-cache curl
script:
- echo "Deploying to production"
- curl -X POST "$PRODUCTION_DEPLOY_WEBHOOK"
environment:
name: production
url: https://example.com
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
when: manual
allow_failure: false
tags:
- docker
# Post-deployment notifications
notify_deployment:
stage: .post
image: alpine:latest
before_script:
- apk add --no-cache curl
script:
- echo "Sending deployment notification"
- |
curl -X POST "$SLACK_WEBHOOK" \
-H 'Content-Type: application/json' \
-d "{\"text\":\"Pipeline completed: $CI_PIPELINE_URL\"}"
when: always
rules:
- if: '$CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH == "develop"'
tags:
- docker
#!/usr/bin/env bash
#
# GitLab CI Validator - Tool Installation Script
#
# This script installs the required tools for local pipeline testing:
# - gitlab-ci-local: For local GitLab CI pipeline execution (similar to act for GitHub Actions)
#
# Usage: bash scripts/install_tools.sh
#
set -euo pipefail
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Script directory
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
TOOLS_DIR="$SCRIPT_DIR/.tools"
# Create tools directory if it doesn't exist
mkdir -p "$TOOLS_DIR"
echo -e "${BLUE}════════════════════════════════════════════════════════════════════════════════${NC}"
echo -e "${BLUE} GitLab CI Validator - Tool Installation${NC}"
echo -e "${BLUE}════════════════════════════════════════════════════════════════════════════════${NC}"
echo ""
# Function to check if a command exists
command_exists() {
command -v "$1" &> /dev/null
}
# Function to get OS type
get_os() {
case "$(uname -s)" in
Darwin*) echo "darwin" ;;
Linux*) echo "linux" ;;
*) echo "unknown" ;;
esac
}
# Function to get architecture
get_arch() {
case "$(uname -m)" in
x86_64) echo "x64" ;;
aarch64) echo "arm64" ;;
arm64) echo "arm64" ;;
*) echo "unknown" ;;
esac
}
#
# Install gitlab-ci-local
#
install_gitlab_ci_local() {
echo -e "${BLUE}[1/1]${NC} Checking for gitlab-ci-local..."
# Check if already installed globally
if command_exists gitlab-ci-local; then
CURRENT_VERSION=$(gitlab-ci-local --version 2>/dev/null || echo "unknown")
echo -e "${GREEN}✓${NC} gitlab-ci-local is already installed: $CURRENT_VERSION"
echo ""
return 0
fi
# Check if installed in tools directory
if [ -f "$TOOLS_DIR/gitlab-ci-local" ]; then
CURRENT_VERSION=$("$TOOLS_DIR/gitlab-ci-local" --version 2>/dev/null || echo "unknown")
echo -e "${GREEN}✓${NC} gitlab-ci-local is installed in .tools: $CURRENT_VERSION"
echo ""
return 0
fi
echo -e "${YELLOW}→${NC} gitlab-ci-local not found. Installing..."
echo ""
# Check for Node.js (required for gitlab-ci-local)
if ! command_exists node; then
echo -e "${RED}✗${NC} Node.js is not installed but is required for gitlab-ci-local"
echo ""
echo "Please install Node.js first:"
echo " - macOS: brew install node"
echo " - Linux: Install from https://nodejs.org/ or use your package manager"
echo ""
echo "After installing Node.js, run this script again."
return 1
fi
NODE_VERSION=$(node --version 2>/dev/null || echo "unknown")
echo -e "${GREEN}✓${NC} Node.js is installed: $NODE_VERSION"
# Install gitlab-ci-local using npm
echo ""
echo -e "${YELLOW}→${NC} Installing gitlab-ci-local via npm..."
echo " Note: This requires Docker to be installed for pipeline execution"
echo ""
# Try to install globally if user has permissions
if npm install -g gitlab-ci-local 2>/dev/null; then
INSTALLED_VERSION=$(gitlab-ci-local --version 2>/dev/null || echo "unknown")
echo ""
echo -e "${GREEN}✓${NC} gitlab-ci-local installed successfully: $INSTALLED_VERSION"
echo " Location: $(which gitlab-ci-local)"
else
echo -e "${YELLOW}⚠${NC} Could not install globally. Trying local installation..."
echo ""
# Install locally in project
cd "$SCRIPT_DIR/.."
if npm install --save-dev gitlab-ci-local 2>/dev/null; then
echo ""
echo -e "${GREEN}✓${NC} gitlab-ci-local installed locally in node_modules"
echo " Use: npx gitlab-ci-local --help"
else
echo -e "${RED}✗${NC} Failed to install gitlab-ci-local"
echo ""
echo "Manual installation options:"
echo " 1. Global install: npm install -g gitlab-ci-local"
echo " 2. Project install: npm install --save-dev gitlab-ci-local"
echo ""
echo "For more information: https://github.com/firecow/gitlab-ci-local"
return 1
fi
fi
echo ""
}
#
# Verify Docker installation (required for gitlab-ci-local)
#
check_docker() {
echo -e "${BLUE}Checking Docker installation...${NC}"
echo ""
if command_exists docker; then
DOCKER_VERSION=$(docker --version 2>/dev/null || echo "unknown")
echo -e "${GREEN}✓${NC} Docker is installed: $DOCKER_VERSION"
# Check if Docker daemon is running
if docker ps &> /dev/null; then
echo -e "${GREEN}✓${NC} Docker daemon is running"
else
echo -e "${YELLOW}⚠${NC} Docker is installed but daemon is not running"
echo " Start Docker Desktop or run: sudo systemctl start docker"
fi
else
echo -e "${YELLOW}⚠${NC} Docker is not installed"
echo ""
echo "Docker is required for gitlab-ci-local to execute pipelines locally."
echo ""
echo "Installation options:"
echo " - macOS: Install Docker Desktop from https://www.docker.com/products/docker-desktop"
echo " - Linux: Install Docker Engine from https://docs.docker.com/engine/install/"
echo ""
fi
echo ""
}
#
# Main installation process
#
main() {
# Install gitlab-ci-local
if ! install_gitlab_ci_local; then
echo -e "${RED}✗ Installation failed${NC}"
exit 1
fi
# Check Docker
check_docker
# Summary
echo -e "${BLUE}════════════════════════════════════════════════════════════════════════════════${NC}"
echo -e "${BLUE} Installation Summary${NC}"
echo -e "${BLUE}════════════════════════════════════════════════════════════════════════════════${NC}"
echo ""
# gitlab-ci-local status
if command_exists gitlab-ci-local; then
echo -e "${GREEN}✓${NC} gitlab-ci-local: $(gitlab-ci-local --version)"
elif [ -f "$TOOLS_DIR/gitlab-ci-local" ]; then
echo -e "${GREEN}✓${NC} gitlab-ci-local: installed in .tools"
elif command_exists npx && npx --no-install gitlab-ci-local --version &> /dev/null; then
echo -e "${GREEN}✓${NC} gitlab-ci-local: installed locally (use npx gitlab-ci-local)"
else
echo -e "${YELLOW}⚠${NC} gitlab-ci-local: not installed"
fi
# Docker status
if command_exists docker && docker ps &> /dev/null; then
echo -e "${GREEN}✓${NC} Docker: running"
elif command_exists docker; then
echo -e "${YELLOW}⚠${NC} Docker: installed but not running"
else
echo -e "${YELLOW}⚠${NC} Docker: not installed"
fi
echo ""
echo -e "${BLUE}════════════════════════════════════════════════════════════════════════════════${NC}"
echo ""
# Usage information
echo -e "${GREEN}✓ Installation complete!${NC}"
echo ""
echo "Next steps:"
echo " 1. Ensure Docker is running"
echo " 2. Test with: gitlab-ci-local --help"
echo " 3. Run local pipeline: gitlab-ci-local"
echo " 4. Validate with: bash scripts/validate_gitlab_ci.sh --test-only .gitlab-ci.yml"
echo ""
echo "For more information:"
echo " - gitlab-ci-local: https://github.com/firecow/gitlab-ci-local"
echo " - GitLab CI Docs: https://docs.gitlab.com/ci/"
echo ""
}
# Run main installation
main
#!/bin/bash
# Wrapper script that handles PyYAML dependency
# Creates a persistent venv if PyYAML is not available
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
VENV_DIR="$SCRIPT_DIR/../.venv"
# Check if we have arguments
if [ $# -lt 2 ]; then
echo "Usage: python_wrapper.sh <python-script> <args...>" >&2
exit 1
fi
PYTHON_SCRIPT="$1"
shift # Remove first argument, rest are passed to the Python script
# Try to run with system Python first
if python3 -c "import yaml" 2>/dev/null; then
# PyYAML is available in system, run directly
python3 "$PYTHON_SCRIPT" "$@"
exit $?
fi
# PyYAML not available in system, check for venv
if [ ! -d "$VENV_DIR" ]; then
# Create persistent venv
echo "PyYAML not found. Creating persistent virtual environment..." >&2
python3 -m venv "$VENV_DIR" >&2
source "$VENV_DIR/bin/activate" >&2
pip install --quiet pyyaml >&2
echo "Virtual environment created at $VENV_DIR" >&2
echo "" >&2
else
# Use existing venv
source "$VENV_DIR/bin/activate" >&2
fi
# Run the script with venv Python
python3 "$PYTHON_SCRIPT" "$@"
Related skills
How it compares
Pick gitlab-ci-validator when validating GitLab CI job graphs and include rules, not a language-agnostic YAML formatter.
FAQ
What gates does gitlab-ci-validator enforce?
gitlab-ci-validator enforces a 6-gate workflow on .gitlab-ci.yml: syntax-only first, then best-practices, security-only, optional --test-only with gitlab-ci-local, and a final --strict merge gate. Syntax and security are required before merge.
Does gitlab-ci-validator run pipelines?
gitlab-ci-validator performs static configuration review and does not execute production pipelines. Optional --test-only can run gitlab-ci-local when Docker and Node are installed, but the primary value is pre-merge YAML validation.