Gitops Workflow

GitOps Workflow

Complete guide to implementing GitOps workflows with ArgoCD and Flux for automated Kubernetes deployments.

Purpose

Implement declarative, Git-based continuous delivery for Kubernetes using ArgoCD or Flux CD, following OpenGitOps principles.

When to Use This Skill

• Set up GitOps for Kubernetes clusters
• Automate application deployments from Git
• Implement progressive delivery strategies
• Manage multi-cluster deployments
• Configure automated sync policies
• Set up secret management in GitOps

OpenGitOps Principles

1. Declarative - Entire system described declaratively 2. Versioned and Immutable - Desired state stored in Git 3. Pulled Automatically - Software agents pull desired state 4. Continuously Reconciled - Agents reconcile actual vs desired state

ArgoCD Setup

1. Installation

# Create namespace
kubectl create namespace argocd

# Install ArgoCD
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# Get admin password
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

Reference: See references/argocd-setup.md for detailed setup

2. Repository Structure

gitops-repo/
├── apps/
│   ├── production/
│   │   ├── app1/
│   │   │   ├── kustomization.yaml
│   │   │   └── deployment.yaml
│   │   └── app2/
│   └── staging/
├── infrastructure/
│   ├── ingress-nginx/
│   ├── cert-manager/
│   └── monitoring/
└── argocd/
    ├── applications/
    └── projects/

3. Create Application

# argocd/applications/my-app.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/org/gitops-repo
    targetRevision: main
    path: apps/production/my-app
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

4. App of Apps Pattern

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: applications
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/org/gitops-repo
    targetRevision: main
    path: argocd/applications
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  syncPolicy:
    automated: {}

Flux CD Setup

1. Installation

# Install Flux CLI
curl -s https://fluxcd.io/install.sh | sudo bash

# Bootstrap Flux
flux bootstrap github \
  --owner=org \
  --repository=gitops-repo \
  --branch=main \
  --path=clusters/production \
  --personal

2. Create GitRepository

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: my-app
  namespace: flux-system
spec:
  interval: 1m
  url: https://github.com/org/my-app
  ref:
    branch: main

3. Create Kustomization

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: my-app
  namespace: flux-system
spec:
  interval: 5m
  path: ./deploy
  prune: true
  sourceRef:
    kind: GitRepository
    name: my-app

Sync Policies

Auto-Sync Configuration

ArgoCD:

syncPolicy:
  automated:
    prune: true # Delete resources not in Git
    selfHeal: true # Reconcile manual changes
    allowEmpty: false
  retry:
    limit: 5
    backoff:
      duration: 5s
      factor: 2
      maxDuration: 3m

Flux:

spec:
  interval: 1m
  prune: true
  wait: true
  timeout: 5m

Reference: See references/sync-policies.md

Progressive Delivery

Canary Deployment with ArgoCD Rollouts

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: my-app
spec:
  replicas: 5
  strategy:
    canary:
      steps:
        - setWeight: 20
        - pause: { duration: 1m }
        - setWeight: 50
        - pause: { duration: 2m }
        - setWeight: 100

Blue-Green Deployment

strategy:
  blueGreen:
    activeService: my-app
    previewService: my-app-preview
    autoPromotionEnabled: false

Secret Management

External Secrets Operator

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-credentials
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets-manager
    kind: SecretStore
  target:
    name: db-credentials
  data:
    - secretKey: password
      remoteRef:
        key: prod/db/password

Sealed Secrets

# Encrypt secret
kubeseal --format yaml < secret.yaml > sealed-secret.yaml

# Commit sealed-secret.yaml to Git

Best Practices

1. Use separate repos or branches for different environments 2. Implement RBAC for Git repositories 3. Enable notifications for sync failures 4. Use health checks for custom resources 5. Implement approval gates for production 6. Keep secrets out of Git (use External Secrets) 7. Use App of Apps pattern for organization 8. Tag releases for easy rollback 9. Monitor sync status with alerts 10. Test changes in staging first

Troubleshooting

Sync failures:

argocd app get my-app
argocd app sync my-app --prune

Out of sync status:

argocd app diff my-app
argocd app sync my-app --force

Related Skills

• k8s-manifest-generator - For creating manifests
• helm-chart-scaffolding - For packaging applications

ArgoCD Setup and Configuration

Installation Methods

1. Standard Installation

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

2. High Availability Installation

kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/ha/install.yaml

3. Helm Installation

helm repo add argo https://argoproj.github.io/argo-helm
helm install argocd argo/argo-cd -n argocd --create-namespace

Initial Configuration

Access ArgoCD UI

# Port forward
kubectl port-forward svc/argocd-server -n argocd 8080:443

# Get initial admin password
argocd admin initial-password -n argocd

Configure Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: argocd-server-ingress
  namespace: argocd
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
  ingressClassName: nginx
  rules:
    - host: argocd.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: argocd-server
                port:
                  number: 443
  tls:
    - hosts:
        - argocd.example.com
      secretName: argocd-secret

CLI Configuration

Login

argocd login argocd.example.com --username admin

Add Repository

argocd repo add https://github.com/org/repo --username user --password token

Create Application

argocd app create my-app \
  --repo https://github.com/org/repo \
  --path apps/my-app \
  --dest-server https://kubernetes.default.svc \
  --dest-namespace production

SSO Configuration

GitHub OAuth

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
  namespace: argocd
data:
  url: https://argocd.example.com
  dex.config: |
    connectors:
      - type: github
        id: github
        name: GitHub
        config:
          clientID: $GITHUB_CLIENT_ID
          clientSecret: $GITHUB_CLIENT_SECRET
          orgs:
          - name: my-org

RBAC Configuration

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-rbac-cm
  namespace: argocd
data:
  policy.default: role:readonly
  policy.csv: |
    p, role:developers, applications, *, */dev, allow
    p, role:operators, applications, *, */*, allow
    g, my-org:devs, role:developers
    g, my-org:ops, role:operators

Best Practices

1. Enable SSO for production 2. Implement RBAC policies 3. Use separate projects for teams 4. Enable audit logging 5. Configure notifications 6. Use ApplicationSets for multi-cluster 7. Implement resource hooks 8. Configure health checks 9. Use sync windows for maintenance 10. Monitor with Prometheus metrics

GitOps Sync Policies

ArgoCD Sync Policies

Automated Sync

syncPolicy:
  automated:
    prune: true # Delete resources removed from Git
    selfHeal: true # Reconcile manual changes
    allowEmpty: false # Prevent empty sync

Manual Sync

syncPolicy:
  syncOptions:
    - PrunePropagationPolicy=foreground
    - CreateNamespace=true

Sync Windows

syncWindows:
  - kind: allow
    schedule: "0 8 * * *"
    duration: 1h
    applications:
      - my-app
  - kind: deny
    schedule: "0 22 * * *"
    duration: 8h
    applications:
      - "*"

Retry Policy

syncPolicy:
  retry:
    limit: 5
    backoff:
      duration: 5s
      factor: 2
      maxDuration: 3m

Flux Sync Policies

Kustomization Sync

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: my-app
spec:
  interval: 5m
  prune: true
  wait: true
  timeout: 5m
  retryInterval: 1m
  force: false

Source Sync Interval

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: my-app
spec:
  interval: 1m
  timeout: 60s

Health Assessment

Custom Health Checks

# ArgoCD
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
  namespace: argocd
data:
  resource.customizations.health.MyCustomResource: |
    hs = {}
    if obj.status ~= nil then
      if obj.status.conditions ~= nil then
        for i, condition in ipairs(obj.status.conditions) do
          if condition.type == "Ready" and condition.status == "False" then
            hs.status = "Degraded"
            hs.message = condition.message
            return hs
          end
          if condition.type == "Ready" and condition.status == "True" then
            hs.status = "Healthy"
            hs.message = condition.message
            return hs
          end
        end
      end
    end
    hs.status = "Progressing"
    hs.message = "Waiting for status"
    return hs

Sync Options

Common Sync Options

• PrunePropagationPolicy=foreground - Wait for pruned resources to be deleted
• CreateNamespace=true - Auto-create namespace
• Validate=false - Skip kubectl validation
• PruneLast=true - Prune resources after sync
• RespectIgnoreDifferences=true - Honor ignore differences
• ApplyOutOfSyncOnly=true - Only apply out-of-sync resources

Best Practices

1. Use automated sync for non-production 2. Require manual approval for production 3. Configure sync windows for maintenance 4. Implement health checks for custom resources 5. Use selective sync for large applications 6. Configure appropriate retry policies 7. Monitor sync failures with alerts 8. Use prune with caution in production 9. Test sync policies in staging 10. Document sync behavior for teams