Env Secrets Manager

Env & Secrets Manager

Tier: POWERFUL Category: Engineering Domain: Security / DevOps / Configuration Management

---

Overview

Manage environment-variable hygiene and secrets safety across local development and production workflows. This skill focuses on practical auditing, drift awareness, and rotation readiness.

Core Capabilities

• .env and .env.example lifecycle guidance
• Secret leak detection for repository working trees
• Severity-based findings for likely credentials
• Operational pointers for rotation and containment
• Integration-ready outputs for CI checks

---

When to Use

• Before pushing commits that touched env/config files
• During security audits and incident triage
• When onboarding contributors who need safe env conventions
• When validating that no obvious secrets are hardcoded

---

Quick Start

# Scan a repository for likely secret leaks
python3 scripts/env_auditor.py /path/to/repo

# JSON output for CI pipelines
python3 scripts/env_auditor.py /path/to/repo --json

---

Recommended Workflow

1. Run scripts/env_auditor.py on the repository root. 2. Prioritize critical and high findings first. 3. Rotate real credentials and remove exposed values. 4. Update .env.example and .gitignore as needed. 5. Add or tighten pre-commit/CI secret scanning gates.

---

Reference Docs

• references/validation-detection-rotation.md
• references/secret-patterns.md

---

Common Pitfalls

• Committing real values in .env.example
• Rotating one system but missing downstream consumers
• Logging secrets during debugging or incident response
• Treating suspected leaks as low urgency without validation

Best Practices

1. Use a secret manager as the production source of truth. 2. Keep dev env files local and gitignored. 3. Enforce detection in CI before merge. 4. Re-test application paths immediately after credential rotation.

---

Cloud Secret Store Integration

Production applications should never read secrets from .env files or environment variables baked into container images. Use a dedicated secret store instead.

Provider Comparison

Selection Guidance

• Single cloud provider — use the cloud-native secret manager. It integrates tightly with IAM, reduces operational overhead, and costs less than self-hosting.
• Multi-cloud or hybrid — use HashiCorp Vault. It provides a uniform API across environments and supports dynamic secret generation (database credentials, cloud IAM keys) that expire automatically.
• Kubernetes-heavy — combine External Secrets Operator with any backend above to sync secrets into K8s Secret objects without hardcoding.

Application Access Patterns

1. SDK/API pull — application fetches secret at startup or on-demand via provider SDK. 2. Sidecar injection — a sidecar container (e.g., Vault Agent) writes secrets to a shared volume or injects them as environment variables. 3. Init container — a Kubernetes init container fetches secrets before the main container starts. 4. CSI driver — secrets mount as a filesystem volume via the Secrets Store CSI Driver.

Cross-reference: See engineering/secrets-vault-manager for production vault infrastructure patterns, HA deployment, and disaster recovery procedures.

---

Secret Rotation Workflow

Stale secrets are a liability. Rotation ensures that even if a credential leaks, its useful lifetime is bounded.

Phase 1: Detection

• Track secret creation and expiry dates in your secret store metadata.
• Set alerts at 30, 14, and 7 days before expiry.
• Use scripts/env_auditor.py to flag secrets with no recorded rotation date.

Phase 2: Rotation

1. Generate a new credential (API key, database password, certificate). 2. Deploy the new credential to all consumers (apps, services, pipelines) in parallel. 3. Verify each consumer can authenticate using the new credential. 4. Revoke the old credential only after all consumers are confirmed healthy. 5. Update metadata with the new rotation timestamp and next rotation date.

Phase 3: Automation

• AWS Secrets Manager — use built-in Lambda-based rotation for RDS, Redshift, and DocumentDB.
• HashiCorp Vault — configure dynamic secrets with TTLs; credentials are generated on-demand and auto-expire.
• Azure Key Vault — use Event Grid notifications to trigger rotation functions.
• GCP Secret Manager — use Pub/Sub notifications tied to Cloud Functions for rotation logic.

Emergency Rotation Checklist

When a secret is confirmed leaked:

1. Immediately revoke the compromised credential at the provider level. 2. Generate and deploy a replacement credential to all consumers. 3. Audit access logs for unauthorized usage during the exposure window. 4. Scan git history, CI logs, and artifact registries for the leaked value. 5. File an incident report documenting scope, timeline, and remediation steps. 6. Review and tighten detection controls to prevent recurrence.

---

CI/CD Secret Injection

Secrets in CI/CD pipelines require careful handling to avoid exposure in logs, artifacts, or pull request contexts.

GitHub Actions

• Use repository secrets or environment secrets via ${{ secrets.SECRET_NAME }}.
• Prefer OIDC federation (aws-actions/configure-aws-credentials with role-to-assume) over long-lived access keys.
• Environment secrets with required reviewers add approval gates for production deployments.
• GitHub automatically masks secrets in logs, but avoid echo or toJSON() on secret values.

GitLab CI

• Store secrets as CI/CD variables with the masked and protected flags enabled.
• Use HashiCorp Vault integration (secrets:vault) for dynamic secret injection without storing values in GitLab.
• Scope variables to specific environments (production, staging) to enforce least privilege.

Universal Patterns

• Never echo or print secret values in pipeline output, even for debugging.
• Use short-lived tokens (OIDC, STS AssumeRole) instead of static credentials wherever possible.
• Restrict PR access — do not expose secrets to pipelines triggered by forks or untrusted branches.
• Rotate CI secrets on the same schedule as application secrets; pipeline credentials are attack vectors too.
• Audit pipeline logs periodically for accidental secret exposure that masking may have missed.

---

Pre-Commit Secret Detection

Catching secrets before they reach version control is the most cost-effective defense. Two leading tools cover this space.

gitleaks

# .gitleaks.toml — minimal configuration
[extend]
useDefault = true

[[rules]]
id = "custom-internal-token"
description = "Internal service token pattern"
regex = '''INTERNAL_TOKEN_[A-Za-z0-9]{32}'''
secretGroup = 0

• Install: brew install gitleaks or download from GitHub releases.
• Pre-commit hook: gitleaks git --pre-commit --staged
• Baseline scanning: gitleaks detect --source . --report-path gitleaks-report.json
• Manage false positives in .gitleaksignore (one fingerprint per line).

detect-secrets

# Generate baseline
detect-secrets scan --all-files > .secrets.baseline

# Pre-commit hook (via pre-commit framework)
# .pre-commit-config.yaml
repos:
  - repo: https://github.com/Yelp/detect-secrets
    rev: v1.5.0
    hooks:
      - id: detect-secrets
        args: ['--baseline', '.secrets.baseline']

• Supports custom plugins for organization-specific patterns.
• Audit workflow: detect-secrets audit .secrets.baseline interactively marks true/false positives.

False Positive Management

• Maintain .gitleaksignore or .secrets.baseline in version control so the whole team shares exclusions.
• Review false positive lists during security audits — patterns may mask real leaks over time.
• Prefer tightening regex patterns over broadly ignoring files.

---

Audit Logging

Knowing who accessed which secret and when is critical for incident investigation and compliance.

Cloud-Native Audit Trails

Alerting Strategy

• Alert on access from unknown IP ranges or service accounts outside the expected set.
• Alert on bulk secret reads (more than N secrets accessed within a time window).
• Alert on access outside deployment windows when no CI/CD pipeline is running.
• Feed audit logs into your SIEM (Splunk, Datadog, Elastic) for correlation with other security events.
• Review audit logs quarterly as part of access recertification.

---

Cross-References

This skill covers env hygiene and secret detection. For deeper coverage of related domains, see:

Secret Pattern Reference

Detection Categories

Critical

• OpenAI-like keys (sk-...)
• GitHub personal access tokens (ghp_...)
• AWS access key IDs (AKIA...)

High

• Slack tokens (xox...)
• Private key PEM blocks
• Hardcoded assignments to secret, token, password, api_key

Medium

• JWT-like tokens in plaintext
• Suspected credentials in docs/scripts that should be redacted

Severity Guidance

• critical: immediate rotation required; treat as active incident
• high: likely sensitive; investigate and rotate if real credential
• medium: possible exposure; verify context and sanitize where needed

Response Playbook

1. Revoke or rotate exposed credential. 2. Identify blast radius (services, environments, users). 3. Remove from code/history where possible. 4. Add preventive controls (pre-commit hooks, CI secret scans). 5. Verify monitoring and access logs for abuse.

Preventive Baseline

• Commit only .env.example, never .env.
• Keep .gitignore patterns for env and key material.
• Use secret managers for staging/prod.
• Redact sensitive values from logs and debug output.

env-secrets-manager reference

Required Variable Validation Script

#!/bin/bash
# scripts/validate-env.sh
# Run at app startup or in CI before deploy
# Exit 1 if any required var is missing or empty

set -euo pipefail

MISSING=()
WARNINGS=()

# --- Define required vars by environment ---
ALWAYS_REQUIRED=(
  APP_SECRET
  APP_URL
  DATABASE_URL
  AUTH_JWT_SECRET
  AUTH_REFRESH_SECRET
)

PROD_REQUIRED=(
  STRIPE_SECRET_KEY
  STRIPE_WEBHOOK_SECRET
  SENTRY_DSN
)

# --- Check always-required vars ---
for var in "${ALWAYS_REQUIRED[@]}"; do
  if [ -z "${!var:-}" ]; then
    MISSING+=("$var")
  fi
done

# --- Check prod-only vars ---
if [ "${APP_ENV:-}" = "production" ] || [ "${NODE_ENV:-}" = "production" ]; then
  for var in "${PROD_REQUIRED[@]}"; do
    if [ -z "${!var:-}" ]; then
      MISSING+=("$var (required in production)")
    fi
  done
fi

# --- Validate format/length constraints ---
if [ -n "${AUTH_JWT_SECRET:-}" ] && [ ${#AUTH_JWT_SECRET} -lt 32 ]; then
  WARNINGS+=("AUTH_JWT_SECRET is shorter than 32 chars — insecure")
fi

if [ -n "${DATABASE_URL:-}" ]; then
  if ! echo "$DATABASE_URL" | grep -qE "^(postgres|postgresql|mysql|mongodb|redis)://"; then
    WARNINGS+=("DATABASE_URL doesn't look like a valid connection string")
  fi
fi

if [ -n "${APP_PORT:-}" ]; then
  if ! [[ "$APP_PORT" =~ ^[0-9]+$ ]] || [ "$APP_PORT" -lt 1 ] || [ "$APP_PORT" -gt 65535 ]; then
    WARNINGS+=("APP_PORT=$APP_PORT is not a valid port number")
  fi
fi

# --- Report ---
if [ ${#WARNINGS[@]} -gt 0 ]; then
  echo "WARNINGS:"
  for w in "${WARNINGS[@]}"; do
    echo "  ⚠️  $w"
  done
fi

if [ ${#MISSING[@]} -gt 0 ]; then
  echo ""
  echo "FATAL: Missing required environment variables:"
  for var in "${MISSING[@]}"; do
    echo "  ❌  $var"
  done
  echo ""
  echo "Copy .env.example to .env and fill in missing values."
  exit 1
fi

echo "✅  All required environment variables are set"

Node.js equivalent:

// src/config/validateEnv.ts
const required = [
  'APP_SECRET', 'APP_URL', 'DATABASE_URL',
  'AUTH_JWT_SECRET', 'AUTH_REFRESH_SECRET',
]

const missing = required.filter(key => !process.env[key])

if (missing.length > 0) {
  console.error('FATAL: Missing required environment variables:', missing)
  process.exit(1)
}

if (process.env.AUTH_JWT_SECRET && process.env.AUTH_JWT_SECRET.length < 32) {
  console.error('FATAL: AUTH_JWT_SECRET must be at least 32 characters')
  process.exit(1)
}

export const config = {
  appSecret: process.env.APP_SECRET!,
  appUrl: process.env.APP_URL!,
  databaseUrl: process.env.DATABASE_URL!,
  jwtSecret: process.env.AUTH_JWT_SECRET!,
  refreshSecret: process.env.AUTH_REFRESH_SECRET!,
  stripeKey: process.env.STRIPE_SECRET_KEY,  // optional
  port: parseInt(process.env.APP_PORT ?? '3000', 10),
} as const

---

Secret Leak Detection

Scan Working Tree

#!/bin/bash
# scripts/scan-secrets.sh
# Scan staged files and working tree for common secret patterns

FAIL=0

check() {
  local label="$1"
  local pattern="$2"
  local matches

  matches=$(git diff --cached -U0 2>/dev/null | grep "^+" | grep -vE "^(\+\+\+|#|\/\/)" | \
    grep -E "$pattern" | grep -v ".env.example" | grep -v "test\|mock\|fixture\|fake" || true)

  if [ -n "$matches" ]; then
    echo "SECRET DETECTED [$label]:"
    echo "$matches" | head -5
    FAIL=1
  fi
}

# AWS Access Keys
check "AWS Access Key" "AKIA[0-9A-Z]{16}"
check "AWS Secret Key" "aws_secret_access_key\s*=\s*['\"]?[A-Za-z0-9/+]{40}"

# Stripe
check "Stripe Live Key"   "sk_live_[0-9a-zA-Z]{24,}"
check "Stripe Test Key"   "sk_test_[0-9a-zA-Z]{24,}"
check "Stripe Webhook"    "whsec_[0-9a-zA-Z]{32,}"

# JWT / Generic secrets
check "Hardcoded JWT"     "eyJ[A-Za-z0-9_-]{20,}\.[A-Za-z0-9_-]{20,}"
check "Generic Secret"    "(secret|password|passwd|api_key|apikey|token)\s*[:=]\s*['\"][^'\"]{12,}['\"]"

# Private keys
check "Private Key Block" "-----BEGIN (RSA |EC |DSA |OPENSSH )?PRIVATE KEY-----"
check "PEM Certificate"   "-----BEGIN CERTIFICATE-----"

# Connection strings with credentials
check "DB Connection"     "(postgres|mysql|mongodb)://[^:]+:[^@]+@"
check "Redis Auth"        "redis://:[^@]+@\|rediss://:[^@]+@"

# Google
check "Google API Key"    "AIza[0-9A-Za-z_-]{35}"
check "Google OAuth"      "[0-9]+-[0-9A-Za-z_]{32}\.apps\.googleusercontent\.com"

# GitHub
check "GitHub Token"      "gh[ps]_[A-Za-z0-9]{36,}"
check "GitHub Fine-grained" "github_pat_[A-Za-z0-9_]{82}"

# Slack
check "Slack Token"       "xox[baprs]-[0-9A-Za-z]{10,}"
check "Slack Webhook"     "https://hooks\.slack\.com/services/[A-Z0-9]{9,}/[A-Z0-9]{9,}/[A-Za-z0-9]{24,}"

# Twilio
check "Twilio SID"        "AC[a-z0-9]{32}"
check "Twilio Token"      "SK[a-z0-9]{32}"

if [ $FAIL -eq 1 ]; then
  echo ""
  echo "BLOCKED: Secrets detected in staged changes."
  echo "Remove secrets before committing. Use environment variables instead."
  echo "If this is a false positive, add it to .secretsignore or use:"
  echo "  git commit --no-verify  (only if you're 100% certain it's safe)"
  exit 1
fi

echo "No secrets detected in staged changes."

Scan Git History (post-incident)

#!/bin/bash
# scripts/scan-history.sh — scan entire git history for leaked secrets

PATTERNS=(
  "AKIA[0-9A-Z]{16}"
  "sk_live_[0-9a-zA-Z]{24}"
  "sk_test_[0-9a-zA-Z]{24}"
  "-----BEGIN.*PRIVATE KEY-----"
  "AIza[0-9A-Za-z_-]{35}"
  "ghp_[A-Za-z0-9]{36}"
  "xox[baprs]-[0-9A-Za-z]{10,}"
)

for pattern in "${PATTERNS[@]}"; do
  echo "Scanning for: $pattern"
  git log --all -p --no-color 2>/dev/null | \
    grep -n "$pattern" | \
    grep "^+" | \
    grep -v "^+++" | \
    head -10
done

# Alternative: use truffleHog or gitleaks for comprehensive scanning
# gitleaks detect --source . --log-opts="--all"
# trufflehog git file://. --only-verified

---

Pre-commit Hook Installation

#!/bin/bash
# Install the pre-commit hook
HOOK_PATH=".git/hooks/pre-commit"

cat > "$HOOK_PATH" << 'HOOK'
#!/bin/bash
# Pre-commit: scan for secrets before every commit

SCRIPT="scripts/scan-secrets.sh"

if [ -f "$SCRIPT" ]; then
  bash "$SCRIPT"
else
  # Inline fallback if script not present
  if git diff --cached -U0 | grep "^+" | grep -qE "AKIA[0-9A-Z]{16}|sk_live_|-----BEGIN.*PRIVATE KEY"; then
    echo "BLOCKED: Possible secret detected in staged changes."
    exit 1
  fi
fi
HOOK

chmod +x "$HOOK_PATH"
echo "Pre-commit hook installed at $HOOK_PATH"

Using pre-commit framework (recommended for teams):

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.18.0
    hooks:
      - id: gitleaks

  - repo: local
    hooks:
      - id: validate-env-example
        name: "check-envexample-is-up-to-date"
        language: script
        entry: bash scripts/check-env-example.sh
        pass_filenames: false

---

Credential Rotation Workflow

When a secret is leaked or compromised:

Step 1 — Detect & Confirm

# Confirm which secret was exposed
git log --all -p --no-color | grep -A2 -B2 "AKIA\|sk_live_\|SECRET"

# Check if secret is in any open PRs
gh pr list --state open | while read pr; do
  gh pr diff $(echo $pr | awk '{print $1}') | grep -E "AKIA|sk_live_" && echo "Found in PR: $pr"
done

Step 2 — Identify Exposure Window

# Find first commit that introduced the secret
git log --all -p --no-color -- "*.env" "*.json" "*.yaml" "*.ts" "*.py" | \
  grep -B 10 "THE_LEAKED_VALUE" | grep "^commit" | tail -1

# Get commit date
git show --format="%ci" COMMIT_HASH | head -1

# Check if secret appears in public repos (GitHub)
gh api search/code -X GET -f q="THE_LEAKED_VALUE" | jq '.total_count, .items[].html_url'

Step 3 — Rotate Credential

Per service — rotate immediately:

• AWS: IAM console → delete access key → create new → update everywhere
• Stripe: Dashboard → Developers → API keys → Roll key
• GitHub PAT: Settings → Developer Settings → Personal access tokens → Revoke → Create new
• DB password: ALTER USER app_user PASSWORD 'new-strong-password-here';
• JWT secret: Rotate key (all existing sessions invalidated — users re-login)

Step 4 — Update All Environments

# Update secret manager (source of truth)
# Then redeploy to pull new values

# Vault KV v2
vault kv put secret/myapp/prod \
  STRIPE_SECRET_KEY="sk_live_NEW..." \
  APP_SECRET="new-secret-here"

# AWS SSM
aws ssm put-parameter \
  --name "/myapp/prod/STRIPE_SECRET_KEY" \
  --value "sk_live_NEW..." \
  --type "SecureString" \
  --overwrite

# 1Password
op item edit "MyApp Prod" \
  --field "STRIPE_SECRET_KEY[password]=sk_live_NEW..."

# Doppler
doppler secrets set STRIPE_SECRET_KEY="sk_live_NEW..." --project myapp --config prod

Step 5 — Remove from Git History

# WARNING: rewrites history — coordinate with team first
git filter-repo --path-glob "*.env" --invert-paths

# Or remove specific string from all commits
git filter-repo --replace-text <(echo "LEAKED_VALUE==>REDACTED")

# Force push all branches (requires team coordination + force push permissions)
git push origin --force --all

# Notify all developers to re-clone

Step 6 — Verify

# Confirm secret no longer in history
git log --all -p | grep "LEAKED_VALUE" | wc -l  # should be 0

# Test new credentials work
curl -H "Authorization: Bearer $NEW_TOKEN" https://api.service.com/test

# Monitor for unauthorized usage of old credential (check service audit logs)

---