Shell Scripting

Shell Scripting

Write defensively. Shell defaults are hostile — unquoted variables split, unset variables vanish silently, failed commands continue. Every rule here exists to counteract a specific shell default that causes bugs.

References

Extended examples, code patterns, and lookup tables for the rules below.

• Strict mode, error handling, traps, debugging — [${CLAUDE_SKILL_DIR}/references/strict-mode.md]: errexit

caveats, pipefail examples, trap patterns, temp file safety, debugging techniques

• Quoting rules, word splitting, globbing — [${CLAUDE_SKILL_DIR}/references/quoting.md]: Three quoting mechanisms,

"$@" vs "$*", array expansion, printf vs echo, nested quoting

• POSIX sh vs bash, portable constructs — [${CLAUDE_SKILL_DIR}/references/portability.md]: Feature comparison, GNU

vs BSD tool differences, portable pattern catalog

• Argument parsing, getopts, validation — [${CLAUDE_SKILL_DIR}/references/arguments.md]: getopts template, manual

long-option parsing, validation patterns, usage messages, stdin detection

• Common shell scripting mistakes — [${CLAUDE_SKILL_DIR}/references/pitfalls.md]: Iteration pitfalls, variable

pitfalls, test pitfalls, pipeline pitfalls, arithmetic traps

• Pure bash/sh alternatives to external commands — [${CLAUDE_SKILL_DIR}/references/builtins.md]: Parameter

expansion, replacing sed/cut/basename/expr, arrays, read patterns, arithmetic

Script Header

Every bash script starts with:

#!/usr/bin/env bash
set -euo pipefail

• Shebang: Use #!/usr/bin/env bash — not #!/bin/bash. The env lookup is more portable across systems where

bash is not at /bin/bash.

• `set -e` (errexit): Exit on command failure. Understand the exceptions: commands in if/while conditions, left

side of &&/||, and negated commands (!) do not trigger errexit.

• `set -u` (nounset): Error on unset variables. Use ${VAR:-default} for optional variables.
• `set -o pipefail`: Pipeline returns the rightmost failing command's exit code, not the last command's.
• For POSIX sh scripts: Use #!/bin/sh. Drop pipefail (not POSIX). Use set -eu with caution — set -e behavior

varies across sh implementations.

• File header comment: After the shebang, add a brief description of what the script does.

#!/usr/bin/env bash
set -euo pipefail
#
# deploy.sh — Build and deploy the application to staging.

Quoting

Quoting is the single most important discipline. Unquoted variables undergo word splitting (breaks on IFS characters) and pathname expansion (glob characters match filenames). Both are silent and devastating.

Core Rules

• Always double-quote variable expansions: "$var", "${var}".
• Always double-quote command substitutions: "$(command)".
• Use `"$@"` to pass arguments through. Never $* or $@ unquoted. "$@" preserves each argument as a separate

word. "$*" joins them.

• Quote array expansions: "${arr[@]}" expands each element as a separate word. Unquoted ${arr[@]} undergoes word

splitting.

• Leave globs unquoted: for f in *.txt — the glob must expand. But always quote variables inside the loop: "$f".
• Leave `[[ ]]` right-hand patterns unquoted when doing glob or regex matching. Quote the right side for literal

string comparison.

• Use single quotes for literal strings that need no expansion: grep 'pattern' file.
• Use `printf` instead of `echo` for data output. echo interprets -n, -e as options on some platforms.

printf '%s\n' "$var" is always safe.

When Quoting Is Not Needed

• Right side of assignment: var=$other (no splitting in assignment context)
• Inside (( )) arithmetic: (( x + y ))
• Inside [[ ]] on the left side: [[ $var == pattern ]]
• Integer special variables: $?, $#, $$ (guaranteed no spaces)
• case word: case $var in ...

Variable Handling

• Naming: lowercase with underscores for local variables (file_path, line_count). UPPER_CASE for

exported/environment variables and constants (PATH, MAX_RETRIES).

• Declare constants with `readonly`:

  readonly CONFIG_DIR="/etc/myapp"

• Use `local` in functions to prevent variable leakage into global scope. Declare and assign on separate lines when

capturing command output:

  local result
  result=$(some_command)

Combined local result=$(cmd) masks the exit code — local always returns 0.

• Default values: Use ${VAR:-default} to provide defaults without modifying the variable. Use ${VAR:=default} to

set and use.

• Required variables: Use ${VAR:?error message} to abort if unset.
• Arrays for lists: Use bash arrays instead of space-delimited strings.

  files=("file one.txt" "file two.txt")
  command "${files[@]}"

Error Handling

• Check every command that can fail. Use || exit 1, || return 1, or explicit if blocks. Especially cd,

mkdir, rm, cp, mv.

  cd "$dir" || exit 1

• Trap for cleanup. Use trap on EXIT for reliable cleanup:

  tmpfile=$(mktemp) || exit 1
  trap 'rm -f "$tmpfile"' EXIT

• Use `mktemp` for temp files. Never hardcoded temp paths. Always clean up via trap.
• Error messages to stderr:

  die() { printf '%s\n' "$1" >&2; exit "${2:-1}"; }

• Exit codes: Return 0 for success, non-zero for failure. Use meaningful codes: 1 for general error, 2 for usage

error, 64+ for application-specific errors (following sysexits convention).

• Never use `set -e` as a substitute for error handling. It has many edge cases. Use it as a safety net, but still

check critical commands explicitly.

Functions

• Declare with `name() { ... }` — no function keyword (it's not POSIX and adds nothing in bash).
• Use `local` for all function variables. Bash functions share the caller's scope by default — every undeclared

variable is global.

• Return values via exit code (0 = success, non-zero = failure) or via stdout. Never rely on global variables for

function output.

• Separate `local` declaration from command substitution:

  my_func() {
    local output
    output=$(some_command) || return 1
  }

• Put all functions before executable code. Only set statements, source commands, and constants should precede

function definitions.

• Use `main` for scripts with multiple functions. Call main "$@" as the last line. This keeps the entry point

obvious and lets all variables be local.

  main() {
    local arg="$1"
    # ...
  }
  main "$@"

Control Flow

Conditionals

• Use `[[ ]]` in bash — it prevents word splitting, supports &&/|| inside the test, and enables pattern/regex

matching. In POSIX sh, use [ ] with all variables quoted.

• Use `(( ))` for numeric comparisons:

  if (( count > 10 )); then ...

In POSIX sh: [ "$count" -gt 10 ].

• Use `==` in `[[ ]]` and `=` in `[ ]` for string equality.
• Test empty/non-empty explicitly: [[ -z "$var" ]] and [[ -n "$var" ]] — not [[ "$var" ]].
• Never use `&&`/`||` as if/then/else:

  # WRONG — cmd3 runs if cmd2 fails, even when cmd1 succeeds
  cmd1 && cmd2 || cmd3
  # RIGHT
  if cmd1; then cmd2; else cmd3; fi

Loops

• Never parse `ls` output. Use globs:

  for f in ./*.txt; do
    [[ -e "$f" ]] || continue
    process "$f"
  done

• Use `while read` for line-oriented input:

  while IFS= read -r line; do
    printf '%s\n' "$line"
  done < file

The IFS= prevents leading/trailing whitespace trimming. The -r prevents backslash interpretation.

• Use process substitution to avoid subshell variable loss:

  while IFS= read -r line; do
    (( count++ ))
  done < <(command)
  echo "$count"  # preserved

• Use `find -print0` with `read -d ''` for filenames with special characters:

  while IFS= read -r -d '' file; do
    process "$file"
  done < <(find . -type f -print0)

Case Statements

• `case` for multi-way branching:

  case "$1" in
    start)   do_start ;;
    stop)    do_stop ;;
    restart) do_stop; do_start ;;
    *)       die "Unknown command: $1" ;;
  esac

• Indent patterns by 2 spaces from case. Put ;; on the same line as the action for one-liners, on its own line

for multi-line actions.

Input Handling

• Use `getopts` for short options. It is POSIX, handles combined flags (-vf), and manages OPTARG/OPTIND

correctly.

• Use manual parsing for long options. while (( $# > 0 )); do case loop with explicit -- handling.
• Always handle `--` to end option processing — prevents filenames starting with - from being interpreted as

options.

• Always use `--` when passing variables to commands:

  rm -- "$file"
  grep -- "$pattern" "$file"

• Prefix globs with `./` to prevent files named -rf from becoming options:

  for f in ./*; do
    rm -- "$f"
  done

• Provide a `usage()` function for any script that takes arguments. Print to stderr and exit with code 64

(EX_USAGE).

• Validate arguments early. Check counts, types, file existence before starting work.
• Detect stdin vs terminal: [[ -t 0 ]] tests whether stdin is a terminal.

Formatting

• Indent with 2 spaces. No tabs (except in <<- heredocs).
• Maximum line length: 80 characters. Use \ continuation or heredocs for long strings.
• `; then` and `; do` on the same line as if/for/while:

  if [[ -f "$file" ]]; then
  for item in "${arr[@]}"; do
  while read -r line; do

• Split long pipelines one per line with | on the continuation line:

  command1 \
    | command2 \
    | command3

• Use `$(command)` not backticks. $() nests cleanly and is readable. Backticks require escaping and don't nest.
• Prefer `${var}` braces for all variables except positional parameters ($1-$9) and special parameters ($?,

$#, etc.).

Portability

• Choose your target. Decide upfront whether you need POSIX sh compatibility or can require bash.
• If targeting bash: use #!/usr/bin/env bash, use [[ ]], arrays, and process substitution freely. Specify

minimum bash version if using 4.0+ features (associative arrays, mapfile, case modification).

• If targeting POSIX sh: use #!/bin/sh, use [ ] with quoted variables, no arrays, no [[ ]], no (( )), no

local (technically non-POSIX but widely supported), no process substitution.

• macOS ships bash 3.2 permanently. If targeting macOS without requiring Homebrew bash, avoid bash 4+ features.
• Avoid GNU-specific tool options when portability matters: sed -i, grep -P, GNU date flags. Document the

dependency when GNU tools are required.

• Use `command -v` to check if a program is available — not which (which is not a builtin and behaves differently

across systems).

ShellCheck Integration

• Run ShellCheck on all scripts. It catches quoting errors, portability issues, and common pitfalls automatically.
• Use a directive comment for intentional violations:

  # shellcheck disable=SC2086
  word_split_is_intentional $var

• Specify shell dialect if the shebang is absent or ambiguous:

  # shellcheck shell=bash

• Common ShellCheck codes to know:
• SC2086: Double quote to prevent globbing and word splitting
• SC2046: Quote this to prevent word splitting
• SC2034: Variable appears unused (might be exported or sourced)
• SC2155: Declare and assign separately to avoid masking return values
• SC2164: Use cd ... || exit in case cd fails

Application

When writing shell scripts: Apply all rules silently. Produce clean, defensive code. Use strict mode, quote everything, handle errors, use arrays for lists.

When reviewing shell scripts: Cite the specific rule violated. Show the fix inline. Prioritize: quoting bugs > error handling gaps > style issues.

Integration

• the-coder provides the overall coding workflow (discover, plan, verify)
• Language plugins (golang, javascript) handle language-specific tooling
• This skill handles shell-specific correctness and defensive patterns

Quote everything. Handle every error. Trust nothing.

Argument Parsing

Shell scripts need robust argument parsing to be usable. This reference covers getopts, manual parsing, and validation patterns.

getopts (POSIX)

The standard, portable approach for short options:

usage() {
  printf 'Usage: %s [-v] [-f FILE] [-n COUNT] [ARGS...]\n' "${0##*/}" >&2
  exit 64
}

verbose=false
file=""
count=1

while getopts ':vf:n:h' opt; do
  case "$opt" in
    v) verbose=true ;;
    f) file="$OPTARG" ;;
    n) count="$OPTARG" ;;
    h) usage ;;
    :) printf 'Error: -%s requires an argument\n' "$OPTARG" >&2; usage ;;
    ?) printf 'Error: unknown option -%s\n' "$OPTARG" >&2; usage ;;
  esac
done
shift $((OPTIND - 1))

# Remaining positional arguments are now in "$@"

`getopts` rules:

1. Leading : in optstring enables silent error handling 2. Colon after letter means option takes an argument (f:) 3. $OPTARG holds the argument for options that take one 4. $OPTIND is the index of the next argument to process 5. Always shift $((OPTIND - 1)) after the loop

Limitation: getopts only handles short options (-v, -f FILE). For long options (--verbose, --file FILE), use manual parsing.

Manual Parsing (Long Options)

verbose=false
file=""
count=1
args=()

while (( $# > 0 )); do
  case "$1" in
    -v|--verbose) verbose=true; shift ;;
    -f|--file)
      [[ -n "${2:-}" ]] || die "Error: --file requires an argument"
      file="$2"; shift 2
      ;;
    -n|--count)
      [[ -n "${2:-}" ]] || die "Error: --count requires an argument"
      count="$2"; shift 2
      ;;
    -h|--help) usage ;;
    --)  shift; break ;;  # end of options
    -*)  die "Error: unknown option $1" ;;
    *)   args+=("$1"); shift ;;
  esac
done

# Append remaining arguments after --
args+=("$@")

Rules:

1. Always handle -- to explicitly end option processing 2. Collect unknown positional arguments into an array 3. Validate required arguments early

Argument Validation

# Check minimum argument count
(( $# >= 2 )) || die "Usage: ${0##*/} SOURCE DEST"

# Validate file exists
[[ -f "$file" ]] || die "Error: file not found: $file"

# Validate directory exists
[[ -d "$dir" ]] || die "Error: directory not found: $dir"

# Validate numeric argument
[[ "$count" =~ ^[0-9]+$ ]] || die "Error: count must be a positive integer"

# Validate non-empty
[[ -n "$name" ]] || die "Error: name cannot be empty"

# Validate file is readable
[[ -r "$file" ]] || die "Error: cannot read file: $file"

Usage Messages

usage() {
  cat <<EOF
Usage: ${0##*/} [OPTIONS] FILE [FILE...]

Process files according to specified options.

Options:
  -v, --verbose    Enable verbose output
  -f, --format FMT Output format (json, csv, text)
  -o, --output DIR Output directory (default: current directory)
  -n, --dry-run    Show what would be done without doing it
  -h, --help       Show this help message

Examples:
  ${0##*/} -v data.csv
  ${0##*/} --format json --output /tmp input.txt
EOF
  exit 64
}

Stdin Detection

# Check if stdin is a terminal (interactive) or pipe/file
if [[ -t 0 ]]; then
  echo "Reading from terminal"
else
  echo "Reading from pipe/file"
fi

# Read from file argument or stdin
input="${1:--}"  # default to stdin
if [[ "$input" == "-" ]]; then
  cat
else
  cat "$input"
fi

Shift Patterns

# Process and remove first argument
command="$1"
shift

# Process pairs of arguments
while (( $# >= 2 )); do
  key="$1"
  value="$2"
  shift 2
done

Builtins and Parameter Expansion

Prefer shell builtins over external commands. Each external command forks a process — in a loop, this creates significant overhead. Shell builtins execute in-process and are both faster and more portable.

Parameter Expansion

String Operations

• Length — ${#var}: name="hello"; echo ${#name} -> 5
• Substring — ${var:offset:length}: ${name:1:3} -> ell
• Remove shortest prefix — ${var#pattern}: ${path#*/} -> b/c.txt
• Remove longest prefix — ${var##pattern}: ${path##*/} -> c.txt
• Remove shortest suffix — ${var%pattern}: ${path%/*} -> a/b
• Remove longest suffix — ${var%%pattern}: ${path%%/*} -> a
• First substitution — ${var/pat/rep}: ${name/l/L} -> heLlo
• All substitutions — ${var//pat/rep}: ${name//l/L} -> heLLo
• Prefix substitution — ${var/#pat/rep}: ${name/#he/HE} -> HEllo
• Suffix substitution — ${var/%lo/LO}: ${name/%lo/LO} -> helLO

Case Modification (bash 4+)

• First char uppercase — ${var^}: ${name^} -> Hello
• All uppercase — ${var^^}: ${name^^} -> HELLO
• First char lowercase — ${var,}: ${VAR,} -> hELLO
• All lowercase — ${var,,}: ${VAR,,} -> hello

Default Values

• `${var:-word}` — Use word if var is unset or empty
• `${var-word}` — Use word if var is unset
• `${var:=word}` — Assign word if var is unset or empty
• `${var:+word}` — Use word if var is set and non-empty
• `${var:?msg}` — Error with msg if var is unset or empty

Indirection (bash)

name="greeting"
greeting="hello"
echo "${!name}"     # hello (indirect expansion)

Replacing External Commands

basename / dirname

path="/home/user/file.txt"

# Instead of: basename "$path"
echo "${path##*/}"              # file.txt

# Instead of: basename "$path" .txt
name="${path##*/}"; echo "${name%.txt}"  # file

# Instead of: dirname "$path"
echo "${path%/*}"               # /home/user

sed for simple substitutions

var="hello-world-foo"

# Instead of: echo "$var" | sed 's/-/_/g'
echo "${var//-/_}"              # hello_world_foo

# Instead of: echo "$var" | sed 's/^hello//'
echo "${var#hello}"             # -world-foo

cut for field extraction

line="field1:field2:field3"

# Instead of: echo "$line" | cut -d: -f1
IFS=: read -r first rest <<< "$line"
echo "$first"                   # field1

# Or using parameter expansion
echo "${line%%:*}"              # field1

wc -c for string length

# Instead of: echo -n "$var" | wc -c
echo "${#var}"

expr for arithmetic

# Instead of: i=$(expr $i + 1)
(( i++ ))                       # bash
i=$(( i + 1 ))                  # POSIX

# Instead of: expr "$string" : '.*'
echo "${#string}"

seq for sequences

# Instead of: for i in $(seq 1 10)
for (( i = 1; i <= 10; i++ )); do
  echo "$i"
done

# Or for fixed ranges:
for i in {1..10}; do
  echo "$i"
done

tr for case conversion (bash 4+)

# Instead of: echo "$var" | tr '[:upper:]' '[:lower:]'
echo "${var,,}"

# Instead of: echo "$var" | tr '[:lower:]' '[:upper:]'
echo "${var^^}"

date (bash 4+)

# Instead of: date '+%Y-%m-%d'
printf '%(%Y-%m-%d)T\n' -1

# Instead of: date '+%s'
printf '%(%s)T\n' -1

cat for reading files

# Instead of: content=$(cat file)
content=$(<file)

# Instead of: cat file | grep pattern
grep pattern file
grep pattern < file

Command existence check

# Instead of: which command
command -v command >/dev/null 2>&1

# Or in a test:
if command -v git >/dev/null 2>&1; then
  echo "git is available"
fi

Arithmetic

# Use (( )) for arithmetic commands
(( count++ ))
(( total = price * quantity ))
if (( a > b )); then echo "a is greater"; fi

# Use $(( )) for arithmetic substitution
result=$(( x + y ))
echo "Total: $(( price * quantity ))"

# POSIX arithmetic (no (( )) command)
result=$(( x + y ))
[ "$a" -gt "$b" ] && echo "a is greater"

read Builtin

# Read a line
IFS= read -r line

# Read with prompt
read -rp "Enter name: " name

# Read with timeout (bash)
read -rt 5 -p "Quick! " answer

# Split on delimiter
IFS=: read -r user _ uid gid _ home shell <<< "$passwd_line"

# Read into array (bash 4+)
mapfile -t lines < file.txt
IFS=: read -ra fields <<< "$line"

# Read null-delimited (for find -print0)
while IFS= read -r -d '' file; do
  echo "$file"
done

Arrays (Bash)

# Declaration
declare -a arr=("one" "two" "three")
arr+=("four")

# Access
echo "${arr[0]}"             # first element
echo "${arr[@]}"             # all elements (separate words)
echo "${arr[*]}"             # all elements (single word)
echo "${#arr[@]}"            # array length
echo "${!arr[@]}"            # all indices

# Iteration
for item in "${arr[@]}"; do
  echo "$item"
done

# Slicing
echo "${arr[@]:1:2}"         # elements 1-2

# Deletion
unset 'arr[1]'               # remove element (leaves gap)

# Associative arrays (bash 4+)
declare -A map
map[key]="value"
echo "${map[key]}"
[[ -v map[key] ]] && echo "key exists"

Common Pitfalls

A catalog of frequent shell scripting mistakes, drawn from Greg Wooledge's BashPitfalls, ShellCheck warnings, and real-world bugs.

Iteration Pitfalls

Parsing ls output

# WRONG — breaks on filenames with spaces, globs, newlines
for f in $(ls *.mp3); do
  something "$f"
done

# RIGHT — use globs directly
for f in ./*.mp3; do
  [[ -e "$f" ]] || continue
  something "$f"
done

Word-split for loops on file content

# WRONG — splits on words, not lines; globs expand
for line in $(cat file); do
  echo "$line"
done

# RIGHT — use while read
while IFS= read -r line; do
  echo "$line"
done < file

find output with for

# WRONG — breaks on spaces, newlines in paths
for f in $(find . -type f); do ...

# RIGHT — use find with -exec or -print0
find . -type f -exec process {} \;

# RIGHT — bash: while read with null delimiter
while IFS= read -r -d '' f; do
  process "$f"
done < <(find . -type f -print0)

Variable Pitfalls

Unquoted variables

# WRONG — word splitting and globbing
cp $file $target
echo $message

# RIGHT
cp -- "$file" "$target"
echo "$message"

Spaces in assignments

var = "value"    # WRONG: runs "var" as command with "=" and "value" as args
var="value"      # RIGHT

local var=$(cmd) masks exit code

# WRONG — local always returns 0, hiding command failure
local result=$(failing_command)
echo $?  # always 0

# RIGHT — separate declaration and assignment
local result
result=$(failing_command)
echo $?  # actual exit code

Test Pitfalls

[ vs [[

# WRONG — [ doesn't handle empty variables
[ $var = "test" ]           # fails if $var is empty

# RIGHT — quote or use [[
[ "$var" = "test" ]         # POSIX way
[[ $var == "test" ]]        # Bash way (no word splitting)

&& / || inside [ ]

# WRONG — && is a command separator, not a test operator
[ "$a" = 1 && "$b" = 2 ]

# RIGHT
[ "$a" = 1 ] && [ "$b" = 2 ]
[[ $a == 1 && $b == 2 ]]

Numeric comparison with >

# WRONG — > is redirection in [ ], string comparison in [[ ]]
[ "$a" > "$b" ]             # creates file named "$b"
[[ $a > $b ]]               # lexicographic, not numeric

# RIGHT
[ "$a" -gt "$b" ]           # POSIX numeric comparison
(( a > b ))                 # Bash arithmetic

[[ $foo == $bar ]] is pattern matching

# This does pattern matching, not string comparison
bar="*.txt"
[[ $foo == $bar ]]  # matches if foo ends in .txt

# For string comparison, quote the right side
[[ $foo == "$bar" ]]

Pipeline Pitfalls

Variables in pipeline subshells

# WRONG — while runs in subshell, count is lost
count=0
cat file | while read -r line; do
  (( count++ ))
done
echo "$count"  # always 0

# RIGHT — use process substitution
count=0
while read -r line; do
  (( count++ ))
done < <(cat file)
echo "$count"  # correct value

Redirecting to the same file

# WRONG — file is truncated before sed reads it
sed 's/foo/bar/' file > file

# RIGHT — use temp file or sed -i
sed 's/foo/bar/' file > tmpfile && mv tmpfile file
sed -i 's/foo/bar/' file  # GNU sed

Arithmetic Pitfalls

(( i++ )) with set -e

set -e
i=0
(( i++ ))    # EXIT! post-increment returns 0 (old value), which is falsy

# Fix options:
(( ++i ))             # pre-increment returns 1
(( i += 1 ))          # addition returns non-zero
i=$(( i + 1 ))        # assignment, not standalone (( ))

Leading zeros in arithmetic

# Bash treats leading zeros as octal
(( 010 == 8 ))   # true! 010 octal = 8 decimal

# Strip leading zeros before arithmetic
n="08"
n=${n#0}  # or use 10#$n to force base-10
(( result = 10#$n + 1 ))

Miscellaneous Pitfalls

cmd1 && cmd2 || cmd3 is not if/then/else

# WRONG — cmd3 runs if cmd2 fails, even when cmd1 succeeded
true && false || echo "this runs unexpectedly"

# RIGHT — use proper if/then/else
if cmd1; then
  cmd2
else
  cmd3
fi

export var=~/path — tilde may not expand

# Tilde expansion in export is implementation-dependent
export dir=~/projects    # may or may not expand ~

# RIGHT
dir=~/projects
export dir
# Or:
export dir="$HOME/projects"

echo "$var" when var might start with -

# echo might interpret -n, -e, etc. as options
var="-n hello"
echo "$var"        # might print "hello" without newline

# RIGHT
printf '%s\n' "$var"

Filenames starting with -

# WRONG — filename "-rf" could be interpreted as option
rm $file

# RIGHT — use -- to end option processing
rm -- "$file"

# Or prefix with ./
rm "./$file"

Forgetting -- with commands

Many commands interpret arguments starting with - as options. Always use -- before variable filenames:

cp -- "$src" "$dst"
mv -- "$old" "$new"
grep -- "$pattern" "$file"
sort -- "$file"

Portability

Shell scripts may need to run across different shells (bash, dash, sh) and different operating systems (Linux, macOS, BSDs). This reference covers when to target POSIX sh vs bash, and what constructs are safe in each.

When to Use What

• POSIX sh (#!/bin/sh) — System scripts, init scripts, maximum portability
• Bash (#!/usr/bin/env bash) — User scripts, CI/CD, anywhere bash is guaranteed
• Bash 4+ (#!/usr/bin/env bash) — Associative arrays, mapfile, ${var,,} case ops

macOS note: macOS ships bash 3.2 permanently (GPL licensing). If targeting macOS, avoid bash 4+ features or require users to install modern bash via Homebrew.

POSIX sh: What You Have

Available

• Parameter expansion: ${var}, ${var:-default}, ${var#pattern}, ${var%pattern}, ${var##pattern},

${var%%pattern}, ${#var}

• Command substitution: $(command) (preferred), ` command ` (legacy)
• Arithmetic: $(( expr )) (no (( )) command form)
• Tests: [ condition ] (test builtin)
• Control flow: if/elif/else/fi, case/esac, for/while/until
• Functions: name() { ...; } (no function keyword)
• Traps: trap 'commands' SIGNAL
• Here documents: <<TAG, <<'TAG' (no quoting), <<-TAG (strip tabs)
• Redirections: >, >>, <, 2>&1, >&2
• Special variables: $?, $#, $@, $*, $$, $!, $0, $-

Not Available in POSIX sh

• `[[ ]]` (Extended test) — POSIX workaround: use [ ] with proper quoting
• `(( ))` (Arithmetic command) — POSIX workaround: [ "$a" -gt "$b" ] or $(( ))
• Arrays (arr=(a b c)) — POSIX workaround: use positional parameters or IFS splitting
• `local` (Function locals) — Technically a widely-supported extension; not POSIX
• `${var,,}` (Lowercase) — POSIX workaround: echo "$var" | tr '[:upper:]' '[:lower:]'
• `${var^^}` (Uppercase) — POSIX workaround: echo "$var" | tr '[:lower:]' '[:upper:]'
• `${var/p/r}` (Substitution) — POSIX workaround: use sed or parameter expansion tricks
• `=~` regex (Regex match) — POSIX workaround: use expr or case with glob patterns
• `<<<` (Here-string) — POSIX workaround: use echo "$var" | command or here-doc
• `<(cmd)` (Process sub) — POSIX workaround: use temp files or pipes
• `source` (Source files) — POSIX workaround: use . (dot command)
• `{1..10}` (Brace expansion) — POSIX workaround: use seq or arithmetic loop
• `$RANDOM` (Random number) — POSIX workaround: read from /dev/urandom

Bash-Specific Features Worth Using

When you target bash, these features are significant improvements over POSIX:

[[ ]] — Extended Test

• No word splitting or globbing on variable expansions
• Supports &&/|| inside the test
• Pattern matching: [[ $var == *.txt ]]
• Regex matching: [[ $var =~ ^[0-9]+$ ]]
• No need for x"$var" defensive prefix

Arrays

# Indexed arrays (bash 2+)
files=("file one.txt" "file two.txt" "file three.txt")
for f in "${files[@]}"; do
  echo "$f"
done

# Associative arrays (bash 4+)
declare -A config
config[host]="localhost"
config[port]="8080"

Process Substitution

# Compare two command outputs
diff <(sort file1) <(sort file2)

# Read from a command without subshell (preserves variables)
while read -r line; do
  count=$((count + 1))
done < <(grep -c pattern file)
echo "$count"  # variable persists

mapfile / readarray (bash 4+)

mapfile -t lines < file.txt
echo "${#lines[@]} lines"
echo "${lines[0]}"

Portable Patterns

String Comparison

# POSIX
[ "$var" = "value" ]           # string equality (single =)
[ "$var" != "value" ]          # string inequality

# Bash
[[ $var == "value" ]]          # string equality
[[ $var == *.txt ]]            # glob pattern
[[ $var =~ ^[0-9]+$ ]]        # regex

Numeric Comparison

# POSIX (always use these operators in [ ])
[ "$a" -eq "$b" ]    # equal
[ "$a" -ne "$b" ]    # not equal
[ "$a" -lt "$b" ]    # less than
[ "$a" -gt "$b" ]    # greater than
[ "$a" -le "$b" ]    # less or equal
[ "$a" -ge "$b" ]    # greater or equal

# Bash arithmetic
(( a > b ))           # cleaner, but not POSIX

Substring Check

# POSIX (case statement for pattern matching)
case "$string" in
  *substring*) echo "found" ;;
  *) echo "not found" ;;
esac

# Bash
[[ $string == *substring* ]]

Reading Input Line by Line

# POSIX
while IFS= read -r line; do
  printf '%s\n' "$line"
done < file

# Bash (also works in POSIX, but || handles missing final newline)
while IFS= read -r line || [ -n "$line" ]; do
  printf '%s\n' "$line"
done < file

Default Values

# POSIX and bash — identical syntax
: "${VAR:=default}"            # set default if unset/empty
echo "${VAR:-fallback}"        # use fallback without modifying VAR

GNU vs BSD Tool Differences

• In-place sed — GNU: sed -i 's/a/b/' file / BSD: sed -i '' 's/a/b/' file (GNU form not compatible on macOS)
• Extended regex — GNU: grep -P / BSD: not available; use grep -E
• `date` format — GNU: date -d '2024-01-01' / BSD: date -j -f '%Y-%m-%d' '2024-01-01'
• `readlink -f` — GNU: canonical path / BSD: not available; use realpath
• `stat` format — GNU: stat -c '%s' file / BSD: stat -f '%z' file
• xargs null-delim — GNU: xargs -0 -r / BSD: xargs -0 (no -r on macOS)

Guideline: When possible, avoid GNU-specific extensions. When they're needed, document the dependency.

Quoting

Quoting is the #1 source of shell scripting bugs. Unquoted expansions undergo word splitting and globbing — two implicit transformations that silently break filenames with spaces, glob characters, or special characters.

The Three Quoting Mechanisms

Single Quotes

Preserve every character literally. No exceptions. No expansions.

echo 'Price is $5.00'      # Price is $5.00
echo 'Backslash: \'         # error: cannot include single quote
echo 'He said "hello"'     # He said "hello"

A single quote cannot appear inside single quotes. Use $'...' or concatenation:

echo 'can'\''t'            # can't (end quote, escaped quote, start quote)
echo $'can\'t'             # can't (ANSI-C quoting, bash extension)

Double Quotes

Preserve most characters literally, but allow these expansions:

• $variable and ${variable} — parameter expansion
• $(command) — command substitution
• $(( expr )) — arithmetic expansion
• \$, \", \\, ` \ `, \newline` — backslash escaping

name="World"
echo "Hello $name"          # Hello World
echo "Path is $(pwd)"       # Path is /home/user
echo "Two plus two is $((2+2))"  # Two plus two is 4

ANSI-C Quoting ($'...')

Bash extension. Interprets backslash escapes like C:

$'\n'     # newline
$'\t'     # tab
$'\\'     # literal backslash
$'\''     # literal single quote
$'\x41'   # hex: A
$'\u0041' # unicode: A (bash 4.4+)

Word Splitting

When a variable or command substitution is unquoted, the shell splits the result into words using characters in $IFS (default: space, tab, newline).

file="my document.txt"
cat $file         # WRONG: runs cat with two args: "my" and "document.txt"
cat "$file"       # RIGHT: runs cat with one arg: "my document.txt"

output=$(ls)
echo $output      # WRONG: all whitespace collapsed to single spaces
echo "$output"    # RIGHT: preserves original formatting

Globbing (Pathname Expansion)

When a variable or command substitution is unquoted, the shell also expands glob characters (*, ?, [...]).

msg="Files: *.txt"
echo $msg         # WRONG: *.txt expands to matching filenames
echo "$msg"       # RIGHT: prints literal "Files: *.txt"

Quoting Rules

Always Quote

1. Variable expansions: "$var", "${var}", "${var:-default}" 2. Command substitutions: "$(command)" 3. Array element access: "${array[0]}" 4. Strings with spaces or special chars: "hello world"

Safe to Leave Unquoted

1. `[[ ]]` left-hand side: [[ $var == pattern ]] (no word splitting inside [[ ]]) 2. Assignments: var=$other_var (right side of assignment is not split) 3. `(( ))` arithmetic: (( x + y )) (arithmetic context, no splitting) 4. Integer special variables: $?, $#, $$, $! (guaranteed no spaces, but quoting won't hurt) 5. Case patterns: case $var in ... (no splitting in case word)

"$@" vs "$*"

This is the most critical quoting distinction for argument passing:

# "$@" preserves each argument as a separate word
# "$*" joins all arguments into a single word (separated by first IFS char)

set -- "arg one" "arg two" "arg three"

for x in "$@"; do echo "[$x]"; done
# [arg one]
# [arg two]
# [arg three]

for x in "$*"; do echo "[$x]"; done
# [arg one arg two arg three]

# NEVER use $* or $@ without quotes — word splitting breaks arguments
for x in $@; do echo "[$x]"; done
# [arg]
# [one]
# [arg]
# [two]
# [arg]
# [three]

Rule: Use "$@" to pass arguments through. Use "$*" only when you deliberately want to join arguments into a single string.

Array Expansion

arr=("one two" "three" "four five")

# RIGHT: each element as separate word
for item in "${arr[@]}"; do echo "$item"; done
# one two
# three
# four five

# WRONG: word splitting destroys element boundaries
for item in ${arr[@]}; do echo "$item"; done
# one
# two
# three
# four
# five

Common Quoting Mistakes

Quoting too much

# WRONG: quoted glob won't expand
for f in "*.txt"; do ...    # iterates once with literal "*.txt"

# RIGHT: glob must be unquoted
for f in *.txt; do ...      # iterates over matching files

# WRONG: quoted tilde won't expand
cd "~/Documents"            # looks for literal "~/Documents"

# RIGHT: tilde must be unquoted, or use $HOME
cd ~/Documents
cd "$HOME/Documents"

Quoting inside [[ ]]

# Pattern matching: RHS must be unquoted for glob/regex
[[ $file == *.txt ]]        # RIGHT: glob pattern
[[ $file == "*.txt" ]]      # matches literal "*.txt"

# Regex: RHS must be unquoted
re='^[0-9]+$'
[[ $var =~ $re ]]           # RIGHT: regex match
[[ $var =~ "$re" ]]         # string comparison, not regex

Nested quoting in command substitution

Quotes inside $(...) are independent from outer quotes:

# This is correct — inner quotes are separate
result="$(command "$(inner_command "$arg")")"

printf vs echo

Prefer printf for reliable output:

# echo behavior varies across platforms and arguments
echo -n "hello"      # some systems print "-n hello"
echo "$var"          # if var is "-n", echo interprets it as option

# printf is consistent and safe
printf '%s' "hello"         # no trailing newline
printf '%s\n' "$var"        # always treats $var as data, never as option
printf '%s\n' "${arr[@]}"   # print each array element on its own line

Strict Mode and Error Handling

Defensive shell scripting starts with strict mode. These settings change shell behavior from "silently continue on failure" to "stop and report."

The Strict Mode Header

#!/usr/bin/env bash
set -euo pipefail

set -e (errexit)

Aborts the script when a command exits with non-zero status. Exceptions:

• Commands in if/while/until conditions
• Commands before && or ||
• Commands in ! (negation)
• Commands in subshells that are part of conditions

Caveats with `errexit`:

# This will NOT trigger errexit because the failing command
# is on the left side of ||
false || true

# This WILL exit — standalone failing command
false

# Careful: (( )) with value 0 returns exit status 1
set -e
i=0
(( i++ ))  # exits! i was 0, post-increment returns 0 (falsy in C)
# Fix: use (( ++i )) or (( i += 1 )) or : $(( i++ ))

Functions and `errexit`:

# errexit does NOT propagate into functions called from conditions
check_something() {
  false       # this won't abort when called from if
  echo "still running"
}
if check_something; then
  echo "ok"
fi

set -u (nounset)

Treats unset variables as errors during expansion. This catches typos in variable names and missing initializations.

set -u
echo "$undefined_var"  # error: undefined_var: unbound variable

# Safe patterns for optional variables:
echo "${OPTIONAL_VAR:-default_value}"  # use default if unset
echo "${OPTIONAL_VAR:+value_if_set}"   # use alternate if set

set -o pipefail

By default, a pipeline's exit status is the exit status of the last command. With pipefail, a pipeline returns the exit status of the rightmost command that failed (or 0 if all succeeded).

set -o pipefail

# Without pipefail: exit status is 0 (from wc)
# With pipefail: exit status is 1 (from grep)
grep "nonexistent" file.txt | wc -l

Use PIPESTATUS (bash) to inspect individual command exit codes:

cmd1 | cmd2 | cmd3
echo "${PIPESTATUS[0]} ${PIPESTATUS[1]} ${PIPESTATUS[2]}"

Exit Codes

• 0 — Success
• 1 — General error
• 2 — Misuse of shell builtin
• 126 — Command not executable
• 127 — Command not found
• 128+N — Killed by signal N

Return meaningful exit codes from scripts:

readonly EX_OK=0
readonly EX_USAGE=64
readonly EX_DATAERR=65
readonly EX_NOINPUT=66
readonly EX_SOFTWARE=70

die() {
  printf '%s\n' "$1" >&2
  exit "${2:-1}"
}

Trap-Based Cleanup

Use trap to run cleanup code on script exit, regardless of how the script terminates:

cleanup() {
  local exit_code=$?
  rm -f "$tmpfile"
  exit "$exit_code"
}
trap cleanup EXIT

tmpfile=$(mktemp)
# ... script body ...
# cleanup runs automatically on any exit

Trap signals:

• `EXIT` — Script exits (any reason)
• `ERR` — Command fails (with set -e)
• `INT` — Ctrl+C
• `TERM` — kill (default signal)
• `HUP` — Terminal hangup

Trap rules:

# Multiple signals in one trap
trap cleanup EXIT INT TERM

# Reset a trap
trap - EXIT

# Show current traps
trap -p

Temporary Files

Always use mktemp for temporary files and clean up with traps:

tmpfile=$(mktemp) || die "Failed to create temp file"
tmpdir=$(mktemp -d) || die "Failed to create temp dir"
trap 'rm -rf "$tmpfile" "$tmpdir"' EXIT

Never use predictable filenames in /tmp — this creates race conditions and symlink attacks.

Subshell Isolation for cd

# Wrong: cd failure leaves you in wrong directory
cd /some/path
rm important_file

# Right: check cd, or use subshell
cd /some/path || exit 1
rm important_file

# Better: subshell isolates directory change
(
  cd /some/path || exit 1
  rm important_file
)

Debugging

# Trace execution (prints each command before running)
set -x

# Trace only a specific section
set -x
# ... debug this section ...
set +x

# Custom trace prefix showing file and line
PS4='+${BASH_SOURCE}:${LINENO}: '
set -x

# Syntax check without executing
bash -n script.sh

# Verbose mode (prints lines as read)
set -v