Gdb

GDB

Purpose

Walk agents through GDB sessions from first launch to advanced workflows: crash diagnosis, reverse debugging, remote debugging, and multi-thread inspection.

Triggers

• "My program segfaults / crashes — how do I debug it?"
• "How do I set a breakpoint on condition X?"
• "How do I inspect memory / variables in GDB?"
• "How do I debug a remote embedded target?"
• "GDB shows ?? frames / no source"
• "How do I replay a bug deterministically?" (record/replay)

Workflow

1. Prerequisite: compile with debug info

Always compile with -g (GCC/Clang). Use -Og or -O0 for most debuggable code.

gcc -g -Og -o prog main.c

For release builds: use -g -O2 and keep the binary with symbols (strip separately with objcopy).

2. Start GDB

gdb ./prog                          # load binary
gdb ./prog core                     # load with core dump
gdb -p 12345                        # attach to running process
gdb --args ./prog arg1 arg2         # pass arguments
gdb -batch -ex 'run' -ex 'bt' ./prog  # non-interactive (CI)

3. Essential commands

4. Breakpoints and watchpoints

break main                          # break at function
break file.c:42                     # break at line
break *0x400abc                     # break at address
break foo if x > 10                 # conditional break
tbreak foo                          # temporary breakpoint (fires once)
rbreak ^mylib_.*                    # regex breakpoint on all matching functions

watch x                             # watchpoint: break when x changes
watch *(int*)0x601060               # watch memory address
rwatch x                            # break when x is read
awatch x                            # break on read or write

info breakpoints                    # list all breakpoints
delete 3                            # delete breakpoint 3
disable 3                           # disable without deleting
enable 3

5. Inspect state

print x                             # print variable
print/x x                           # print in hex
print *ptr                          # dereference pointer
print arr[0]@10                     # print 10 elements of array
display x                           # auto-print x on every stop
undisplay 1

info locals                         # all local variables
info args                           # function arguments
info registers                      # all CPU registers
info registers rip rsp rbp          # specific registers
x/10wx 0x7fff0000                   # examine 10 words at address
x/s 0x400abc                        # examine as string
x/i $rip                            # examine current instruction

backtrace                           # call stack (bt)
bt full                             # bt + local vars
frame 2                             # switch to frame 2
up / down                           # move up/down the stack

6. Multi-thread debugging

info threads                        # list threads
thread 3                            # switch to thread 3
thread apply all bt                 # backtrace all threads
thread apply all bt full            # full bt all threads
set scheduler-locking on            # pause other threads while stepping

7. Reverse debugging (record/replay)

Record requires target record-full or target record-btrace (Intel PT):

# Software record (slow but universal)
record                              # start recording
run
# ... trigger the bug ...
reverse-continue                    # go back to last break
reverse-next                        # step backwards
reverse-step
reverse-finish

# Intel Processor Trace (fast, hardware)
target record-btrace pt
run
# view instruction history
record instruction-history

8. Remote debugging with gdbserver

On target:

gdbserver :1234 ./prog
# Or attach:
gdbserver :1234 --attach 5678

On host:

gdb ./prog
(gdb) target remote 192.168.1.10:1234
(gdb) break main
(gdb) continue

For cross-compilation: use aarch64-linux-gnu-gdb on the host.

9. Common problems

10. GDB init file (~/.gdbinit)

set history save on
set history size 1000
set print pretty on
set print array on
set print array-indexes on
set pagination off
set confirm off

For a command cheatsheet, see references/cheatsheet.md. For pretty-printers and Python scripting, see references/scripting.md.

Related skills

• Use skills/debuggers/core-dumps for loading core files
• Use skills/debuggers/lldb for LLDB-based workflows
• Use skills/runtimes/sanitizers to catch bugs before needing the debugger
• Use skills/compilers/gcc for -g flag details

GDB Command Cheatsheet

Source: <https://sourceware.org/gdb/documentation/> Source: <https://aaronbloomfield.github.io/pdr/docs/gdb_vs_lldb.html>

Table of Contents

1. Startup 2. Execution control 3. Breakpoints & watchpoints 4. Inspection 5. Stack 6. Memory 7. Threads 8. Reverse debugging 9. Display formats

---

Startup

gdb prog                         Load binary
gdb prog core                    Load with core
gdb -p PID                       Attach to process
gdb --args prog a b c            Pass arguments
gdb -batch -ex 'run' -ex 'bt'   Non-interactive
set args a b c                   Set args after load

---

Execution control

run / r [args]                   Start
continue / c                     Resume
next / n                         Step over (source)
step / s                         Step into (source)
nexti / ni                       Step over (instruction)
stepi / si                       Step into (instruction)
finish                           Run to end of function
until N                          Run to line N in current file
until file.c:N                   Run to specific line
advance foo                      Run to next call of foo
return [expr]                    Force return value
signal SIGUSR1                   Deliver signal
kill                             Kill program

---

Breakpoints & watchpoints

break main                       Function
break file.c:42                  Line
break *0x400abc                  Address
break foo if x > 0               Conditional
tbreak foo                       Temporary (once)
rbreak regex                     All matching functions
catch throw                      C++ exception thrown
catch syscall mmap               Syscall breakpoint

watch var                        Write watchpoint
rwatch var                       Read watchpoint
awatch var                       Read/write watchpoint
watch *(int*)addr                Memory watchpoint

info breakpoints / info watch    List
delete N                         Remove breakpoint N
disable N / enable N             Toggle
ignore N count                   Skip N times
commands N                       Run commands on hit

---

Inspection

print expr / p expr              Print expression
print/x expr                     Print hex
print/t expr                     Print binary
print/f expr                     Print float
print/c expr                     Print char
print/d expr                     Print signed int
print/u expr                     Print unsigned int
print/a expr                     Print as address

print arr@N                      Print N elements
print *ptr                       Dereference
print *((int*)ptr)               Cast and deref

display expr                     Print on every stop
undisplay N                      Remove auto-display
info display                     List

ptype var                        Print type of var
whatis var                       Brief type info
info locals                      All locals
info args                        Function arguments
info variables                   All global variables

---

Stack

backtrace / bt                   Call stack
bt N                             Top N frames
bt full                          Frames + locals
bt -N                            Bottom N frames
frame N / f N                    Select frame N
up / down                        Move in stack
info frame                       Frame info
info args                        Current frame args
info locals                      Current frame locals

---

Memory

x/Nuf addr                       Examine memory
  N = count
  u = unit: b(byte) h(half=2) w(word=4) g(giant=8)
  f = format: x(hex) d(dec) u(uint) o(octal) t(bin) f(float) s(string) i(insn)

Examples:
x/10wx 0x7fff0000                10 words (hex)
x/4gx $rsp                       4 giants at stack top
x/20i $rip                       20 instructions from RIP
x/s 0x400abc                     String
x/b &var                         Single byte

set {int}0xaddr = 42             Write memory
set var = 42                     Set variable

---

Threads

info threads                     List threads
thread N                         Switch to thread N
thread apply all bt              Backtrace all
thread apply all bt full
thread apply 1 2 print x         Apply to specific threads
set scheduler-locking on/off     Lock/unlock other threads during step

---

Reverse debugging

record                           Start software record
record btrace                    Start hardware trace
record stop                      Stop recording
reverse-continue / rc            Go backward to last event
reverse-next / rn                Reverse step-over
reverse-step / rs                Reverse step-into
reverse-finish                   Reverse finish
set exec-direction reverse       Make n/s go backward
set exec-direction forward       Restore
record instruction-history       Show recorded instructions (btrace)
record function-call-history     Show call history (btrace)

---

Display formats

GDB Python Scripting and Pretty-Printers

Source: <https://sourceware.org/gdb/current/onlinedocs/gdb.html/Python.html>

GDB Python API basics

GDB embeds a Python interpreter. Scripts can be loaded with source script.py or placed in ~/.gdb_scripts/.

import gdb

# Execute GDB command
gdb.execute('bt full')

# Evaluate expression
val = gdb.parse_and_eval('x')
print(val)

# Read inferior memory
inf = gdb.selected_inferior()
mem = inf.read_memory(0x601060, 16)

# List threads
for thr in gdb.selected_inferior().threads():
    print(thr.num, thr.name)

Custom pretty-printer

import gdb
import gdb.printing

class MyVectorPrinter:
    def __init__(self, val):
        self.val = val

    def to_string(self):
        data = self.val['_data']
        size = int(self.val['_size'])
        return f'MyVector of {size} elements'

    def children(self):
        data = self.val['_data']
        size = int(self.val['_size'])
        for i in range(size):
            yield f'[{i}]', (data + i).dereference()

    def display_hint(self):
        return 'array'

def build_printer(val):
    t = val.type.strip_typedefs()
    if t.name == 'MyVector':
        return MyVectorPrinter(val)
    return None

gdb.pretty_printers.append(build_printer)

Load in GDB:

source /path/to/printer.py

STL pretty-printers (libstdc++)

# Install python3-gdb package (Debian/Ubuntu)
sudo apt install python3-gdb

# Or use the GCC source tree printers
# They load automatically if libstdc++ was built with GDB support

Breakpoint commands

break foo
commands
  silent
  print "hit foo, x =", x
  continue
end

Convenience variables

set $i = 0
while $i < 10
  print arr[$i]
  set $i = $i + 1
end

Useful define aliases

define pbt
  thread apply all bt full
end

define hex
  print/x $arg0
end

Place in ~/.gdbinit.