Nib

Nib — Visual Feedback

You have eyes. Use them. Any time something visual is relevant, capture the screen and look.

When to Use This

Use nib aggressively. If any of these apply, capture and look:

• You just made a UI change → capture and verify it looks right
• You're debugging something visual → capture and inspect
• You need to know where something is on screen → capture and find it
• You're unsure about layout, spacing, colors, alignment → capture and check
• The user mentions anything visual → capture immediately
• You want to show the user what you did → capture and annotate
• You're about to ask "does this look right?" → capture instead of asking
• You need coordinates for clicking or positioning → capture and use grid

Don't ask what to capture. Don't ask permission. Just capture and look.

The Loop

1. Find the right window → nib windows --json
2. Capture it → nib capture --app "AppName" -o /tmp/nib_shot.png
3. Look at the screenshot (read the file)
4. Decide what you need feedback on
5. nib feedback /tmp/nib_shot.png -a '[annotations]' -m "question"
6. Parse the human's response
7. Act on it

Step 1: Find the Right Window

Don't just screenshot everything. Find the window you actually care about:

# List all windows with metadata
nib windows --json

# Filter by app name
nib windows --app "Safari" --json

Output gives you app name, title, size, position, and focus state for every window.

Step 2: Capture the Right Thing

# Capture a specific app's window
nib capture --app "Safari" -o /tmp/nib_shot.png

# Capture by window title
nib capture --title "index.html" -o /tmp/nib_shot.png

# Capture the focused window
nib capture --mode window -o /tmp/nib_shot.png

# Capture full screen (fallback)
nib capture -o /tmp/nib_shot.png

Always prefer `--app` or `--title` over full screen. You get just the window you need, no distractions.

Step 3: Look

Read the screenshot file to see what's on screen. Use grid or OCR if you need precision:

# Coordinate grid for positioning
nib grid /tmp/nib_shot.png --spacing 100 -o /tmp/nib_grid.png

# Find text via OCR
nib find-text /tmp/nib_shot.png -s "Submit"

Step 3: Annotate and Ask

Point at the things you want feedback on. Be specific.

nib feedback /tmp/nib_shot.png \
  -a '[{"type":"arrow","from":[300,200],"to":[450,350]},{"type":"text","at":[300,180],"content":"This spacing looks off?"}]' \
  -m "Does the spacing between these elements look right?"

Step 4: Parse Response

Human draws annotations and hits Cmd+Enter. You get back only their annotations:

{"annotations": [{"id": "a1", "type": "arrow", "at": [150, 200, 300, 100], "owner": "human"}]}

Timeout returns {"event": "timeout"} (exit code 0, not an error).

Step 5: Act

Use the human's visual feedback to take action. Then capture again to verify.

Annotation Format

[
  {"type": "arrow", "from": [x, y], "to": [x, y]},
  {"type": "rectangle", "at": [x, y], "size": [w, h]},
  {"type": "text", "at": [x, y], "content": "Label"},
  {"type": "highlight", "at": [x, y], "size": [w, h], "color": "#ffff0080"},
  {"type": "number", "at": [x, y], "value": 1},
  {"type": "ellipse", "center": [x, y], "radius": [rx, ry]},
  {"type": "line", "from": [x, y], "to": [x, y]},
  {"type": "blur", "at": [x, y], "size": [w, h]}
]

All types accept optional "color" (hex). Use blue #3b82f6 for your annotations.

Headless Mode

When you don't need human feedback, just annotate and render:

nib annotation add image.png -t rectangle -x 100 -y 100 -w 200 -H 50 -c "#ff0000"
nib render image.png  # → image.rendered.png

Key Flags

See full command reference for all commands and options.

Nib Command Reference

Complete reference for all nib CLI commands and options.

CLI Global Flags

Commands

nib capture

Capture a screenshot.

nib capture -o output.png
nib capture --mode region -o region.png
nib capture --mode window -o window.png

nib feedback

Primary tool for visual collaboration. Opens GUI, shows annotations, waits for human response.

nib feedback image.png \
  -a '[{"type":"arrow","from":[100,100],"to":[200,150]}]' \
  -m "Where should the button go?" \
  -t 120

Owner-based filtering: Annotations passed via -a are tagged owner: "claude" and excluded from the response. Only the human's new annotations are returned.

Response format:

{"annotations": [{"id": "a1", "type": "rectangle", "at": [150, 200, 300, 100], "owner": "human"}]}

Timeout response:

{"event": "timeout"}

Exit code 0 on timeout (not an error). GUI stays open.

nib gui

Launch the GUI editor directly.

nib gui image.png

nib annotation add

Add an annotation headlessly (no GUI needed).

nib annotation add image.png -t rectangle -x 100 -y 100 -w 200 -H 50 -c "#ff0000"

Output: Added annotation [a1] rectangle at (100, 100)

nib annotation remove

nib annotation remove image.png a1

nib annotation clear

nib annotation clear image.png

nib annotation list

nib annotation list image.png --json

nib render

Bake annotations onto the image.

nib render image.png
# Output: image.rendered.png

nib render image.png -o custom_output.png

nib grid

Overlay a coordinate grid for precision positioning.

# Visual overlay
nib grid image.png --spacing 100 -o grid.png

# Zoom into region (x1,y1,x2,y2)
nib grid image.png --region "300,150,500,300" --spacing 10 -o zoomed.png

# JSON metadata
nib grid image.png --spacing 100 --json

Grid labels use base36 encoding (0-9, A-Z). Convert label to pixels: pixel_x = col * spacing, pixel_y = row * spacing.

nib find-text

OCR text search and highlighting.

# Find text
nib find-text image.png -s "Submit"

# Highlight matches
nib find-text image.png -s "Error" --highlight --color "#ff0000"

# Search within region (x,y,width,height)
nib find-text image.png -r "100,200,400,300" -s "Button"

nib import

Convert image to .nib format.

nib import image.png

nib export

Export .nib file.

nib export image.nib --export-format rendered
nib export image.nib --export-format json
nib export image.nib --export-format qml

nib validate

Check QML syntax.

nib validate image.png

nib info

Show .nib file metadata.

nib info image.nib --json

nib list

List recent captures.

nib list

nib tile

Manage tiled captures for large images.

# Query a point
nib tile query capture.tiles/ --point "500,300"

# Extract a region at full resolution
nib tile extract capture.tiles/ -r "100,100,400,300" -o region.png

# List tiles
nib tile list capture.tiles/

Annotation JSON Format (Full)

All annotation types with every field:

[
  {
    "type": "arrow",
    "from": [x, y],
    "to": [x, y],
    "color": "#ff0000"
  },
  {
    "type": "rectangle",
    "at": [x, y],
    "size": [w, h],
    "color": "#00ff00"
  },
  {
    "type": "text",
    "at": [x, y],
    "content": "Label text",
    "color": "#0000ff"
  },
  {
    "type": "highlight",
    "at": [x, y],
    "size": [w, h],
    "color": "#ffff0080"
  },
  {
    "type": "number",
    "at": [x, y],
    "value": 1,
    "color": "#3b82f6"
  },
  {
    "type": "ellipse",
    "center": [x, y],
    "radius": [rx, ry],
    "color": "#dc2626"
  },
  {
    "type": "line",
    "from": [x, y],
    "to": [x, y],
    "color": "#ffffff"
  },
  {
    "type": "blur",
    "at": [x, y],
    "size": [w, h]
  }
]

All types accept an optional "color" field as a hex string. Highlight colors typically include alpha (e.g. "#ffff0080").

Recipes

OCR Highlight All Text

IMG=/tmp/shot.png && \
  nib capture -o $IMG && \
  nib annotation clear $IMG 2>/dev/null; \
  nib find-text $IMG --highlight --color "#ffff0080" && \
  nib render $IMG -o ${IMG%.png}_highlighted.png

Capture, Annotate, Render Pipeline

nib capture -o shot.png
nib annotation add shot.png -t rectangle -x 100 -y 100 -w 50 -H 30 -c "#ff0000"
nib render shot.png
# Inspect shot.rendered.png

Zoom and Inspect Region

nib grid image.png --region "1900,650,2300,850" --spacing 10 -o zoomed.png
# Inspect zoomed.png for precise coordinates

Multi-Turn Visual Dialogue

# Round 1
RESP=$(nib feedback mockup.png \
  -a '[{"type":"text","at":[50,30],"content":"HEADER"}]' \
  -m "Where should the navigation go?" -t 120)

# Parse response, use coordinates for round 2
nib feedback mockup.png \
  -a '[{"type":"rectangle","at":[50,80],"size":[200,40],"color":"#22c55e"}]' \
  -m "Got it. Where should the sidebar be?" -t 120