Image Generator Sd Webui

Image Generator (sd-webui API)

Overview

Drive a Stable Diffusion WebUI / Forge server through its REST API to enumerate available resources, run txt2img, poll progress, and interrupt jobs. All scripts under scripts/ are thin curl wrappers; they print JSON or extracted fields to stdout so the agent can pipe / parse them.

Server connection

Before doing anything, confirm the server URL (and optional HTTP Basic Auth) with the user. Pass them as environment variables to every script:

export SD_WEBUI_URL="http://localhost:7860"   # required, no trailing slash
export SD_WEBUI_USER=""                       # optional, HTTP Basic Auth
export SD_WEBUI_PASS=""                       # optional

If unset, scripts default to http://localhost:7860 with no auth.

Quick connectivity test (returns OK <url> on success, exits non-zero on failure):

scripts/probe.sh

Workflow

1. Probe — Verify the server is reachable (scripts/probe.sh). On failure, ask the user for the correct URL / credentials. 2. Enumerate & choose — List the resources to pick (models, modules, samplers, schedulers, styles) and ask the user to choose. Capture their choice verbatim in the API's English name / title / model_name — sd-webui matches exactly, do not translate or rename. 3. Prompt — Obtain the positive prompt, negative prompt, and any extra params (steps, CFG, size). See "Prompt engineering" for sourcing these. 4. Generate — Call scripts/generate.sh with a request JSON. It returns a JSON object containing the base64 PNG image and the generation info. 5. (Optional) Track progress — While generation is running (in another shell / background), call scripts/progress.sh to print progress (0–1), eta_relative, and state. 6. (Optional) Cancel — Call scripts/cancel.sh to interrupt the current job.

Tasks

Listing available resources

scripts/list.sh <kind> prints the canonical English identifier for each entry, one per line — pipe to column, fzf, etc. Add --json for the raw JSON.

After listing, present the options to the user (use ask_user with an enum if the list is short). For models, prefer the full title (which embeds the hash suffix, e.g. anima/animaika_v36.safetensors [d50fb5b9a0]) over model_name because the title is unambiguous — if the user supplies a bare filename without the hash, verify it via list.sh models and substitute the exact title before sending it to the API. For schedulers, list.sh schedulers prints the human-readable label (e.g. Beta); both label and the lowercase name (beta) are accepted by the txt2img scheduler field.

Generating an image (txt2img)

1. Build a JSON request. Required field: prompt. Recommended: negative_prompt, steps, cfg_scale, width, height, sampler_name, scheduler, styles (array of style names), and override_settings.sd_model_checkpoint (model title) / override_settings.forge_additional_modules (array of module names, Forge only). See references/txt2img-parameters.md for every field. 2. Run:

   scripts/generate.sh request.json > result.json
   # or pipe:
   cat request.json | scripts/generate.sh - > result.json

3. Extract the image (base64 PNG):

   jq -r '.images[0]' result.json | base64 -d > out.png

4. The info field is a JSON string with seed, all_prompts, sampler_name, etc. — parse with jq -r '.info | fromjson'.

Important behaviour notes:

• `samples_format` pre-pin: sd-webui/Forge validates samples_format before applying override_settings, so if the server's persistent value is unsupported (e.g. avif), txt2img fails. generate.sh preemptively POSTs samples_format=png to /sdapi/v1/options and redundantly injects override_settings.samples_format=png. ⚠️ The pre-pin mutates the server's persistent default to "png" — override_settings_restore_afterwards cannot undo it. If the user shares the server with clients expecting a different default, restore manually after: scripts/options.sh set samples_format '"webp"'. Convert locally if you need non-PNG output (see "Converting to another format" below).
• override_settings_restore_afterwards: true is forced on by generate.sh so the other override_settings keys (model checkpoint, modules, VAE) do not stick.
• Generation is synchronous — the POST blocks until the image is ready. The script uses a 600s curl timeout; override with SD_WEBUI_TIMEOUT=900 scripts/generate.sh ....

Converting to another format

If the user wants the output in a non-PNG format (WebP, AVIF, JPEG, etc.), do not try to re-enable a different samples_format on the server. Instead, convert locally while preserving the embedded sd-webui generation metadata:

1. Check whether both format-converter.sh and copy-info.sh are available on PATH (e.g. command -v format-converter.sh && command -v copy-info.sh). 2. If both are present, run format-converter.sh on the PNG — it calls copy-info.sh internally to carry the parameters over. Run format-converter.sh -h to see the current usage. 3. If either is missing, guide the user to install the helper project once: <https://github.com/jim60105/sd-image-format-converter>. It has system dependencies that must be set up manually, so it can't be auto-installed. After install, both scripts should be on PATH and -h will show usage.

Tracking progress

Call from another terminal (or background the generate.sh call with & first):

scripts/progress.sh                  # one-shot, prints JSON
scripts/progress.sh --watch          # poll every 1s until progress reaches 1.0 or state.job is empty
scripts/progress.sh --watch --interval 2
scripts/progress.sh --field progress # just the numeric 0..1 value
scripts/progress.sh --field state.job

Endpoint: GET /sdapi/v1/progress?skip_current_image=true. Key response fields:

• progress — float 0..1, fraction of current job complete.
• eta_relative — estimated seconds remaining.
• state.job — current job name (empty string when idle).
• state.sampling_step / state.sampling_steps — current step index / total.
• current_image — base64 PNG preview of the in-progress image (omitted by the script via skip_current_image=true to keep responses small; fetch raw with curl if needed).

Cancelling

scripts/cancel.sh         # POST /sdapi/v1/interrupt — stop current job, return current partial result
scripts/cancel.sh --skip  # POST /sdapi/v1/skip — skip current job in a batch

Note: interrupt is cooperative — it tells the sampler to stop at the next step. The pending generate.sh call will return with whatever the model produced so far (often a usable but partial image). It does not raise an HTTP error on the txt2img call.

Global options (advanced)

scripts/options.sh wraps GET /sdapi/v1/options and POST /sdapi/v1/options:

scripts/options.sh get                                # print all options as JSON
scripts/options.sh get sd_model_checkpoint            # print one key
scripts/options.sh set sd_model_checkpoint '"<title>"' # set one key (value is JSON; string must be quoted)
scripts/options.sh set-json '{"k1":"v1","k2":"v2"}'   # set multiple keys
scripts/options.sh refresh-checkpoints                # POST /sdapi/v1/refresh-checkpoints

Prefer override_settings inside the txt2img request over options set — override_settings is request-scoped and reverts after the call, while options set persists globally and affects every other client.

Prompt engineering

This skill does not generate or refine prompts. When the user asks for prompt help:

1. Check whether another agent skill is available for prompt engineering (search by name: e.g. sd-prompt-builder, danbooru-prompt, image-prompt-*). If so, delegate to it. 2. Otherwise, ask the user for the prompt explicitly, or accept a natural-language description and pass it through verbatim as the prompt field. Do not invent Danbooru tags or stylistic modifiers on your own.

References

• references/api-endpoints.md — full sd-webui / Forge endpoint reference with request / response shapes for every endpoint this skill uses, plus useful adjacent ones (/sdapi/v1/memory, /sdapi/v1/png-info, etc.).
• references/txt2img-parameters.md — every txt2img request field including HiRes-fix, refiner, Forge-specific extensions (forge_additional_modules, forge_inference_memory, forge_preset), and override_settings keys.

Read these only when constructing a non-trivial request or hitting an error that needs deeper investigation.

sd-webui / Forge API endpoint reference

All paths are relative to the server base URL (e.g. http://localhost:7860). Requests use Content-Type: application/json. Authentication, when enabled, is HTTP Basic (-u user:pass).

This document covers the endpoints actually used by this skill plus a few useful adjacent ones. The full surface area of the sd-webui API is larger — consult the running server's /docs (OpenAPI Swagger UI) for everything else.

Connectivity / health

GET /sdapi/v1/samplers

Lightweight "is the server up?" probe. Returns 200 OK with an array of {name, aliases, options} once the server has finished loading. Used by probe.sh.

Enumeration

GET /sdapi/v1/sd-models

Returns the installed checkpoints:

[
  {
    "title": "model.safetensors [a1b2c3d4]",
    "model_name": "model",
    "hash": "a1b2c3d4",
    "sha256": "...",
    "filename": "/path/to/model.safetensors",
    "config": null
  }
]

Use title (includes the hash suffix) as the value of override_settings.sd_model_checkpoint — that is the unambiguous identifier the server matches against.

GET /sdapi/v1/sd-modules (Forge-only)

Forge-style extra modules (text encoders, VAE, etc.) that can be stacked at inference time:

[{"model_name": "ae.safetensors", "filename": "..."}]

Pass an array of model_name strings as override_settings.forge_additional_modules. AUTOMATIC1111 vanilla returns 404 here.

GET /sdapi/v1/samplers

[{"name": "Euler a", "aliases": ["k_euler_a"], "options": {}}]

Use name as sampler_name in the txt2img request.

GET /sdapi/v1/schedulers

[{"name": "automatic", "label": "Automatic"}]

Use label (or name) as scheduler in the txt2img request. Vanilla AUTOMATIC1111 may not expose schedulers separately from samplers — in that case the endpoint returns an empty array and scheduler is ignored.

GET /sdapi/v1/prompt-styles

[{"name": "my-style", "prompt": "...", "negative_prompt": "..."}]

Use name (as an array element) in the styles field of the txt2img request.

GET /sdapi/v1/upscalers

[{"name": "Lanczos", "model_name": null, "model_path": null, "model_url": null, "scale": 4}]

Used for extras API or hr_upscaler in HiRes-fix.

GET /sdapi/v1/loras

LoRAs installed on the server. Reference them inline in prompts as <lora:name:weight>.

GET /sdapi/v1/embeddings

{"loaded": {...}, "skipped": {...}} — textual inversion embeddings. Reference them in prompts by their key name.

Generation

POST /sdapi/v1/txt2img

Submits a text-to-image job. Synchronous: the request blocks until the image is generated. Body is the txt2img request JSON — see txt2img-parameters.md. Response:

{
  "images": ["<base64 PNG>", "..."],
  "parameters": { "...": "echoed request" },
  "info": "<JSON-encoded string with seed, all_prompts, subseed, sampler_name, etc.>"
}

To get the seed: jq -r '.info | fromjson | .seed'.

POST /sdapi/v1/img2img

Same structure as txt2img but additionally accepts init_images (array of base64-encoded source images) and denoising_strength. Not wrapped by a script in this skill — call directly with curl if needed.

Progress / control

GET /sdapi/v1/progress?skip_current_image=true

{
  "progress": 0.42,
  "eta_relative": 3.7,
  "state": {
    "skipped": false,
    "interrupted": false,
    "job": "txt2img",
    "job_count": 1,
    "job_timestamp": "20240101000000",
    "job_no": 0,
    "sampling_step": 12,
    "sampling_steps": 28
  },
  "current_image": null,
  "textinfo": "..."
}

skip_current_image=true omits the base64 preview to keep responses small. Set to false to fetch the in-progress image.

state.job is the empty string when the server is idle.

POST /sdapi/v1/interrupt

Body {} (or empty). Returns {}. Cooperatively stops the current sampler — the in-flight txt2img call returns normally with the partial result. Use to cancel a job.

POST /sdapi/v1/skip

Body {}. Skips the current job in a batch (job_count > 1); subsequent jobs in the batch continue.

GET /queue/status

Not under /sdapi/v1/. Returns the underlying Gradio queue layer status as an EstimationMessage:

{
  "event_id": "...",
  "msg": "estimation",
  "queue_size": 0,
  "rank": null,
  "rank_eta": null
}

queue_size is the queue depth; rank / rank_eta are populated when the caller's request is queued behind others. Useful when multiple clients share the server. The reference plugin wraps this as getQueueStatus(). This skill does not provide a dedicated script — call directly:

curl -sS "${SD_WEBUI_URL}/queue/status"

Options (global / persistent)

GET /sdapi/v1/options

Returns a large flat JSON object of every setting in the UI. Notable keys:

• sd_model_checkpoint — active checkpoint title
• samples_format — "png" / "jpg" / "webp" etc.
• CLIP_stop_at_last_layers — clip-skip integer
• sd_vae — active VAE name or "Automatic"
• eta_noise_seed_delta

POST /sdapi/v1/options

Body: a JSON object with one or more option keys to update. Returns {}. Persists across requests and affects every client connected to the server.

For request-scoped overrides, use override_settings inside the txt2img/img2img body together with override_settings_restore_afterwards: true (this skill does both).

POST /sdapi/v1/refresh-checkpoints

Body {}. Tells the server to rescan its models folder. Useful after dropping a new .safetensors file in.

Less-common but useful

Note: vanilla AUTOMATIC1111 historically exposed /sdapi/v1/hypernetworks and /sdapi/v1/sd-vae (GET); both are absent on the current Forge-classic API surface. Always confirm against the running server:

curl -sS "${SD_WEBUI_URL}/openapi.json" | jq -r '.paths | keys[]'

txt2img request parameters

Body of POST /sdapi/v1/txt2img. Only prompt is strictly required; everything else has a server-side default. Numeric defaults shown match AUTOMATIC1111 / Forge stock behaviour and may differ on your server depending on /sdapi/v1/options.

Core

HiRes-fix

Refiner (SDXL)

override_settings

Object whose keys are entries from GET /sdapi/v1/options. Applied only for this request when override_settings_restore_afterwards: true (recommended — always set this).

Most commonly used keys:

Forge-specific override_settings

These keys only exist on Forge / Forge-classic forks. Verify against your server with scripts/options.sh get | jq 'keys[] | select(startswith("forge_"))'.

override_settings_restore_afterwards

Boolean, default false. Always set to `true` unless you specifically intend to mutate the server's persistent settings.

Script-args fields (advanced)

Minimal request example

{
  "prompt": "a watercolor portrait of a fox",
  "negative_prompt": "lowres, jpeg artifacts",
  "steps": 28,
  "cfg_scale": 4.0,
  "width": 832,
  "height": 1216,
  "sampler_name": "Euler a",
  "scheduler": "Beta",
  "styles": [],
  "override_settings": {
    "sd_model_checkpoint": "myModel.safetensors [a1b2c3d4]"
  }
}

Forge example with extra modules

{
  "prompt": "1girl, looking at viewer, masterpiece",
  "negative_prompt": "lowres, bad anatomy",
  "steps": 32,
  "cfg_scale": 4.0,
  "width": 832,
  "height": 1216,
  "sampler_name": "ER SDE",
  "scheduler": "Beta",
  "styles": ["anime-detail"],
  "override_settings": {
    "sd_model_checkpoint": "illustriousXL.safetensors [deadbeef]",
    "forge_additional_modules": ["clip_l.safetensors", "t5xxl_fp16.safetensors", "ae.safetensors"]
  }
}