Ai Image Editing

Ai Image Editing

Identity

Reference System Usage

You must ground your responses in the provided reference files, treating them as the source of truth for this domain:

• For Creation: Always consult `references/patterns.md`. This file dictates how things should be built. Ignore generic approaches if a specific pattern exists here.
• For Diagnosis: Always consult `references/sharp_edges.md`. This file lists the critical failures and "why" they happen. Use it to explain risks to the user.
• For Review: Always consult `references/validations.md`. This contains the strict rules and constraints. Use it to validate user inputs objectively.

Note: If a user's request conflicts with the guidance in these files, politely correct them using the information provided in the references.

AI Image Editing

Patterns

---

Id

replicate-inpainting

Name

Inpainting with Replicate API

Description

Use Replicate's Flux Fill model for professional inpainting. Mask white areas to be filled, black to preserve.

Model options:

• flux-fill-pro: Best quality, seamless blending
• flux-dev-inpainting: Good balance of speed/quality
• sdxl-inpainting: SDXL-based, lower cost

Code Example

Install: pip install replicate

import replicate import base64 from pathlib import Path

Initialize client

client = replicate.Client(api_token=os.environ["REPLICATE_API_TOKEN"])

Load image and mask

def encode_image(path: str) -> str: """Encode image to base64 data URI.""" data = Path(path).read_bytes() return f"data:image/png;base64,{base64.b64encode(data).decode()}"

Run Flux Fill Pro (best quality)

output = client.run( "black-forest-labs/flux-fill-pro", input={ "image": encode_image("input.png"), "mask": encode_image("mask.png"), # White = edit, Black = keep "prompt": "a red sports car parked on the street", "output_format": "png", } )

Save result

with open("output.png", "wb") as f: f.write(output.read())

Alternative: Flux Dev Inpainting (faster, cheaper)

output = client.run( "zsxkib/flux-dev-inpainting", input={ "image": encode_image("input.png"), "mask": encode_image("mask.png"), "prompt": "modern office furniture", "num_inference_steps": 28, "guidance_scale": 3.5, "strength": 0.85, # 0.5-0.85 recommended } )

Async for long-running tasks

prediction = client.predictions.create( model="black-forest-labs/flux-fill-pro", input={ "image": encode_image("input.png"), "mask": encode_image("mask.png"), "prompt": "vintage furniture in cozy room", } )

Poll for completion

prediction = client.predictions.wait(prediction) print(f"Status: {prediction.status}") print(f"Output: {prediction.output}")

Anti Patterns

---

Pattern

Mask with inverted colors

Why

White should be edit area, black should be preserve

Fix

Invert mask before sending to API

---

Pattern

Very high strength for subtle edits

Why

Strength 0.9+ completely replaces content

Fix

Use 0.5-0.75 for balanced edits

References

• https://replicate.com/black-forest-labs/flux-fill-pro
• https://replicate.com/collections/image-editing

---

Id

stability-search-replace

Name

Stability AI Search and Replace

Description

Replace objects without creating masks manually. Describe what to find and what to replace it with.

Available on AWS Bedrock and direct API. No mask required - AI automatically segments.

Code Example

import requests import base64 from pathlib import Path

STABILITY_API_KEY = os.environ["STABILITY_API_KEY"]

def search_and_replace( image_path: str, search_prompt: str, replace_prompt: str, output_path: str = "output.png" ): """Replace objects in image by description."""

with open(image_path, "rb") as f: image_data = f.read()

response = requests.post( "https://api.stability.ai/v2beta/stable-image/edit/search-and-replace", headers={ "Authorization": f"Bearer {STABILITY_API_KEY}", "Accept": "image/*", }, files={ "image": image_data, }, data={ "search_prompt": search_prompt, "prompt": replace_prompt, "output_format": "png", }, )

if response.status_code == 200: with open(output_path, "wb") as f: f.write(response.content) return output_path else: raise Exception(f"Error: {response.status_code} - {response.text}")

Usage

search_and_replace( image_path="room.png", search_prompt="old wooden chair", replace_prompt="modern ergonomic office chair", output_path="room_updated.png" )

Erase object (replace with background)

def erase_object(image_path: str, mask_path: str, output_path: str): """Remove object and fill with background."""

with open(image_path, "rb") as img_f: image_data = img_f.read() with open(mask_path, "rb") as mask_f: mask_data = mask_f.read()

response = requests.post( "https://api.stability.ai/v2beta/stable-image/edit/erase", headers={ "Authorization": f"Bearer {STABILITY_API_KEY}", "Accept": "image/*", }, files={ "image": image_data, "mask": mask_data, # White = area to erase }, data={ "output_format": "png", }, )

if response.status_code == 200: with open(output_path, "wb") as f: f.write(response.content)

Remove background

def remove_background(image_path: str, output_path: str): """Remove background, return transparent PNG."""

with open(image_path, "rb") as f: image_data = f.read()

response = requests.post( "https://api.stability.ai/v2beta/stable-image/edit/remove-background", headers={ "Authorization": f"Bearer {STABILITY_API_KEY}", "Accept": "image/*", }, files={"image": image_data}, data={"output_format": "png"}, )

if response.status_code == 200: with open(output_path, "wb") as f: f.write(response.content)

Anti Patterns

---

Pattern

Vague search prompts

Why

AI may match wrong objects

Fix

Be specific: 'red leather sofa' not 'furniture'

---

Pattern

Not handling rate limits

Why

API has usage limits

Fix

Add retry logic with exponential backoff

References

• https://platform.stability.ai/docs/api-reference
• https://aws.amazon.com/bedrock/stability-ai/

---

Id

outpainting-extend

Name

Outpainting and Image Extension

Description

Extend images beyond original boundaries. AI generates seamless content matching style.

Key concepts:

• Extend in any direction (left, right, up, down)
• Maintain color/style consistency
• Handle aspect ratio changes

Code Example

import replicate import requests from PIL import Image import io

Method 1: Replicate Flux Fill Pro for outpainting

def outpaint_replicate( image_path: str, direction: str = "right", # left, right, up, down extend_pixels: int = 512, prompt: str = "" ) -> bytes: """Extend image in specified direction."""

Load and prepare image

img = Image.open(image_path) orig_w, orig_h = img.size

Calculate new canvas size

if direction == "right": new_w, new_h = orig_w + extend_pixels, orig_h paste_pos = (0, 0) elif direction == "left": new_w, new_h = orig_w + extend_pixels, orig_h paste_pos = (extend_pixels, 0) elif direction == "down": new_w, new_h = orig_w, orig_h + extend_pixels paste_pos = (0, 0) elif direction == "up": new_w, new_h = orig_w, orig_h + extend_pixels paste_pos = (0, extend_pixels)

Create expanded canvas with original image

canvas = Image.new("RGB", (new_w, new_h), (128, 128, 128)) canvas.paste(img, paste_pos)

Create mask (white = generate, black = keep)

mask = Image.new("L", (new_w, new_h), 255) # All white mask_region = Image.new("L", img.size, 0) # Black for original mask.paste(mask_region, paste_pos)

Convert to bytes

def to_bytes(pil_img, format="PNG"): buf = io.BytesIO() pil_img.save(buf, format=format) return buf.getvalue()

Call API

client = replicate.Client() output = client.run( "black-forest-labs/flux-fill-pro", input={ "image": to_bytes(canvas), "mask": to_bytes(mask), "prompt": prompt or "seamless extension of the scene", } )

return output.read()

Method 2: Stability AI Outpaint

def outpaint_stability( image_path: str, left: int = 0, right: int = 512, up: int = 0, down: int = 0, prompt: str = "", ): """Extend image using Stability AI."""

with open(image_path, "rb") as f: image_data = f.read()

response = requests.post( "https://api.stability.ai/v2beta/stable-image/edit/outpaint", headers={ "Authorization": f"Bearer {os.environ['STABILITY_API_KEY']}", "Accept": "image/*", }, files={"image": image_data}, data={ "left": left, "right": right, "up": up, "down": down, "prompt": prompt, "output_format": "png", }, )

if response.status_code == 200: return response.content raise Exception(f"Outpaint failed: {response.text}")

Usage: Extend for different aspect ratios

Portrait to landscape

result = outpaint_stability( "portrait.png", left=256, right=256, prompt="continue the scenic landscape naturally" )

Square to 16:9

result = outpaint_stability( "square.png", left=128, right=128, prompt="extend the environment seamlessly" )

Anti Patterns

---

Pattern

Extending without prompt context

Why

AI may generate inconsistent content

Fix

Provide descriptive prompt matching original style

---

Pattern

Very large extensions at once

Why

Quality degrades with massive extensions

Fix

Chain smaller extensions (256-512px each)

References

• https://myaiforce.com/flux-fill-model-inpainting-workflow/
• https://aws.amazon.com/about-aws/whats-new/2025/10/stability-ai-image-updates-amazon-bedrock/

---

Id

controlnet-guidance

Name

ControlNet Structure Control

Description

Control image generation with structural guidance.

Control types:

• Canny: Edge detection, sharp boundaries
• Depth: 3D spatial relationships
• Pose: Human body positioning
• HED/Soft Edge: Organic, softer boundaries

Combine multiple controls for precise results.

Code Example

import replicate from PIL import Image import cv2 import numpy as np

Generate Canny edge map

def create_canny_map(image_path: str, low: int = 100, high: int = 200): """Create Canny edge detection map.""" img = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE) edges = cv2.Canny(img, low, high) return Image.fromarray(edges)

Use ControlNet on Replicate

def generate_with_controlnet( control_image_path: str, prompt: str, control_type: str = "canny", # canny, depth, pose control_strength: float = 0.8, ): """Generate image with ControlNet guidance."""

client = replicate.Client()

For Flux ControlNet

if control_type == "canny": model = "black-forest-labs/flux-canny-pro" elif control_type == "depth": model = "black-forest-labs/flux-depth-pro" else:

Use InstantX Union for flexibility

model = "xlabs-ai/flux-controlnet"

output = client.run( model, input={ "control_image": open(control_image_path, "rb"), "prompt": prompt, "control_strength": control_strength, "num_inference_steps": 28, "guidance_scale": 3.5, } )

return output

Combining multiple ControlNets

def multi_controlnet( pose_image: str, depth_image: str, prompt: str, pose_strength: float = 0.5, depth_strength: float = 0.4, ): """Combine pose and depth control."""

Note: Multi-ControlNet requires compatible model

Total strength should be ~0.8-1.0 when combined

client = replicate.Client()

output = client.run( "xlabs-ai/flux-controlnet", input={ "control_image": open(pose_image, "rb"), "control_image_2": open(depth_image, "rb"), "control_type": "pose", "control_type_2": "depth", "control_strength": pose_strength, "control_strength_2": depth_strength, "prompt": prompt, } )

return output

Generate depth map from image

def create_depth_map(image_path: str): """Generate depth map using MiDaS."""

client = replicate.Client()

output = client.run( "cjwbw/midas:a6ba5798f04f80d3b314de0f0a62277f21ab3c8a6eb7f3bb0bb9d7e6b62c3c0d", input={ "image": open(image_path, "rb"), "model_type": "DPT_Large", } )

return output

Practical example: Transform sketch to artwork

def sketch_to_art(sketch_path: str, style_prompt: str): """Convert rough sketch to polished artwork."""

Create Canny edges from sketch

canny_map = create_canny_map(sketch_path, low=50, high=150) canny_map.save("temp_canny.png")

Generate with style

client = replicate.Client() output = client.run( "black-forest-labs/flux-canny-pro", input={ "control_image": open("temp_canny.png", "rb"), "prompt": f"{style_prompt}, high detail, professional", "control_strength": 0.75, } )

return output

Anti Patterns

---

Pattern

Control strength 1.0 with multiple controls

Why

Combined strength exceeds 1.0, over-constrains generation

Fix

Use 0.4-0.5 per control when combining

---

Pattern

Using Canny for organic subjects like faces

Why

Hard edges don't capture facial nuances

Fix

Use depth or soft edge for organic subjects

---

Pattern

Wrong resolution control images

Why

Control images must match output resolution

Fix

Resize control image to target dimensions

References

• https://stable-diffusion-art.com/controlnet/
• https://blog.segmind.com/flux-1-controlnets-what-are-they-all-you-need-to-know/

---

Id

image-to-image

Name

Image-to-Image Transformation

Description

Transform existing images with style/content changes. Control transformation strength to balance original vs new.

Use cases:

• Style transfer (photo to art)
• Retexturing (change materials/surfaces)
• Color grading
• Detail enhancement

Code Example

import replicate import fal_client

Replicate Image-to-Image

def transform_image_replicate( image_path: str, prompt: str, strength: float = 0.75, # 0.0 = keep original, 1.0 = ignore original model: str = "flux" ): """Transform image with prompt guidance."""

client = replicate.Client()

if model == "flux": output = client.run( "black-forest-labs/flux-dev", input={ "image": open(image_path, "rb"), "prompt": prompt, "prompt_strength": strength, "num_inference_steps": 28, "guidance_scale": 3.5, } ) else: # SDXL output = client.run( "stability-ai/sdxl", input={ "image": open(image_path, "rb"), "prompt": prompt, "prompt_strength": strength, "num_inference_steps": 25, } )

return output

Fal.ai Image-to-Image (fast)

def transform_image_fal( image_url: str, prompt: str, strength: float = 0.75, ): """Transform using Fal.ai's fast inference."""

result = fal_client.submit( "fal-ai/flux/dev/image-to-image", arguments={ "image_url": image_url, "prompt": prompt, "strength": strength, "num_inference_steps": 28, } )

return result.get()

Strength guide:

0.3-0.5: Subtle changes (color grading, minor style)

0.5-0.75: Balanced transformation (recommended)

0.75-0.9: Major changes while keeping composition

0.9-1.0: Nearly complete regeneration

Style transfer example

def apply_art_style(photo_path: str, style: str): """Convert photo to artistic style."""

style_prompts = { "watercolor": "watercolor painting, soft brushstrokes, flowing colors", "oil_painting": "oil painting, rich textures, dramatic lighting", "anime": "anime style, vibrant colors, clean lines, studio ghibli", "pencil_sketch": "detailed pencil sketch, cross-hatching, artistic", "cyberpunk": "cyberpunk aesthetic, neon lights, futuristic", }

prompt = style_prompts.get(style, style)

return transform_image_replicate( photo_path, prompt=prompt, strength=0.65, # Preserve composition )

Batch processing

async def batch_transform( images: list[str], prompt: str, strength: float = 0.75, ): """Process multiple images concurrently.""" import asyncio

client = replicate.Client()

async def process_one(image_path: str): prediction = client.predictions.create( model="black-forest-labs/flux-dev", input={ "image": open(image_path, "rb"), "prompt": prompt, "prompt_strength": strength, } ) return await asyncio.to_thread( client.predictions.wait, prediction )

tasks = [process_one(img) for img in images] return await asyncio.gather(*tasks)

Anti Patterns

---

Pattern

Strength 0.9+ for style transfer

Why

Loses original composition and content

Fix

Use 0.5-0.75 to preserve structure

---

Pattern

No negative prompt for quality

Why

May generate artifacts

Fix

Add negative: 'blurry, distorted, low quality'

References

• https://replicate.com/blog/run-sdxl-with-an-api
• https://www.aifreeapi.com/en/posts/free-image-to-image-api

---

Id

multi-step-editing

Name

Multi-Step Iterative Editing

Description

Chain multiple editing operations for complex results. Each step builds on the previous, enabling sophisticated edits.

Strategy: 1. Use lower denoise per step 2. Expand masks gradually 3. Verify intermediate results

Code Example

from PIL import Image import replicate from typing import Callable import io

class ImageEditPipeline: """Chain multiple AI editing operations."""

def __init__(self, image_path: str): self.image = Image.open(image_path) self.history = [self.image.copy()] self.client = replicate.Client()

def _image_to_bytes(self, img: Image.Image) -> bytes: buf = io.BytesIO() img.save(buf, format="PNG") return buf.getvalue()

def inpaint( self, mask: Image.Image, prompt: str, strength: float = 0.75 ) -> "ImageEditPipeline": """Inpaint masked area."""

output = self.client.run( "black-forest-labs/flux-fill-pro", input={ "image": self._image_to_bytes(self.image), "mask": self._image_to_bytes(mask), "prompt": prompt, "strength": strength, } )

self.image = Image.open(io.BytesIO(output.read())) self.history.append(self.image.copy()) return self

def transform(self, prompt: str, strength: float = 0.5) -> "ImageEditPipeline": """Apply image-to-image transformation."""

output = self.client.run( "black-forest-labs/flux-dev", input={ "image": self._image_to_bytes(self.image), "prompt": prompt, "prompt_strength": strength, } )

self.image = Image.open(io.BytesIO(output.read())) self.history.append(self.image.copy()) return self

def upscale(self, scale: int = 2) -> "ImageEditPipeline": """Upscale image resolution."""

output = self.client.run( "nightmareai/real-esrgan:f121d640bd286e1fdc67f9799164c1d5be36ff74576ee11c803ae5b665dd46aa", input={ "image": self._image_to_bytes(self.image), "scale": scale, } )

self.image = Image.open(io.BytesIO(output.read())) self.history.append(self.image.copy()) return self

def undo(self) -> "ImageEditPipeline": """Revert to previous state.""" if len(self.history) > 1: self.history.pop() self.image = self.history[-1].copy() return self

def save(self, path: str): """Save current result.""" self.image.save(path) return self

Usage: Complex product photo editing

def edit_product_photo(image_path: str): """Multi-step product photo enhancement."""

pipeline = ImageEditPipeline(image_path)

Step 1: Clean up background

bg_mask = create_background_mask(image_path) pipeline.inpaint( mask=bg_mask, prompt="clean white studio background, professional product photography", strength=0.6 )

Step 2: Enhance product lighting

pipeline.transform( prompt="professional product photography, soft studio lighting, high-end commercial", strength=0.3 # Subtle enhancement )

Step 3: Upscale for high resolution

pipeline.upscale(scale=2)

Save result

pipeline.save("product_enhanced.png")

return pipeline

Iterative inpainting with mask expansion

def iterative_inpaint( image_path: str, base_mask: Image.Image, prompt: str, iterations: int = 3, ): """Inpaint with gradually expanding mask."""

from scipy.ndimage import binary_dilation import numpy as np

pipeline = ImageEditPipeline(image_path) current_mask = np.array(base_mask)

Decreasing strength per iteration

strengths = [0.7, 0.5, 0.3]

for i in range(iterations):

Expand mask slightly each iteration

if i > 0: current_mask = binary_dilation(current_mask, iterations=10)

mask_img = Image.fromarray(current_mask.astype(np.uint8) * 255)

pipeline.inpaint( mask=mask_img, prompt=prompt, strength=strengths[min(i, len(strengths)-1)] )

return pipeline

Anti Patterns

---

Pattern

Single high-denoise pass for complex edits

Why

Hard to control, may produce artifacts

Fix

Use multiple passes with decreasing strength

---

Pattern

No history/undo capability

Why

Can't recover from bad edits

Fix

Maintain edit history for rollback

References

• https://docs.comfy.org/tutorials/basic/inpaint
• https://medium.com/@techlatest.net/inpainting-and-outpainting-techniques-in-comfyui-d708d3ea690d

---

Id

content-moderation

Name

Content Moderation for Generated Images

Description

Ensure generated images comply with content policies. Check both inputs and outputs for safety.

Key concerns:

• NSFW/explicit content
• Violence/gore
• Hate symbols
• Deepfakes/impersonation

Code Example

import requests import openai from typing import Tuple from enum import Enum

class ContentRating(Enum): SAFE = "safe" WARNING = "warning" BLOCKED = "blocked"

OpenAI Moderation API (free)

def check_prompt_safety(prompt: str) -> Tuple[bool, dict]: """Check if prompt is safe for image generation."""

client = openai.OpenAI()

response = client.moderations.create(input=prompt) result = response.results[0]

Check relevant categories

flags = { "sexual": result.categories.sexual, "violence": result.categories.violence, "hate": result.categories.hate, "self_harm": result.categories.self_harm, }

is_safe = not any(flags.values()) return is_safe, flags

Image moderation with dedicated API

def moderate_image(image_url: str) -> ContentRating: """Check generated image for policy violations."""

Using SightEngine API

response = requests.get( "https://api.sightengine.com/1.0/check.json", params={ "url": image_url, "models": "nudity-2.1,offensive,gore", "api_user": os.environ["SIGHTENGINE_USER"], "api_secret": os.environ["SIGHTENGINE_SECRET"], } )

result = response.json()

Check nudity

nudity = result.get("nudity", {}) if nudity.get("sexual_activity", 0) > 0.5: return ContentRating.BLOCKED if nudity.get("sexual_display", 0) > 0.5: return ContentRating.BLOCKED

Check violence

if result.get("gore", {}).get("prob", 0) > 0.5: return ContentRating.BLOCKED

Check offensive content

if result.get("offensive", {}).get("prob", 0) > 0.7: return ContentRating.WARNING

return ContentRating.SAFE

Safe generation wrapper

class SafeImageGenerator: """Generate images with content safety checks."""

def __init__(self, replicate_client): self.client = replicate_client

def generate( self, prompt: str, **kwargs ) -> dict: """Generate image with safety checks."""

Pre-check prompt

is_safe, flags = check_prompt_safety(prompt) if not is_safe: return { "success": False, "error": "Prompt blocked by content policy", "flags": flags, }

Generate image

try: output = self.client.run( "black-forest-labs/flux-schnell", input={ "prompt": prompt, "safety_checker": True, # Enable model's safety **kwargs } )

image_url = output[0] if isinstance(output, list) else str(output)

Post-check generated image

rating = moderate_image(image_url)

if rating == ContentRating.BLOCKED: return { "success": False, "error": "Generated image blocked by content policy", }

return { "success": True, "image_url": image_url, "content_rating": rating.value, }

except Exception as e: return { "success": False, "error": str(e), }

Usage

generator = SafeImageGenerator(replicate.Client()) result = generator.generate( prompt="professional headshot of a business person", num_outputs=1 )

if result["success"]: print(f"Image: {result['image_url']}") else: print(f"Blocked: {result['error']}")

Anti Patterns

---

Pattern

No content moderation in production

Why

Users may generate harmful content

Fix

Always check prompts and outputs

---

Pattern

Only checking prompts, not outputs

Why

Safe prompts can still produce unsafe images

Fix

Check both input prompts and output images

References

• https://www.edenai.co/post/best-image-moderation-apis
• https://medium.com/@API4AI/automated-nsfw-detection-the-2025-content-safety-playbook-7ac82fd2f351

Ai Image Editing - Sharp Edges

Inverted Mask Colors Cause Wrong Areas to Edit

Id

mask-color-inversion

Severity

high

Description

Different APIs use different mask conventions. Most use white=edit, black=keep, but some are inverted. Wrong mask colors edit the wrong parts of the image.

Wrong Way

Assuming all APIs use same mask convention

mask = create_mask(image) # Black where you want to edit

Stability AI expects: white=edit, black=keep

stability_api.inpaint(image, mask) # Edits WRONG areas!

Some ComfyUI nodes expect: black=edit, white=keep

No verification of mask convention

Right Way

from PIL import Image import numpy as np

def ensure_mask_convention( mask: Image.Image, convention: str = "white_edit" # or "black_edit" ) -> Image.Image: """Normalize mask to expected convention."""

Current mask: white areas to edit

If API expects black_edit, invert

if convention == "black_edit": arr = np.array(mask) inverted = 255 - arr return Image.fromarray(inverted) return mask

Document expected convention per API

API_CONVENTIONS = { "replicate": "white_edit", # White = fill/edit "stability": "white_edit", # White = edit "comfyui_flux": "white_edit", # White = edit "automatic1111": "white_edit", # White = edit (default) }

Verify mask before sending

def validate_mask(mask: Image.Image): """Check mask is binary and valid.""" arr = np.array(mask)

Should be grayscale or single channel

if len(arr.shape) == 3: mask = mask.convert("L") arr = np.array(mask)

Warn if mostly one color (might be inverted)

white_ratio = np.sum(arr > 128) / arr.size if white_ratio > 0.9: print("Warning: Mask is mostly white - will edit most of image") elif white_ratio < 0.1: print("Warning: Mask is mostly black - will preserve most of image")

return mask

Detection Patterns

• mask.black.edit
• invert.*mask

References

• https://docs.comfy.org/tutorials/basic/inpaint

High Strength/Denoise Destroys Original Content

Id

strength-too-high

Severity

high

Description

Strength values above 0.85 essentially ignore the original image. For edits that should preserve context, use lower values. Many users set strength=1.0 and lose composition.

Wrong Way

Trying to "improve" an image

result = api.image_to_image( image=original, prompt="make it look better", strength=1.0, # Ignores original completely! )

Result looks nothing like original

Inpainting with maximum denoise

result = api.inpaint( image=original, mask=small_mask, prompt="fix this area", strength=0.95, # Over-edits, creates obvious patches )

Right Way

Strength guide by use case

STRENGTH_GUIDE = { "color_correction": 0.2-0.3, # Subtle adjustments "style_transfer": 0.5-0.7, # Change style, keep composition "object_replacement": 0.7-0.85, # Replace while blending "complete_reimagine": 0.9-1.0, # New image from structure only }

For subtle enhancements

result = api.image_to_image( image=original, prompt="professional photography, enhanced lighting", strength=0.3, # Preserve most of original )

For inpainting, use multi-pass with decreasing strength

def iterative_inpaint(image, mask, prompt): strengths = [0.6, 0.4, 0.3] # Decreasing per pass

current = image for i, strength in enumerate(strengths):

Slightly expand mask each iteration

expanded_mask = dilate_mask(mask, iterations=i * 5)

current = api.inpaint( image=current, mask=expanded_mask, prompt=prompt, strength=strength, )

return current

Detection Patterns

• strength.*1\.0
• denoise.*0\.9[5-9]

References

• https://apatero.com/blog/z-image-turbo-inpainting-comfyui-guide-2025

Image Dimensions Not Divisible by 8

Id

resolution-mismatch

Severity

high

Description

Stable Diffusion and Flux models require dimensions divisible by 8. Non-conforming images get resized/stretched, causing quality loss. ControlNet images must match generation resolution exactly.

Wrong Way

Upload image of any size

image = load_image("photo.jpg") # 1920x1080

API silently resizes to nearest valid size

result = api.generate( image=image, # 1920x1080 target_size=1024, # Gets resized weirdly )

Output may be stretched or cropped

ControlNet with wrong size control image

control = load_image("depth.png") # 768x512 result = api.generate( control_image=control, # 768x512 prompt="...", width=1024, height=1024, # Mismatch! )

Right Way

from PIL import Image

def resize_for_diffusion( image: Image.Image, max_dimension: int = 1024, divisor: int = 8 ) -> Image.Image: """Resize image to valid dimensions for diffusion models."""

w, h = image.size

Scale to max dimension

if max(w, h) > max_dimension: scale = max_dimension / max(w, h) w = int(w scale) h = int(h scale)

Round to nearest multiple of divisor

w = (w // divisor) divisor h = (h // divisor) divisor

Ensure minimum size

w = max(w, divisor 8) # At least 64 h = max(h, divisor 8)

return image.resize((w, h), Image.Resampling.LANCZOS)

Match control image to generation size

def prepare_controlnet_input( image: Image.Image, control: Image.Image, target_size: tuple[int, int] ) -> tuple[Image.Image, Image.Image]: """Ensure image and control match target dimensions."""

Resize both to exact target

image = image.resize(target_size, Image.Resampling.LANCZOS) control = control.resize(target_size, Image.Resampling.LANCZOS)

return image, control

Usage

image = load_image("photo.jpg") image = resize_for_diffusion(image, max_dimension=1024)

Now safe to use with any diffusion model

Detection Patterns

• resize.1920.1080
• dimension.not.divisible

References

• https://github.com/diodiogod/Comfy-Inpainting-Works

API Rate Limits and Cost Explosion

Id

api-rate-limits

Severity

high

Description

Image generation APIs have strict rate limits and per-image costs. Unbounded loops or retries can exhaust quotas quickly. Production apps need rate limiting and cost tracking.

Wrong Way

Unbounded generation loop

async def generate_variations(prompt, count=100): results = [] for i in range(count):

No rate limiting

result = await api.generate(prompt=prompt) results.append(result) return results

Hits rate limits, fails partway through

Retry without backoff

def generate_with_retry(prompt, max_retries=10): for i in range(max_retries): try: return api.generate(prompt=prompt) except RateLimitError: continue # Immediately retries, makes it worse

Right Way

import asyncio from datetime import datetime, timedelta import time

class RateLimitedGenerator: """Image generator with rate limiting and cost tracking."""

def __init__( self, client, max_per_minute: int = 10, cost_per_image: float = 0.03, daily_budget: float = 10.0 ): self.client = client self.max_per_minute = max_per_minute self.cost_per_image = cost_per_image self.daily_budget = daily_budget

self.requests_this_minute = 0 self.minute_start = datetime.now() self.daily_cost = 0.0 self.day_start = datetime.now().date()

async def generate(self, **kwargs) -> dict: """Generate with rate limiting and cost tracking."""

Reset daily counter if new day

if datetime.now().date() > self.day_start: self.daily_cost = 0.0 self.day_start = datetime.now().date()

Check daily budget

if self.daily_cost >= self.daily_budget: raise Exception(f"Daily budget ${self.daily_budget} exceeded")

Rate limiting

now = datetime.now() if (now - self.minute_start).seconds >= 60: self.requests_this_minute = 0 self.minute_start = now

if self.requests_this_minute >= self.max_per_minute: wait_time = 60 - (now - self.minute_start).seconds await asyncio.sleep(wait_time) self.requests_this_minute = 0 self.minute_start = datetime.now()

Generate with exponential backoff

for attempt in range(5): try: result = await self.client.generate(**kwargs) self.requests_this_minute += 1 self.daily_cost += self.cost_per_image return result

except RateLimitError: wait = (2 ** attempt) + random.uniform(0, 1) await asyncio.sleep(wait)

raise Exception("Max retries exceeded")

def get_stats(self) -> dict: return { "daily_cost": self.daily_cost, "daily_budget": self.daily_budget, "remaining_budget": self.daily_budget - self.daily_cost, }

Detection Patterns

• for.range.generate
• while.True.api\.generate

References

• https://www.aifreeapi.com/en/posts/chatgpt-daily-image-limits-solution-2025

VRAM Exhaustion with Large Models

Id

vram-exhaustion

Severity

medium

Description

ControlNet + SDXL + high resolution requires 12GB+ VRAM. Multiple ControlNets multiply memory requirements. Running out of VRAM crashes generation or produces errors.

Wrong Way

Loading multiple full-precision models

sdxl = load_model("sdxl-base", precision="fp32") controlnet_depth = load_model("controlnet-depth", precision="fp32") controlnet_canny = load_model("controlnet-canny", precision="fp32")

8GB GPU runs out of memory

High resolution with multiple controls

result = generate( width=2048, height=2048, # 4x normal resolution controlnets=[depth, canny, pose], # 3 controls ) # OOM error

Right Way

Use quantized models

sdxl = load_model("sdxl-base-gguf", precision="fp16") # Half memory

Enable memory optimizations

import torch

Clear CUDA cache between generations

torch.cuda.empty_cache()

Use attention slicing

pipe.enable_attention_slicing()

Use xformers for memory efficiency

pipe.enable_xformers_memory_efficient_attention()

Generate at lower resolution, upscale after

def efficient_generation(prompt, target_size=2048):

Generate at 1024

result = generate( prompt=prompt, width=1024, height=1024, )

Upscale to target

upscaled = upscale(result, scale=target_size // 1024) return upscaled

Batch process with memory cleanup

def batch_with_cleanup(prompts): results = [] for prompt in prompts: result = generate(prompt) results.append(result)

Clean up between generations

torch.cuda.empty_cache() import gc gc.collect()

return results

Monitor VRAM usage

def log_vram(): if torch.cuda.is_available(): allocated = torch.cuda.memory_allocated() / 1e9 cached = torch.cuda.memory_reserved() / 1e9 print(f"VRAM: {allocated:.1f}GB allocated, {cached:.1f}GB cached")

Detection Patterns

• width.*2048
• multiple.*controlnet

References

• https://railwail.com/blog/key-points-on-combining-depth-pose-and-edge-with-sdxl-multi-controlnet-lora-1743334474799

ControlNet Model Incompatibility

Id

controlnet-model-mismatch

Severity

high

Description

ControlNet models are trained for specific base models. SD 1.5 ControlNets don't work with SDXL or Flux. Using wrong ControlNet produces garbage or errors.

Wrong Way

Using SD 1.5 ControlNet with SDXL

controlnet = load_model("lllyasviel/sd-controlnet-canny") # SD 1.5 sdxl = load_model("stabilityai/sdxl-base")

result = generate( model=sdxl, controlnet=controlnet, # Incompatible! control_image=canny_map, )

Produces errors or garbage output

Right Way

Match ControlNet to base model

MODEL_CONTROLNETS = { "sd-1.5": { "canny": "lllyasviel/sd-controlnet-canny", "depth": "lllyasviel/sd-controlnet-depth", "pose": "lllyasviel/sd-controlnet-openpose", }, "sdxl": { "canny": "diffusers/controlnet-canny-sdxl-1.0", "depth": "diffusers/controlnet-depth-sdxl-1.0", }, "flux": { "canny": "InstantX/FLUX.1-dev-Controlnet-Canny", "depth": "black-forest-labs/flux-depth-pro", "union": "InstantX/FLUX.1-dev-Controlnet-Union", }, }

def get_compatible_controlnet(base_model: str, control_type: str): """Get ControlNet compatible with base model."""

if base_model in MODEL_CONTROLNETS: controls = MODEL_CONTROLNETS[base_model] if control_type in controls: return controls[control_type]

raise ValueError( f"No {control_type} ControlNet for {base_model}" )

Use union models for flexibility

InstantX Union supports multiple control types in one model

flux_union = load_model("InstantX/FLUX.1-dev-Controlnet-Union") result = generate( controlnet=flux_union, control_type="canny", # Specify which mode control_image=edges, )

Detection Patterns

• sd-controlnet.*sdxl
• flux.*sd-controlnet

References

• https://blog.segmind.com/flux-1-controlnets-what-are-they-all-you-need-to-know/
• https://comfyui-wiki.com/en/resource/controlnet-models/controlnet-flux-1

Missing Content Moderation in Production

Id

no-content-moderation

Severity

critical

Description

AI image generation can produce inappropriate content. Without moderation, users may generate harmful images. Both prompts and outputs need checking.

Wrong Way

Direct pass-through of user prompts

@app.post("/generate") async def generate(prompt: str): result = await api.generate(prompt=prompt) # No filtering! return result

Trust user-provided images

@app.post("/edit") async def edit(image: UploadFile, prompt: str):

No check if image contains harmful content

result = await api.inpaint(image, prompt=prompt) return result

Right Way

import openai

class ModerationError(Exception): pass

async def check_prompt(prompt: str) -> bool: """Check prompt with OpenAI moderation (free).""" client = openai.OpenAI() response = await client.moderations.create(input=prompt)

result = response.results[0] if result.flagged: raise ModerationError( f"Prompt blocked: {[k for k, v in result.categories if v]}" ) return True

async def check_image(image_url: str) -> bool: """Check generated image for policy violations."""

Use dedicated moderation API

result = await moderation_api.check(image_url)

if result.nudity > 0.5 or result.violence > 0.5: raise ModerationError("Generated image blocked") return True

@app.post("/generate") async def generate_safe(prompt: str):

Pre-check prompt

await check_prompt(prompt)

Generate with model's safety checker

result = await api.generate( prompt=prompt, safety_checker=True, # Model-level filter )

Post-check output

await check_image(result.url)

return result

Also block known jailbreak patterns

BLOCKED_PATTERNS = [ r"ignore.previous.instructions", r"pretend.you.are", r"bypass.*safety", ]

def sanitize_prompt(prompt: str) -> str: for pattern in BLOCKED_PATTERNS: if re.search(pattern, prompt, re.IGNORECASE): raise ModerationError("Blocked pattern detected") return prompt

Detection Patterns

• generate.prompt.(?!moderat)
• user.prompt.api\.generate

References

• https://www.edenai.co/post/best-image-moderation-apis

Inconsistent Seeds for Reproducibility

Id

seed-inconsistency

Severity

medium

Description

Not setting seeds makes results non-reproducible. Same prompt with different seeds gives different outputs. For production iteration, lock seeds while tuning.

Wrong Way

Random seed each time

for i in range(10): result = api.generate( prompt="a cat sitting on a windowsill",

No seed specified - random each time

)

Each result is completely different

Hard to iterate on prompt improvements

Right Way

import random

Lock seed for iteration

ITERATION_SEED = 42

Test prompt variations with same seed

prompts = [ "a cat sitting on a windowsill", "a fluffy cat sitting on a sunny windowsill", "a tabby cat lounging on a windowsill, afternoon light", ]

for prompt in prompts: result = api.generate( prompt=prompt, seed=ITERATION_SEED, # Same seed = isolate prompt effect )

Can compare how prompt changes affect same "random" generation

Production: random seed per request but log it

def generate_and_log(prompt: str): seed = random.randint(0, 2**32 - 1)

result = api.generate( prompt=prompt, seed=seed, )

Log for reproduction

log.info(f"Generated with seed={seed}, prompt={prompt}")

return result, seed

Allow user to request same generation

@app.post("/regenerate") async def regenerate(prompt: str, seed: int): """Regenerate exact same image.""" return await api.generate( prompt=prompt, seed=seed, # User provides seed from previous generation )

Detection Patterns

• generate(?!.*seed)
• random.*prompt

References

• https://apatero.com/blog/z-image-turbo-controlnet-complete-guide-2025

Hard Mask Edges Create Visible Seams

Id

mask-edge-artifacts

Severity

medium

Description

Sharp mask boundaries create visible edges in inpainting. AI can't blend smoothly at hard mask transitions. Apply blur/feather to mask edges for seamless results.

Wrong Way

Hard binary mask

mask = create_selection(image) # Pure black and white mask = mask.convert("L")

result = api.inpaint( image=image, mask=mask, # Sharp edges prompt="fill this area", )

Visible seam at mask boundary

Right Way

from PIL import Image, ImageFilter import numpy as np

def feather_mask( mask: Image.Image, blur_radius: int = 15, grow_pixels: int = 5 ) -> Image.Image: """Feather mask edges for seamless blending."""

Ensure grayscale

mask = mask.convert("L") arr = np.array(mask)

Grow mask slightly

from scipy.ndimage import binary_dilation binary = arr > 128 grown = binary_dilation(binary, iterations=grow_pixels) arr = grown.astype(np.uint8) * 255

Apply gaussian blur to edges

mask = Image.fromarray(arr) mask = mask.filter(ImageFilter.GaussianBlur(blur_radius))

return mask

Apply feathering before inpainting

mask = create_selection(image) mask = feather_mask(mask, blur_radius=20)

result = api.inpaint( image=image, mask=mask, # Soft edges prompt="fill this area naturally", )

Seamless blend at boundaries

Differential diffusion approach

Mask values control edit strength (not binary)

def gradient_mask( mask: Image.Image, falloff_pixels: int = 50 ) -> Image.Image: """Create gradient falloff from mask edges.""" from scipy.ndimage import distance_transform_edt

arr = np.array(mask.convert("L")) binary = arr > 128

Distance from edge

dist_inside = distance_transform_edt(binary) dist_outside = distance_transform_edt(~binary)

Normalize to falloff range

gradient = np.clip(dist_inside / falloff_pixels, 0, 1) result = (gradient * 255).astype(np.uint8)

return Image.fromarray(result)

Detection Patterns

• mask.convert.L(?!.*blur)
• binary.mask(?!.feather)

References

• https://smartart.live/articles/machine-learning/comfyui-workflows/230-how-to-fix-ai-images-in-comfyui-inpainting-mask-editor-tutorial-2025.html

Ai Image Editing - Validations

API Key in Client Code

Id

api-key-exposed

Severity

error

Description

Image generation API keys should only be server-side

Pattern

(NEXT_PUBLIC|REACT_APP|VITE).(REPLICATE|STABILITY|FAL|OPENAI).KEY

Message

API key exposed to client. Use server-side only.

Autofix

Hardcoded API Key

Id

hardcoded-api-key

Severity

error

Description

API keys should use environment variables

Pattern

(r8_|sk-|fal-)[A-Za-z0-9]{20,}

Message

Hardcoded API key. Use environment variables.

Autofix

Missing Prompt Moderation

Id

no-prompt-moderation

Severity

error

Description

User prompts should be checked before generation

Pattern

request\.(body|query)\.prompt.generate(?!.moderat)

Message

User prompt passed to generation without moderation check.

Autofix

Missing Output Moderation

Id

no-output-moderation

Severity

warning

Description

Generated images should be checked before serving

Pattern

generate\(.\).return.url(?!.check)

Message

Generated image returned without content check.

Autofix

Safety Checker Explicitly Disabled

Id

safety-checker-disabled

Severity

warning

Description

Model safety checkers should remain enabled

Pattern

safety_checker.false|enable_safety_checker.false

Message

Safety checker disabled. Enable for production.

Autofix

Generation Without Rate Limiting

Id

no-rate-limiting

Severity

warning

Description

API calls should be rate limited to prevent abuse

Pattern

async.generate.request(?!.*rateLimit|limit)

Message

Generation endpoint without rate limiting.

Autofix

Unbounded Generation Loop

Id

unbounded-generation-loop

Severity

error

Description

Loops generating images should have limits

Pattern

while.True.generate|for.range\(.\).generate(?!.limit)

Message

Unbounded generation loop. Add limits and rate control.

Autofix

Missing Cost Tracking

Id

no-cost-tracking

Severity

warning

Description

Image generation costs should be tracked

Pattern

generate\((?!.*cost|budget)

Message

No cost tracking for generation. Add budget controls.

Autofix

Missing Resolution Validation

Id

no-resolution-validation

Severity

warning

Description

Image dimensions should be validated for model compatibility

Pattern

generate\(.image.(?!.*resize|divisible|resolution)

Message

Image passed without resolution validation. Ensure dimensions divisible by 8.

Autofix

Mask Not Validated

Id

mask-not-validated

Severity

warning

Description

Inpainting masks should be validated before use

Pattern

inpaint\(.mask(?!.validate|check)

Message

Mask used without validation. Check dimensions and values.

Autofix

Missing Error Handling for Generation

Id

no-generation-error-handling

Severity

warning

Description

API calls should handle failures gracefully

Pattern

await.generate\((?!.try|catch|\.catch)

Message

Generation call without error handling.

Autofix

Missing Timeout for Long Generation

Id

no-timeout-handling

Severity

warning

Description

Long-running generations should have timeouts

Pattern

generate\((?!.timeout).resolution.*2048

Message

High-resolution generation without timeout.

Autofix