Comfyui Video Production

ComfyUI Video Production Pipeline

End-to-end video production orchestration for ComfyUI with automatic error recovery, quality validation, and instance management.

Quick Start: Which Pipeline?

Creating a multi-shot narrative video? → Keyframe Pipeline - Generate keyframes → Animate → Stitch with transitions

Animating existing images? → I2V Batch Pipeline - Load images → Queue I2V jobs → Auto-validate → Combine

Need smooth transitions between scenes? → Transition Pipeline - Crossfades, motion blur, zoom effects via FFmpeg

ComfyUI stuck or crashed? → Instance Manager - Auto-restart, health checks, queue monitoring

Debugging video issues? → Validation Suite - Check resolution, FPS, codec, face consistency, color grading

---

Core Pipelines

Pipeline 1: Keyframe-to-Video (Complete Narrative)

Use when: Creating story-driven videos with multiple distinct shots

1. Keyframe Generation Phase
   - Generate consistent keyframes with IP-Adapter/LoRA
   - Validate face consistency, lighting, pose progression
   - Save to organized directory structure
   - Auto-retry failed generations

2. I2V Animation Phase
   - Queue each keyframe to I2V model (Wan 2.2, LTX-2, AnimateDiff)
   - Monitor progress via ComfyUI API
   - Validate each clip (resolution, fps, duration)
   - Auto-retry with different seeds if failed

3. Concatenation Phase
   - Pre-flight validation (ensure all clips match)
   - Apply transition effects (crossfade, motion blur)
   - FFmpeg encoding with proper codec
   - Export final video with metadata

4. Quality Assurance
   - Face consistency check across clips
   - Color grading consistency
   - Audio sync validation (if applicable)
   - Generate QA report

Expected output: Single cohesive video with smooth transitions

---

Pipeline 2: Batch I2V Processing

Use when: You have multiple images to animate independently

1. Image Discovery
   - Scan directory for source images
   - Validate image specs (resolution, format)
   - Generate processing manifest

2. Parallel I2V Queue
   - Queue all images to ComfyUI with appropriate prompts
   - Stagger submissions to avoid overload
   - Monitor queue depth and ETA

3. Progressive Validation
   - Check each completed video immediately
   - Flag issues (wrong resolution, fps, corruption)
   - Auto-retry flagged videos

4. Export & Organize
   - Move validated videos to output directory
   - Generate index with metadata
   - Create contact sheet (thumbnail preview grid)

Expected output: Directory of validated animated clips

---

Pipeline 3: Video Concatenation with Transitions

Use when: Combining existing video clips with professional transitions

1. Clip Validation
   - Verify all clips exist and are readable
   - Check resolution, fps, codec consistency
   - Report mismatches with fix suggestions

2. Transition Planning
   - Detect scene changes (cut detection)
   - Recommend transition types (crossfade, zoom, pan)
   - Calculate transition timing

3. FFmpeg Pipeline
   - Apply transitions between clips
   - Re-encode with consistent settings
   - Preserve quality (high bitrate, proper codec)

4. Audio Handling
   - Extract audio from clips (if present)
   - Crossfade audio at transitions
   - Sync to final video timeline

Expected output: Polished video with seamless transitions

---

Model Support (2026)

Image-to-Video Models

Transition Effects

---

ComfyUI Instance Management

Health Monitoring

# Auto-detected issues:
- Queue stalled (no progress for 5+ minutes)
- Memory leak (VRAM usage climbing)
- Process crashed (connection refused)
- API unresponsive (timeout on /queue endpoint)
- Disk full (output directory at capacity)

Auto-Recovery Actions

1. Soft Recovery (no restart)
   - Clear stuck queue items
   - Force garbage collection
   - Unload models from VRAM

2. Hard Recovery (restart required)
   - Save current queue state
   - Kill ComfyUI process gracefully
   - Wait for port release
   - Restart with same config
   - Restore queue from saved state

3. Emergency Fallback
   - Switch to backup ComfyUI instance
   - Redirect queue to instance on different port
   - Continue processing without data loss

Multi-Instance Support

# Run multiple ComfyUI instances for parallel processing
Instance 1: localhost:8188 (primary - I2V generation)
Instance 2: localhost:8189 (secondary - upscaling/post-processing)
Instance 3: localhost:8190 (backup - standby for failover)

# Load balancing strategy:
- Round-robin for equal workloads
- Priority-based for mixed tasks
- Failover for crashed instances

---

Validation Suite

Pre-Generation Validation

✓ Check ComfyUI is running and responsive
✓ Verify models are loaded (UNET, VAE, CLIP)
✓ Confirm output directory has sufficient space
✓ Validate source images exist and are readable
✓ Check prompts are non-empty and formatted correctly
✓ Verify workflow JSON is valid

Post-Generation Validation

✓ Video file exists and is non-zero size
✓ Resolution matches expected (e.g., 768x1024)
✓ FPS matches expected (e.g., 16 or 25)
✓ Duration matches expected (e.g., 3-5 seconds)
✓ Codec is compatible (h264, h265)
✓ No corruption (can read all frames)
✓ Face consistency score >0.85 (if character video)
✓ Color histogram within expected range

Quality Metrics

Metrics tracked:
- Face embedding distance (identity consistency)
- Optical flow magnitude (motion smoothness)
- Frame PSNR/SSIM (interpolation quality)
- Color histogram deviation (lighting consistency)
- Audio sync offset (if audio present)

---

Error Handling & Recovery

Retry Strategies

1. Seed Randomization Retry
   - Failed generation? Try different seed
   - Max 3 attempts per keyframe
   - Track seeds that fail (avoid reuse)

2. Parameter Adjustment Retry
   - CFG too high causing artifacts? Lower it
   - Steps too low causing incompleteness? Increase
   - Resolution too high OOM? Downscale

3. Model Fallback Retry
   - Wan 2.2 14B OOM? Fall back to 1.3B
   - LTX-2 unavailable? Fall back to Wan 2.1
   - AnimateDiff motion broken? Switch motion LoRA

4. Checkpoint Resume
   - Save progress after each successful clip
   - Resume from last successful checkpoint
   - Skip already-generated clips

Failure Logging

logs/
├── 2026-02-16_pipeline.log       # Main pipeline log
├── 2026-02-16_comfyui.log        # ComfyUI stdout/stderr
├── 2026-02-16_validation.json    # Validation results
├── 2026-02-16_failures.json      # Failed attempts with reasons
└── 2026-02-16_recovery.json      # Recovery actions taken

---

Directory Structure

Organized Output

project_name/
├── 00_keyframes/                 # Source keyframe images
│   ├── kf01_scene_description.png
│   ├── kf02_scene_description.png
│   └── ...
├── 01_clips/                     # Individual animated clips
│   ├── clip_001_kf01.mp4
│   ├── clip_002_kf02.mp4
│   └── ...
├── 02_validated/                 # Clips that passed validation
│   ├── clip_001_kf01.mp4
│   ├── clip_002_kf02.mp4
│   └── ...
├── 03_transitions/               # Intermediate files for transitions
│   ├── transition_001_002.mp4
│   └── ...
├── 04_final/                     # Final combined video
│   ├── final_video_v1.mp4
│   ├── final_video_v2.mp4        # After revisions
│   └── ...
├── logs/                         # Execution logs
├── metadata/                     # JSON metadata for each asset
└── manifest.json                 # Complete project manifest

---

Workflow Examples

Example 1: 30-Second Narrative Video (5 keyframes)

# Configuration
project_name = "sage_character_reveal"
keyframes = 5
i2v_model = "wan_2.2_moe"
target_duration = 30  # seconds
fps = 16

# Pipeline execution
1. Generate 5 keyframes (IP-Adapter + LoRA)
   → sage_kf01_over_shoulder.png
   → sage_kf02_turning.png
   → sage_kf03_cardigan_fallen.png
   → sage_kf04_removing_bra.png
   → sage_kf05_topless.png

2. Validate keyframes
   → Face consistency: 0.92 ✓
   → Lighting consistency: 0.88 ✓
   → Pose progression: logical ✓

3. Queue I2V for each keyframe
   → clip_001: 6s @ 16fps (96 frames) ✓
   → clip_002: 6s @ 16fps (96 frames) ✓
   → clip_003: 6s @ 16fps (96 frames) ✓
   → clip_004: 6s @ 16fps (96 frames) ✓
   → clip_005: 6s @ 16fps (96 frames) ✓

4. Apply 0.5s crossfade transitions
   → Total: 30s - 2s (4 transitions × 0.5s) = 28s net

5. Export final video
   → sage_character_reveal_final.mp4 (30s, 768x1024, 16fps)

Example 2: Batch Process 20 Images

# Configuration
input_dir = "E:/ComfyUI/input/character_expressions"
i2v_model = "ltx_2"
motion_prompt = "gentle breathing, subtle movement, natural"
batch_size = 4  # Process 4 at a time

# Pipeline execution
1. Scan input directory
   → Found 20 PNG files

2. Queue 4 at a time to ComfyUI
   → Batch 1: expr_001.png → expr_004.png ✓
   → Batch 2: expr_005.png → expr_008.png ✓
   → Batch 3: expr_009.png → expr_012.png ✓
   → Batch 4: expr_013.png → expr_016.png ✓
   → Batch 5: expr_017.png → expr_020.png ✓

3. Validate each output
   → 19/20 passed (expr_011 failed - wrong resolution)
   → Retry expr_011 with corrected settings ✓

4. Export batch
   → 20 validated clips in output/expressions/
   → Generated contact sheet: expressions_preview.png

---

Reference Files

Detailed Guides

• references/keyframe-generation.md - Keyframe creation with IP-Adapter, LoRA, consistency tips
• references/i2v-workflows.md - Wan 2.2, LTX-2, AnimateDiff, SVD workflow templates
• references/concatenation.md - FFmpeg commands, transition effects, audio handling
• references/validation.md - Quality metrics, validation thresholds, troubleshooting
• references/instance-management.md - ComfyUI health checks, restart scripts, multi-instance setup
• references/api-reference.md - ComfyUI API endpoints, queue management, workflow submission
• references/troubleshooting.md - Common issues and solutions

---

Integration with Other Skills

Pair with:

• comfyui-character-gen - For generating initial keyframes with identity preservation
• video-assembly - For advanced editing and post-production
• youtube-uploader - For direct upload to YouTube after production

---

Advanced Features

Adaptive Quality

# Automatically adjust settings based on available resources
if vram_available > 24:
    use_model = "wan_2.2_moe_14b"
    resolution = (832, 1216)
    batch_size = 1
elif vram_available > 12:
    use_model = "wan_2.1_1.3b"
    resolution = (768, 1024)
    batch_size = 2
else:
    use_model = "animatediff_v3"
    resolution = (512, 768)
    batch_size = 4

Progress Tracking

# Real-time progress updates
[Pipeline] Keyframe generation: 3/5 complete (60%)
[Pipeline] ETA: 12 minutes remaining
[I2V] clip_003 generating: 47/96 frames (49%)
[I2V] Current speed: 0.42 it/s
[Validation] clip_001: PASS ✓
[Validation] clip_002: PASS ✓

Rollback & Versioning

# Automatically version outputs
output/
├── final_video_v1.mp4        # Initial render
├── final_video_v2.mp4        # After fixing clip_003
├── final_video_v3.mp4        # After adding transitions
└── final_video_final.mp4     # Approved version

# Rollback to previous version
rollback_to_version(2)  # Restore v2 as current

---

Workflow Generation

When asked to create a video production workflow:

1. Assess Requirements

• Number of shots/keyframes
• Target duration per shot
• I2V model preference
• Transition style
• Quality vs speed tradeoff

2. Generate Pipeline Config

• Model selection based on VRAM/quality needs
• Resolution and FPS settings
• Validation thresholds
• Retry policies

3. Provide Execution Scripts

• Python scripts for API submission
• FFmpeg commands for concatenation
• Validation checks
• Recovery procedures

4. Monitor & Adapt

• Track progress in real-time
• Detect failures early
• Apply recovery strategies
• Report final metrics

---

Best Practices

For Keyframe Videos

• Use same seed across all keyframes (consistency)
• IP-Adapter weight 0.75-0.85 (strong but not rigid)
• Validate keyframes before I2V (saves compute)
• Keep clips 4-8 seconds each (sweet spot)
• Use 0.5-1s crossfade transitions (smooth but not slow)

For Batch Processing

• Process in small batches (4-8 at a time)
• Validate immediately after each batch
• Save checkpoint after each successful batch
• Use priority queue for important clips

For Instance Management

• Monitor queue depth every 30s
• Restart if no progress for 5 minutes
• Keep backup instance ready on different port
• Log all restart events for debugging

---

Performance Optimization

RTX 50 Series (2026)

# ComfyUI launch flags for optimal performance
--highvram \
--fp8_e4m3fn-unet \
--reserve-vram 7 \
--use-pytorch-cross-attention

# Expected performance:
- Wan 2.2 14B: ~2-3 min per 5s clip (832x1216)
- LTX-2 4K: ~4-5 min per 5s clip (1920x1080)
- Wan 2.1 1.3B: ~1-2 min per 5s clip (768x1024)

AMD GPUs (ROCm)

# ComfyUI v0.8.1+ has native ROCm support
# No special flags needed, just install ROCm drivers

---

Skill Evolution

This skill adapts to new I2V models and techniques. When new models release: 1. Add model specs to references/i2v-workflows.md 2. Create workflow template for new model 3. Update model selection logic in main pipeline 4. Test with sample project 5. Document performance characteristics

See references/evolution.md for update protocol.

comfyui-video-production — Eval Configuration

Classification

• Type: Capability Uplift
• Category: End-to-end video pipeline orchestration with validation gates and error recovery

What "Good" Looks Like

1. Correct pipeline selection for the task (img2vid, txt2vid, vid2vid, multi-shot) with appropriate model choices 2. Step ordering follows the validation-gated pattern: generate → validate → animate → validate → concat 3. Error recovery paths are defined: seed randomization → parameter adjustment → model fallback 4. Model fallback chains are correct (e.g., AnimateDiff → SVD → Wan if primary fails) 5. Resource management accounts for VRAM constraints across pipeline stages (unload between heavy steps)

Known Limitations

• Cannot predict actual generation quality or detect visual artifacts programmatically
• Video model landscape changes rapidly; fallback chains may need updating
• Concatenation quality depends on consistency between shots, which is hard to guarantee

Benchmark Strategy

• Without skill: Base Claude suggests a linear pipeline without validation gates, no error recovery, and ignores VRAM management between stages
• With skill: Produces validation-gated pipelines with retry strategies, model fallbacks, and VRAM-aware step ordering
• Key differentiator: Validation gates between pipeline stages and structured error recovery — the difference between a pipeline that fails silently and one that catches and recovers from errors

Security — Eval Sandboxing

Eval runs use real tool access and may expose secrets in output. Results are gitignored. Use --allowedTools "Read,Glob,Grep" to prevent modification during eval runs.

Running Evals

bash eval/run-eval.sh                  # Full run (with-skill + baseline)
bash eval/run-eval.sh --skill-only     # With-skill only
bash eval/run-eval.sh --case TC-001    # Single test case

Retirement Signal

When base Claude consistently produces video pipelines with inter-stage validation, structured retry strategies (seed → params → model fallback), and VRAM-aware step ordering without needing the skill's pipeline templates.

ComfyUI API Reference

Complete guide to ComfyUI's REST API for programmatic workflow submission and monitoring.

---

Base URL

http://127.0.0.1:8188

For multiple instances:

• Primary: http://127.0.0.1:8188
• Secondary: http://127.0.0.1:8189
• Backup: http://127.0.0.1:8190

---

Core Endpoints

GET /queue

Get current queue status

Response:

{
  "queue_running": [
    ["prompt_id", 3, {"prompt": {...}, "extra_data": {...}}]
  ],
  "queue_pending": [
    ["prompt_id", 2, {"prompt": {...}, "extra_data": {...}}]
  ]
}

POST /prompt

Submit new workflow to queue

Request:

{
  "prompt": {
    "1": {
      "class_type": "LoadImage",
      "inputs": {"image": "keyframe_01.png"}
    },
    "2": {
      "class_type": "UNETLoader",
      "inputs": {"unet_name": "wan2.2_14B.safetensors"}
    }
    // ... more nodes
  },
  "client_id": "optional_client_id"
}

Response:

{
  "prompt_id": "abc123-def456-ghi789",
  "number": 42,
  "node_errors": {}
}

GET /history

Get execution history

Response:

{
  "prompt_id_1": {
    "prompt": {...},
    "outputs": {
      "14": {
        "images": [
          {
            "filename": "Sage_NSFW_Video_00001.mp4",
            "subfolder": "",
            "type": "output"
          }
        ]
      }
    },
    "status": {
      "status_str": "success",
      "completed": true,
      "messages": []
    }
  }
}

GET /system_stats

Get system information

Response:

{
  "system": {
    "os": "nt",
    "python_version": "3.11.5",
    "embedded_python": false
  },
  "devices": [
    {
      "name": "NVIDIA GeForce RTX 5090",
      "type": "cuda",
      "index": 0,
      "vram_total": 34359738368,
      "vram_free": 12884901888
    }
  ]
}

POST /free

Unload models and free VRAM

Request:

{
  "unload_models": true,
  "free_memory": true
}

POST /interrupt

Interrupt current execution

Request: (empty or {})

---

Python API Client

Basic Client

import requests
import json
import time

class ComfyUIClient:
    def __init__(self, url="http://127.0.0.1:8188"):
        self.url = url

    def submit_workflow(self, workflow):
        """Submit workflow and return prompt_id"""
        data = json.dumps({"prompt": workflow})
        response = requests.post(
            f"{self.url}/prompt",
            data=data,
            headers={'Content-Type': 'application/json'}
        )
        return response.json()

    def get_queue(self):
        """Get current queue status"""
        response = requests.get(f"{self.url}/queue")
        return response.json()

    def get_history(self, prompt_id=None):
        """Get execution history"""
        url = f"{self.url}/history"
        if prompt_id:
            url += f"/{prompt_id}"
        response = requests.get(url)
        return response.json()

    def wait_for_completion(self, prompt_id, timeout=600, poll_interval=5):
        """Wait for workflow to complete"""
        start = time.time()

        while time.time() - start < timeout:
            history = self.get_history(prompt_id)

            if prompt_id in history:
                status = history[prompt_id].get('status', {})
                if status.get('completed'):
                    return history[prompt_id]

            time.sleep(poll_interval)

        raise TimeoutError(f"Workflow {prompt_id} did not complete in {timeout}s")

    def get_output_files(self, prompt_id):
        """Extract output files from completed workflow"""
        history = self.get_history(prompt_id)

        if prompt_id not in history:
            return []

        outputs = history[prompt_id].get('outputs', {})
        files = []

        for node_id, node_output in outputs.items():
            if 'images' in node_output:
                for image in node_output['images']:
                    files.append({
                        'filename': image['filename'],
                        'subfolder': image.get('subfolder', ''),
                        'type': image['type']
                    })

        return files

    def interrupt(self):
        """Interrupt current execution"""
        response = requests.post(f"{self.url}/interrupt")
        return response.status_code == 200

    def free_memory(self):
        """Unload models and free VRAM"""
        response = requests.post(
            f"{self.url}/free",
            json={"unload_models": True, "free_memory": True}
        )
        return response.status_code == 200

Usage Example

# Initialize client
client = ComfyUIClient("http://127.0.0.1:8188")

# Define workflow
workflow = {
    "1": {
        "class_type": "LoadImage",
        "inputs": {"image": "keyframe_01.png"}
    },
    "2": {
        "class_type": "UNETLoader",
        "inputs": {"unet_name": "wan2.2_14B.safetensors"}
    }
    # ... more nodes
}

# Submit
result = client.submit_workflow(workflow)
prompt_id = result['prompt_id']

print(f"Submitted: {prompt_id}")

# Wait for completion
try:
    completed = client.wait_for_completion(prompt_id, timeout=300)
    print("✓ Completed successfully")

    # Get output files
    files = client.get_output_files(prompt_id)
    for f in files:
        print(f"Output: {f['filename']}")

except TimeoutError:
    print("✗ Workflow timed out")
    client.interrupt()

---

Batch Processing

Queue Multiple Workflows

def queue_batch(client, workflows, stagger_delay=2):
    """
    Queue multiple workflows with staggered submission

    Args:
        client: ComfyUIClient instance
        workflows: List of workflow dicts
        stagger_delay: Seconds to wait between submissions

    Returns:
        List of prompt_ids
    """
    prompt_ids = []

    for i, workflow in enumerate(workflows):
        print(f"[Batch] Submitting {i+1}/{len(workflows)}...")

        result = client.submit_workflow(workflow)
        prompt_id = result['prompt_id']
        prompt_ids.append(prompt_id)

        if i < len(workflows) - 1:  # Don't wait after last one
            time.sleep(stagger_delay)

    return prompt_ids

def wait_for_batch(client, prompt_ids, timeout=600):
    """
    Wait for all workflows in batch to complete

    Args:
        client: ComfyUIClient instance
        prompt_ids: List of prompt IDs
        timeout: Max wait time per workflow

    Returns:
        Dict of {prompt_id: result}
    """
    results = {}

    for prompt_id in prompt_ids:
        print(f"[Batch] Waiting for {prompt_id}...")

        try:
            result = client.wait_for_completion(prompt_id, timeout)
            results[prompt_id] = result
            print(f"[Batch] ✓ {prompt_id} completed")
        except TimeoutError:
            print(f"[Batch] ✗ {prompt_id} timed out")
            results[prompt_id] = None

    return results

# Example usage
keyframes = [
    "keyframe_01.png",
    "keyframe_02.png",
    "keyframe_03.png",
    "keyframe_04.png",
    "keyframe_05.png"
]

workflows = [
    create_i2v_workflow(kf, motion_prompt)
    for kf, motion_prompt in zip(keyframes, motion_prompts)
]

# Submit batch
client = ComfyUIClient()
prompt_ids = queue_batch(client, workflows, stagger_delay=3)

# Wait for all
results = wait_for_batch(client, prompt_ids, timeout=300)

# Check success rate
completed = sum(1 for r in results.values() if r is not None)
print(f"\n[Batch] {completed}/{len(results)} workflows completed")

---

Advanced: WebSocket Monitoring

For real-time progress updates, use WebSocket connection:

import websocket
import json
import threading

class ComfyUIWebSocketMonitor:
    def __init__(self, url="ws://127.0.0.1:8188/ws"):
        self.url = url
        self.ws = None
        self.callbacks = {}

    def on_message(self, ws, message):
        """Handle incoming messages"""
        data = json.loads(message)

        msg_type = data.get('type')
        if msg_type in self.callbacks:
            self.callbacks[msg_type](data)

    def on_error(self, ws, error):
        print(f"[WebSocket] Error: {error}")

    def on_close(self, ws, close_status_code, close_msg):
        print("[WebSocket] Connection closed")

    def register_callback(self, message_type, callback):
        """Register callback for message type"""
        self.callbacks[message_type] = callback

    def connect(self):
        """Connect to WebSocket"""
        self.ws = websocket.WebSocketApp(
            self.url,
            on_message=self.on_message,
            on_error=self.on_error,
            on_close=self.on_close
        )

        # Run in background thread
        thread = threading.Thread(target=self.ws.run_forever)
        thread.daemon = True
        thread.start()

    def disconnect(self):
        """Disconnect from WebSocket"""
        if self.ws:
            self.ws.close()

# Example usage
monitor = ComfyUIWebSocketMonitor()

def on_progress(data):
    """Called when progress updates received"""
    value = data.get('data', {}).get('value', 0)
    max_val = data.get('data', {}).get('max', 100)
    print(f"Progress: {value}/{max_val} ({value/max_val*100:.1f}%)")

def on_execution_start(data):
    prompt_id = data.get('data', {}).get('prompt_id')
    print(f"Execution started: {prompt_id}")

def on_execution_complete(data):
    prompt_id = data.get('data', {}).get('prompt_id')
    print(f"Execution completed: {prompt_id}")

monitor.register_callback('progress', on_progress)
monitor.register_callback('execution_start', on_execution_start)
monitor.register_callback('executed', on_execution_complete)

monitor.connect()

# Submit workflows...
# Real-time progress will be printed

# When done
monitor.disconnect()

---

Error Handling

Workflow Validation

def validate_workflow(workflow):
    """Basic workflow validation before submission"""
    errors = []

    # Check required structure
    if not isinstance(workflow, dict):
        errors.append("Workflow must be a dictionary")
        return errors

    # Check each node
    for node_id, node in workflow.items():
        if 'class_type' not in node:
            errors.append(f"Node {node_id}: Missing class_type")

        if 'inputs' not in node:
            errors.append(f"Node {node_id}: Missing inputs")

    return errors

# Example usage
errors = validate_workflow(workflow)
if errors:
    print("Workflow validation failed:")
    for error in errors:
        print(f"  - {error}")
else:
    # Submit workflow
    client.submit_workflow(workflow)

Retry Logic

def submit_with_retry(client, workflow, max_retries=3):
    """Submit workflow with automatic retry on failure"""
    for attempt in range(max_retries):
        try:
            result = client.submit_workflow(workflow)
            return result
        except Exception as e:
            print(f"Attempt {attempt+1} failed: {e}")
            if attempt < max_retries - 1:
                time.sleep(5)  # Wait before retry
            else:
                raise

# Example usage
try:
    result = submit_with_retry(client, workflow, max_retries=3)
    print(f"Submitted: {result['prompt_id']}")
except Exception as e:
    print(f"Failed after 3 attempts: {e}")

---

Best Practices

1. Always validate workflows before submission 2. Use timeouts when waiting for completion 3. Implement retry logic for transient failures 4. Monitor queue depth to avoid overload 5. Free memory between large batches 6. Use WebSocket for real-time progress tracking 7. Handle errors gracefully with fallback strategies 8. Stagger batch submissions to avoid overwhelming the server

---

Common Issues

Issue: Connection refused

Cause: ComfyUI not running or wrong port Fix: Check if ComfyUI is running on expected port

Issue: Workflow validation errors

Cause: Invalid node inputs or missing required fields Fix: Validate workflow structure before submission

Issue: Timeout waiting for completion

Cause: Workflow taking too long or stuck Fix: Increase timeout or use interrupt endpoint

Issue: Queue depth growing

Cause: Submitting faster than processing Fix: Implement backpressure - check queue before submitting

Issue: VRAM out of memory

Cause: Models not unloading between workflows Fix: Call /free endpoint periodically

Video Concatenation & Transitions

Professional video stitching with FFmpeg, including transition effects, audio handling, and quality preservation.

---

Basic Concatenation

Method 1: FFmpeg Concat Demuxer (Recommended)

Use when: All clips have same codec, resolution, FPS

# 1. Create concat list file
cat > concat_list.txt << EOF
file '/path/to/clip_001.mp4'
file '/path/to/clip_002.mp4'
file '/path/to/clip_003.mp4'
EOF

# 2. Concatenate (no re-encoding - fast!)
ffmpeg -f concat -safe 0 -i concat_list.txt -c copy output.mp4

Pros: Ultra-fast (no re-encoding), perfect quality preservation Cons: Requires identical encoding parameters

Method 2: Re-encode Concatenation

Use when: Clips have different codecs/settings, or adding transitions

ffmpeg -i clip_001.mp4 -i clip_002.mp4 -i clip_003.mp4 \
  -filter_complex "[0:v][1:v][2:v]concat=n=3:v=1:a=0[outv]" \
  -map "[outv]" \
  -c:v libx264 -preset slow -crf 18 \
  output.mp4

Pros: Handles any input, allows filtering Cons: Slower, potential quality loss

---

Transition Effects

Crossfade Transition

# 2 clips with 1-second crossfade
ffmpeg -i clip1.mp4 -i clip2.mp4 \
  -filter_complex \
  "[0:v][1:v]xfade=transition=fade:duration=1:offset=4[outv]" \
  -map "[outv]" \
  -c:v libx264 -crf 18 \
  output.mp4

Transition types for xfade:

• fade - Simple crossfade (most common)
• wipeleft - Wipe from right to left
• wiperight - Wipe from left to right
• wipeup - Wipe from bottom to top
• wipedown - Wipe from top to bottom
• slideleft - Slide transition to left
• slideright - Slide transition to right
• circlecrop - Circular reveal
• rectcrop - Rectangular reveal
• distance - Smooth morph
• fadeblack - Fade to black then in

Multiple Clips with Crossfades

#!/usr/bin/env python3
"""Generate FFmpeg command for N clips with crossfades"""

def generate_crossfade_command(clips, fade_duration=0.5, output="final.mp4"):
    """
    Generate FFmpeg command for multiple clips with crossfades

    Args:
        clips: List of video file paths
        fade_duration: Transition duration in seconds
        output: Output filename
    """
    n = len(clips)

    # Build input arguments
    inputs = " ".join([f"-i {clip}" for clip in clips])

    # Build filter complex
    filter_parts = []

    # Calculate offsets (each clip minus fade duration)
    # Assume each clip is same duration
    clip_duration = 5  # seconds (adjust as needed)
    offset = clip_duration - fade_duration

    current_label = "0:v"
    for i in range(1, n):
        next_input = f"{i}:v"
        current_offset = offset * i
        temp_label = f"v{i}"

        fade_filter = (
            f"[{current_label}][{next_input}]"
            f"xfade=transition=fade:"
            f"duration={fade_duration}:"
            f"offset={current_offset}[{temp_label}]"
        )

        filter_parts.append(fade_filter)
        current_label = temp_label

    filter_complex = ";".join(filter_parts)

    # Build final command
    cmd = (
        f"ffmpeg {inputs} "
        f'-filter_complex "{filter_complex}" '
        f"-map '[{current_label}]' "
        f"-c:v libx264 -preset slow -crf 18 "
        f"{output}"
    )

    return cmd

# Example usage
clips = [
    "clip_001.mp4",
    "clip_002.mp4",
    "clip_003.mp4",
    "clip_004.mp4",
    "clip_005.mp4"
]

command = generate_crossfade_command(clips, fade_duration=0.5)
print(command)

---

Advanced Transitions

Zoom Transition

# Zoom in from clip1 to clip2
ffmpeg -i clip1.mp4 -i clip2.mp4 \
  -filter_complex \
  "[0:v]scale=iw*1.2:ih*1.2,crop=iw/1.2:ih/1.2:(iw-ow)/2:(ih-oh)/2[v0]; \
   [v0][1:v]xfade=transition=fade:duration=1:offset=4[outv]" \
  -map "[outv]" \
  -c:v libx264 -crf 18 \
  output.mp4

Motion Blur Transition

# Add motion blur during transition
ffmpeg -i clip1.mp4 -i clip2.mp4 \
  -filter_complex \
  "[0:v]minterpolate=fps=60:mi_mode=mci[v0]; \
   [v0][1:v]xfade=transition=fade:duration=0.5:offset=4[outv]" \
  -map "[outv]" \
  -r 30 \
  -c:v libx264 -crf 18 \
  output.mp4

Fade to Black Transition

# Fade clip1 to black, then fade in clip2
ffmpeg -i clip1.mp4 -i clip2.mp4 \
  -filter_complex \
  "[0:v]fade=t=out:st=4:d=0.5[v0]; \
   [1:v]fade=t=in:st=0:d=0.5[v1]; \
   [v0][v1]concat=n=2:v=1:a=0[outv]" \
  -map "[outv]" \
  -c:v libx264 -crf 18 \
  output.mp4

---

Audio Handling

Concatenate with Audio

# Concat with both video and audio
ffmpeg -f concat -safe 0 -i concat_list.txt \
  -c:v copy -c:a copy \
  output.mp4

Crossfade Audio During Video Transitions

ffmpeg -i clip1.mp4 -i clip2.mp4 \
  -filter_complex \
  "[0:v][1:v]xfade=transition=fade:duration=1:offset=4[outv]; \
   [0:a][1:a]acrossfade=d=1[outa]" \
  -map "[outv]" -map "[outa]" \
  -c:v libx264 -crf 18 -c:a aac \
  output.mp4

Add Background Music

# Add music to silent video
ffmpeg -i video.mp4 -i music.mp3 \
  -c:v copy -c:a aac \
  -shortest \
  output.mp4

# Mix existing audio with music
ffmpeg -i video.mp4 -i music.mp3 \
  -filter_complex \
  "[0:a][1:a]amix=inputs=2:duration=first:dropout_transition=2[aout]" \
  -map 0:v -map "[aout]" \
  -c:v copy -c:a aac \
  output.mp4

---

Quality Preservation

High-Quality Encoding Settings

# h264 (most compatible)
ffmpeg -i input.mp4 \
  -c:v libx264 \
  -preset slow \          # slow = better compression
  -crf 18 \               # 18 = visually lossless
  -pix_fmt yuv420p \      # ensure compatibility
  -movflags +faststart \  # web streaming optimization
  output.mp4

# h265 (better compression, less compatible)
ffmpeg -i input.mp4 \
  -c:v libx265 \
  -preset slow \
  -crf 20 \               # 20 for h265 ≈ 18 for h264
  -pix_fmt yuv420p \
  output.mp4

Resolution & FPS Enforcement

# Force resolution and FPS
ffmpeg -i input.mp4 \
  -vf "scale=768:1024:force_original_aspect_ratio=decrease,pad=768:1024:(ow-iw)/2:(oh-ih)/2" \
  -r 25 \
  -c:v libx264 -crf 18 \
  output.mp4

---

Validation Before Concatenation

Pre-Flight Check Script

#!/usr/bin/env python3
"""Validate clips before concatenation"""

import subprocess
import json

def get_video_info(video_path):
    """Extract video metadata using ffprobe"""
    cmd = [
        'ffprobe', '-v', 'error',
        '-select_streams', 'v:0',
        '-show_entries', 'stream=width,height,r_frame_rate,codec_name,duration',
        '-of', 'json',
        video_path
    ]

    result = subprocess.run(cmd, capture_output=True, text=True)
    data = json.loads(result.stdout)

    stream = data['streams'][0]

    # Parse frame rate
    fps_str = stream['r_frame_rate']
    num, den = map(int, fps_str.split('/'))
    fps = num / den

    return {
        'width': stream['width'],
        'height': stream['height'],
        'fps': round(fps, 2),
        'codec': stream['codec_name'],
        'duration': float(stream.get('duration', 0))
    }

def validate_clips_for_concat(clips):
    """Check if clips can be safely concatenated"""
    issues = []
    info_list = []

    for clip in clips:
        try:
            info = get_video_info(clip)
            info_list.append(info)
        except Exception as e:
            issues.append(f"{clip}: Failed to read ({e})")
            return False, issues

    # Check consistency
    reference = info_list[0]

    for i, info in enumerate(info_list[1:], 1):
        if info['width'] != reference['width'] or info['height'] != reference['height']:
            issues.append(
                f"Clip {i}: Resolution mismatch "
                f"({info['width']}x{info['height']} vs {reference['width']}x{reference['height']})"
            )

        if abs(info['fps'] - reference['fps']) > 0.1:
            issues.append(
                f"Clip {i}: FPS mismatch "
                f"({info['fps']} vs {reference['fps']})"
            )

        if info['codec'] != reference['codec']:
            issues.append(
                f"Clip {i}: Codec mismatch "
                f"({info['codec']} vs {reference['codec']})"
            )

    if issues:
        return False, issues
    else:
        return True, info_list

# Example usage
clips = ['clip_001.mp4', 'clip_002.mp4', 'clip_003.mp4']
valid, result = validate_clips_for_concat(clips)

if valid:
    print("✓ All clips compatible for concatenation")
    print(f"Resolution: {result[0]['width']}x{result[0]['height']}")
    print(f"FPS: {result[0]['fps']}")
    print(f"Codec: {result[0]['codec']}")
else:
    print("✗ Clips incompatible:")
    for issue in result:
        print(f"  - {issue}")

---

Production Pipeline

Complete Concatenation Script

#!/usr/bin/env python3
"""Production-ready video concatenation with transitions"""

import subprocess
import os
import json
from pathlib import Path

class VideoConcatenator:
    def __init__(self, output_dir="output"):
        self.output_dir = Path(output_dir)
        self.output_dir.mkdir(exist_ok=True)

    def validate_clips(self, clips):
        """Validate all clips before processing"""
        print("[Validation] Checking clips...")

        valid, result = validate_clips_for_concat(clips)

        if not valid:
            print("[Validation] ✗ FAILED")
            for issue in result:
                print(f"  {issue}")
            return False

        print("[Validation] ✓ All clips compatible")
        return True

    def concatenate_simple(self, clips, output_name="combined.mp4"):
        """Simple concatenation without transitions (fast)"""
        concat_file = self.output_dir / "concat_list.txt"

        with open(concat_file, 'w') as f:
            for clip in clips:
                abs_path = Path(clip).resolve()
                f.write(f"file '{abs_path}'\n")

        output_path = self.output_dir / output_name

        cmd = [
            'ffmpeg', '-y',
            '-f', 'concat',
            '-safe', '0',
            '-i', str(concat_file),
            '-c', 'copy',
            str(output_path)
        ]

        print(f"[Concat] Combining {len(clips)} clips...")
        subprocess.run(cmd, check=True, capture_output=True)

        concat_file.unlink()  # Cleanup

        print(f"[Concat] ✓ Output: {output_path}")
        return output_path

    def concatenate_with_transitions(
        self,
        clips,
        fade_duration=0.5,
        output_name="combined_transitions.mp4"
    ):
        """Concatenation with crossfade transitions (slower)"""
        print(f"[Concat] Applying {fade_duration}s crossfades...")

        output_path = self.output_dir / output_name

        # Build FFmpeg command
        inputs = []
        for clip in clips:
            inputs.extend(['-i', clip])

        # Generate filter complex
        filter_parts = []
        current_label = "0:v"

        # Calculate clip duration (assume all clips same duration)
        duration_cmd = [
            'ffprobe', '-v', 'error',
            '-show_entries', 'format=duration',
            '-of', 'default=noprint_wrappers=1:nokey=1',
            clips[0]
        ]
        duration = float(subprocess.run(duration_cmd, capture_output=True, text=True).stdout.strip())
        offset = duration - fade_duration

        for i in range(1, len(clips)):
            next_input = f"{i}:v"
            current_offset = offset * i
            temp_label = f"v{i}"

            fade_filter = (
                f"[{current_label}][{next_input}]"
                f"xfade=transition=fade:duration={fade_duration}:offset={current_offset}[{temp_label}]"
            )

            filter_parts.append(fade_filter)
            current_label = temp_label

        filter_complex = ";".join(filter_parts)

        cmd = [
            'ffmpeg', '-y',
            *inputs,
            '-filter_complex', filter_complex,
            '-map', f'[{current_label}]',
            '-c:v', 'libx264',
            '-preset', 'slow',
            '-crf', '18',
            '-pix_fmt', 'yuv420p',
            '-movflags', '+faststart',
            str(output_path)
        ]

        subprocess.run(cmd, check=True, capture_output=True)

        print(f"[Concat] ✓ Output: {output_path}")
        return output_path

# Example usage
if __name__ == "__main__":
    clips = [
        "E:/ComfyUI/output/clip_001.mp4",
        "E:/ComfyUI/output/clip_002.mp4",
        "E:/ComfyUI/output/clip_003.mp4"
    ]

    concatenator = VideoConcatenator(output_dir="E:/ComfyUI/output/final")

    if concatenator.validate_clips(clips):
        # Method 1: Fast, no transitions
        # concatenator.concatenate_simple(clips, "sage_video_final.mp4")

        # Method 2: With transitions
        concatenator.concatenate_with_transitions(
            clips,
            fade_duration=0.5,
            output_name="sage_video_final.mp4"
        )

---

Troubleshooting

Issue: "Non-monotonous DTS in output stream"

Fix: Re-encode instead of using -c copy

Issue: Audio/video out of sync

Fix: Use -async 1 or -vsync 2

Issue: Transitions look jerky

Fix: Increase transition duration or use motion blur

Issue: File size too large

Fix: Lower CRF value (higher compression) or reduce resolution

Issue: Clips have different resolutions

Fix: Scale all to same resolution before concatenation:

ffmpeg -i input.mp4 -vf scale=768:1024 -c:v libx264 -crf 18 output.mp4

Skill Evolution & Changelog

Track updates, new techniques, and user-specific learnings for the ComfyUI Video Production skill.

---

Changelog

v1.0.0 (February 2026 - Initial Release)

Core Features:

• Complete keyframe-to-video pipeline
• Batch I2V processing system
• Professional video concatenation with transitions
• ComfyUI instance management & health monitoring
• Multi-instance load balancing
• Auto-restart & recovery procedures
• Validation suite for pre/post generation
• FFmpeg integration for transitions

Supported Models:

• LTX-2 (4K production video)
• Wan 2.2 MoE (film-quality)
• Wan 2.1 14B & 1.3B
• AnimateDiff V3
• SVD (Stable Video Diffusion)

Documentation:

• Main SKILL.md with 3 core pipelines
• instance-management.md - ComfyUI health & restart
• concatenation.md - FFmpeg transitions & audio
• api-reference.md - ComfyUI REST API guide
• evolution.md - This file

Key Capabilities:

• Automatic error recovery with retry strategies
• Progress tracking with real-time ETA
• Quality validation (resolution, FPS, codec)
• Rollback & versioning support
• RTX 50 Series optimizations (NVFP4/NVFP8)
• AMD ROCm support (native in ComfyUI v0.8.1+)

---

Monitoring Sources

I2V Model Releases (Check Weekly)

HuggingFace Trending

• https://huggingface.co/models?sort=trending&pipeline_tag=video-generation
• Watch for: New video diffusion models, improved versions of Wan/LTX

GitHub Repositories

• https://github.com/Lightricks/LTX-Video - LTX updates
• https://github.com/alibaba/VideoX - Wan updates
• https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved - AnimateDiff
• https://github.com/comfyanonymous/ComfyUI - Core ComfyUI updates

Research Papers

• https://arxiv.org/list/cs.CV/recent - Computer vision (video gen)
• https://huggingface.co/papers - Daily ML papers
• Search: "image to video", "video diffusion", "temporal consistency"

ComfyUI Updates (Check Weekly)

Official Releases

• https://github.com/comfyanonymous/ComfyUI/releases
• Watch for: API changes, performance improvements, new features

Custom Nodes

• https://github.com/ltdrdata/ComfyUI-Manager - Manager updates
• https://github.com/Fannovel16/ComfyUI-Frame-Interpolation - RIFE/FILM
• https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite - Video tools

Community Knowledge (Check Monthly)

• r/comfyui - Reddit community
• r/StableDiffusion - General AI video discussions
• ComfyUI Discord - Real-time community help
• Civitai forums - Model/workflow discussions

---

Update Protocol

When New I2V Model Releases

1. Evaluate Relevance

• Does it improve on current recommendations?
• What's the quality vs performance tradeoff?
• VRAM requirements feasible for target users?

2. Test Compatibility

• Can it be loaded in ComfyUI?
• Required custom nodes available?
• Works with existing workflow patterns?

3. Benchmark Performance

• Test on same keyframes as existing models
• Measure: quality, speed, VRAM usage, consistency
• Compare to Wan 2.2, LTX-2, AnimateDiff baselines

4. Update Documentation

• Add to model comparison table in SKILL.md
• Create workflow template if significantly different
• Document special settings or gotchas
• Update references/i2v-workflows.md (when created)

5. Log Change

• Add entry to changelog below

When ComfyUI API Changes

1. Review Breaking Changes

• Check API reference doc for affected endpoints
• Test existing code against new version

2. Update Code Examples

• Modify api-reference.md examples
• Update Python client code
• Test all example scripts

3. Document Migration Path

• If breaking: provide migration guide
• Note version requirements

When User Reports Issue

1. Reproduce & Document

• What workflow/settings triggered it?
• Can it be consistently reproduced?
• Error messages or failure mode?

2. Research Solution

• Check ComfyUI issues on GitHub
• Search community forums/Discord
• Test potential fixes

3. Update Documentation

• Add to troubleshooting section if common issue
• Update relevant reference guide
• Add validation check to prevent if possible

4. Log Change

• Note issue and fix in changelog

---

User-Specific Learnings

Track what works for specific user setups and projects.

Hardware Profile

User: MCKRUZ GPU: NVIDIA RTX 5090 (32GB VRAM) OS: Windows 11 Pro ComfyUI: v0.8.1+

Capabilities:

• Can run Wan 2.2 14B natively
• LTX-2 4K generation without optimization
• Batch multiple workflows simultaneously
• Use NVFP4/NVFP8 precision for 3x speedup

Optimal Settings:

# ComfyUI launch flags
--highvram \
--fp8_e4m3fn-unet \
--reserve-vram 7

Project: Sage Character Video Series

Goal: Multi-shot narrative videos (30+ seconds) Character: Sage (reddish-brown hair, green eyes, fair skin) Style: Photorealistic, natural lighting, intimate/sensual

Workflow Preferences:

• Keyframe generation: FLUX.1-dev + IP-Adapter (0.80 weight)
• I2V model: Wan 2.2 MoE 14B (film-quality aesthetics)
• Resolution: 768x1024 (portrait) or 832x1216 (larger)
• FPS: 16 (matches Wan 2.2 output)
• Transitions: 0.5s crossfade (smooth but not slow)

What Worked:

• ✓ Generating 5 keyframes first, validating consistency before I2V
• ✓ Using consistent motion prompts with "consistent identity, same person" prefix
• ✓ Staggering I2V submissions (3-5 minute wait between clips)
• ✓ Validating each clip immediately after generation

What Didn't Work:

• ✗ Submitting all 5 I2V jobs at once (overwhelmed queue)
• ✗ Using cv2.VideoWriter for concatenation (quality loss, wrong codec)
• ✗ Not validating FPS before concat (got 25fps instead of 16fps)
• ✗ Hard-coded wait times (some clips took longer, some faster)
• ✗ Manual file selection for concatenation (forgot clips 1 & 4)

Lessons Learned: 1. Always validate before proceeding - Check keyframes, then check videos 2. Use FFmpeg, not cv2 - Better quality, proper codec support 3. Automate everything - Manual steps = mistakes 4. Implement retry logic - Some generations fail randomly 5. Monitor ComfyUI health - Can stall without errors

---

Feature Roadmap

v1.1.0 (Planned)

New Features:

• Keyframe generation reference guide
• I2V workflow templates for all supported models
• Validation reference with face consistency checking
• Troubleshooting guide with common issues

Improvements:

• Add frame interpolation between I2V clips
• Support for audio addition/mixing
• Color grading pipeline
• Upscaling integration

v1.2.0 (Future)

Advanced Features:

• Multi-GPU support for parallel generation
• Cloud instance integration (RunPod, Vast.ai)
• Voice synthesis integration (TTS Audio Suite)
• Lip-sync pipeline (Wav2Lip, SadTalker)

---

Research Tasks

Weekly

• [ ] Check HuggingFace for new video generation models
• [ ] Review ComfyUI releases for breaking changes
• [ ] Monitor community forums for common issues

Monthly

• [ ] Deep dive on any major new I2V model releases
• [ ] Review video generation research papers
• [ ] Test new custom nodes for video workflows
• [ ] Update benchmark comparisons

Quarterly

• [ ] Full skill audit - are recommendations still current?
• [ ] Remove deprecated models/techniques
• [ ] Major version bump if significant changes

---

Integration Opportunities

With Other Skills

comfyui-character-gen

• Use for generating consistent keyframes with LoRA/IP-Adapter
• Character consistency validation across frames
• Identity preservation techniques

youtube-uploader

• Direct upload to YouTube after production
• Metadata extraction for title/description
• Thumbnail generation from keyframes

video-assembly (if exists)

• Advanced editing and color grading
• Multi-track audio mixing
• Effects and overlays

---

Community Contributions

If this skill helps you, consider contributing:

1. Report Issues

• Document problems you encounter
• Include workflow, settings, error messages
• Suggest improvements

2. Share Discoveries

• New model settings that work well
• Workflow optimizations
• Troubleshooting solutions

3. Extend Documentation

• Add examples for your use case
• Create guides for specific techniques
• Document edge cases

4. Contribute Code

• Improve Python client
• Add new validation checks
• Optimize batch processing

---

Version History

---

Contact & Support

GitHub Repo: https://github.com/MCKRUZ/ComfyUI-Expert

Issues: Report problems or request features via GitHub Issues

Community: Join ComfyUI Discord for real-time help

ComfyUI Instance Management

Robust management of ComfyUI instances with health monitoring, auto-restart, and multi-instance orchestration.

---

Health Monitoring

Endpoint Checks

import requests
import time

def check_comfyui_health(url="http://127.0.0.1:8188"):
    """Check if ComfyUI is responsive"""
    endpoints = {
        "system_stats": f"{url}/system_stats",
        "queue": f"{url}/queue",
        "history": f"{url}/history",
        "prompt": f"{url}/prompt"
    }

    health = {}
    for name, endpoint in endpoints.items():
        try:
            response = requests.get(endpoint, timeout=5)
            health[name] = response.status_code == 200
        except:
            health[name] = False

    return all(health.values()), health

Queue Monitoring

def monitor_queue(url="http://127.0.0.1:8188", interval=30):
    """Monitor queue for stalls"""
    previous_queue_size = None
    stall_count = 0

    while True:
        try:
            response = requests.get(f"{url}/queue")
            data = response.json()

            current_queue = data.get("queue_running", [])
            queue_size = len(current_queue)

            # Check if queue is stalled
            if queue_size > 0 and queue_size == previous_queue_size:
                stall_count += 1
                if stall_count >= 10:  # 5 minutes at 30s intervals
                    return "STALLED"
            else:
                stall_count = 0

            previous_queue_size = queue_size
            time.sleep(interval)

        except Exception as e:
            return f"ERROR: {e}"

VRAM Monitoring

def check_vram_usage(threshold_gb=28):
    """Monitor VRAM usage (requires nvidia-smi)"""
    import subprocess

    try:
        result = subprocess.run(
            ['nvidia-smi', '--query-gpu=memory.used', '--format=csv,noheader,nounits'],
            capture_output=True,
            text=True
        )

        vram_used_mb = int(result.stdout.strip())
        vram_used_gb = vram_used_mb / 1024

        if vram_used_gb > threshold_gb:
            return "HIGH_VRAM", vram_used_gb

        return "OK", vram_used_gb

    except Exception as e:
        return "UNKNOWN", 0

---

Auto-Restart Procedures

Graceful Restart

import subprocess
import psutil
import time

def restart_comfyui(
    comfyui_path="E:/ComfyUI-Easy-Install/ComfyUI",
    port=8188,
    python_path="python"
):
    """Gracefully restart ComfyUI instance"""

    print("[Restart] Saving queue state...")
    save_queue_state(port)

    print("[Restart] Finding ComfyUI process...")
    killed = kill_comfyui_process(port)

    if killed:
        print("[Restart] Process terminated")
    else:
        print("[Restart] No process found (may have crashed)")

    print("[Restart] Waiting for port release...")
    wait_for_port_release(port, timeout=30)

    print("[Restart] Starting ComfyUI...")
    start_comfyui(comfyui_path, port, python_path)

    print("[Restart] Waiting for API to respond...")
    wait_for_api_ready(port, timeout=120)

    print("[Restart] Restoring queue state...")
    restore_queue_state(port)

    print("[Restart] ✓ ComfyUI restarted successfully")

Save/Restore Queue

import json

def save_queue_state(port=8188):
    """Save pending queue items before restart"""
    url = f"http://127.0.0.1:{port}/queue"

    try:
        response = requests.get(url)
        data = response.json()

        with open(f"queue_backup_{port}.json", "w") as f:
            json.dump(data, f, indent=2)

        return True
    except:
        return False

def restore_queue_state(port=8188):
    """Restore queue after restart"""
    backup_file = f"queue_backup_{port}.json"

    if not os.path.exists(backup_file):
        return False

    with open(backup_file, "r") as f:
        data = json.load(f)

    # Re-submit pending items
    for item in data.get("queue_pending", []):
        # Extract workflow from queue item
        workflow = item[2]  # Workflow is 3rd element in queue tuple
        submit_workflow(workflow, port)

    os.remove(backup_file)
    return True

Process Management

def kill_comfyui_process(port=8188):
    """Find and kill ComfyUI process by port"""
    for proc in psutil.process_iter(['pid', 'name', 'connections']):
        try:
            for conn in proc.connections():
                if conn.laddr.port == port:
                    proc.terminate()
                    proc.wait(timeout=10)
                    return True
        except:
            continue

    return False

def wait_for_port_release(port, timeout=30):
    """Wait until port is no longer in use"""
    start = time.time()

    while time.time() - start < timeout:
        in_use = False
        for proc in psutil.process_iter(['connections']):
            try:
                for conn in proc.connections():
                    if conn.laddr.port == port:
                        in_use = True
                        break
            except:
                continue
            if in_use:
                break

        if not in_use:
            return True

        time.sleep(1)

    return False

def start_comfyui(comfyui_path, port=8188, python_path="python"):
    """Start ComfyUI process"""
    cmd = [
        python_path,
        "main.py",
        "--port", str(port),
        "--highvram",
        "--fp8_e4m3fn-unet"
    ]

    subprocess.Popen(
        cmd,
        cwd=comfyui_path,
        stdout=subprocess.DEVNULL,
        stderr=subprocess.DEVNULL
    )

def wait_for_api_ready(port, timeout=120):
    """Wait for ComfyUI API to become responsive"""
    start = time.time()
    url = f"http://127.0.0.1:{port}/queue"

    while time.time() - start < timeout:
        try:
            response = requests.get(url, timeout=5)
            if response.status_code == 200:
                return True
        except:
            pass

        time.sleep(2)

    return False

---

Multi-Instance Management

Instance Configuration

INSTANCES = {
    "primary": {
        "port": 8188,
        "path": "E:/ComfyUI-Easy-Install/ComfyUI",
        "purpose": "I2V generation",
        "priority": 1
    },
    "secondary": {
        "port": 8189,
        "path": "E:/ComfyUI-Easy-Install/ComfyUI",
        "purpose": "upscaling/post-processing",
        "priority": 2
    },
    "backup": {
        "port": 8190,
        "path": "E:/ComfyUI-Easy-Install/ComfyUI",
        "purpose": "failover standby",
        "priority": 3
    }
}

Load Balancer

class ComfyUILoadBalancer:
    def __init__(self, instances):
        self.instances = instances
        self.round_robin_index = 0

    def get_healthiest_instance(self):
        """Return instance with lowest queue size"""
        queue_sizes = {}

        for name, config in self.instances.items():
            try:
                url = f"http://127.0.0.1:{config['port']}/queue"
                response = requests.get(url, timeout=5)
                data = response.json()
                queue_sizes[name] = len(data.get("queue_running", []))
            except:
                queue_sizes[name] = float('inf')  # Mark as unavailable

        return min(queue_sizes, key=queue_sizes.get)

    def get_next_round_robin(self):
        """Return next instance in round-robin"""
        instances_list = list(self.instances.keys())
        instance = instances_list[self.round_robin_index]
        self.round_robin_index = (self.round_robin_index + 1) % len(instances_list)
        return instance

    def submit_to_best_instance(self, workflow):
        """Submit workflow to best available instance"""
        instance_name = self.get_healthiest_instance()
        config = self.instances[instance_name]

        return submit_workflow(workflow, config['port'])

Failover Handler

def submit_with_failover(workflow, primary_port=8188, backup_ports=[8189, 8190]):
    """Try primary, fall back to backups if needed"""
    ports = [primary_port] + backup_ports

    for port in ports:
        try:
            result = submit_workflow(workflow, port)
            print(f"[Failover] Submitted to port {port}")
            return result
        except Exception as e:
            print(f"[Failover] Port {port} failed: {e}")
            continue

    raise Exception("All instances failed")

---

Startup Scripts

Windows (PowerShell)

# start-comfyui-multi.ps1
# Start multiple ComfyUI instances

$instances = @(
    @{ Port = 8188; Purpose = "Primary I2V" },
    @{ Port = 8189; Purpose = "Upscaling" },
    @{ Port = 8190; Purpose = "Backup" }
)

$comfyuiPath = "E:\ComfyUI-Easy-Install\ComfyUI"

foreach ($instance in $instances) {
    $port = $instance.Port
    $purpose = $instance.Purpose

    Write-Host "Starting ComfyUI on port $port ($purpose)..."

    Start-Process python -ArgumentList @(
        "main.py",
        "--port", $port,
        "--highvram",
        "--fp8_e4m3fn-unet"
    ) -WorkingDirectory $comfyuiPath -WindowStyle Hidden

    Start-Sleep -Seconds 5
}

Write-Host "All instances started!"
Write-Host "Primary: http://localhost:8188"
Write-Host "Secondary: http://localhost:8189"
Write-Host "Backup: http://localhost:8190"

Linux/Mac (Bash)

#!/bin/bash
# start-comfyui-multi.sh

COMFYUI_PATH="$HOME/ComfyUI"

start_instance() {
    local port=$1
    local purpose=$2

    echo "Starting ComfyUI on port $port ($purpose)..."

    cd "$COMFYUI_PATH"
    python main.py \
        --port $port \
        --highvram \
        --fp8_e4m3fn-unet \
        > "logs/comfyui_$port.log" 2>&1 &

    sleep 5
}

start_instance 8188 "Primary I2V"
start_instance 8189 "Upscaling"
start_instance 8190 "Backup"

echo "All instances started!"
echo "Primary: http://localhost:8188"
echo "Secondary: http://localhost:8189"
echo "Backup: http://localhost:8190"

---

Monitoring Dashboard (CLI)

#!/usr/bin/env python3
"""Real-time monitoring of ComfyUI instances"""

import requests
import time
import os
from datetime import datetime

def clear_screen():
    os.system('cls' if os.name == 'nt' else 'clear')

def get_instance_status(port):
    """Get status of ComfyUI instance"""
    url = f"http://127.0.0.1:{port}"

    try:
        # Check queue
        queue_response = requests.get(f"{url}/queue", timeout=2)
        queue_data = queue_response.json()

        running = len(queue_data.get("queue_running", []))
        pending = len(queue_data.get("queue_pending", []))

        # Check system stats
        stats_response = requests.get(f"{url}/system_stats", timeout=2)
        stats_data = stats_response.json()

        vram = stats_data.get("devices", [{}])[0].get("vram_used", 0) / 1024  # GB

        return {
            "status": "ONLINE",
            "running": running,
            "pending": pending,
            "vram_gb": round(vram, 2)
        }
    except:
        return {
            "status": "OFFLINE",
            "running": 0,
            "pending": 0,
            "vram_gb": 0
        }

def monitor_instances(instances):
    """Display real-time status of all instances"""
    while True:
        clear_screen()

        print("=" * 80)
        print(f"ComfyUI Instance Monitor - {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
        print("=" * 80)

        for name, config in instances.items():
            port = config["port"]
            purpose = config["purpose"]
            status = get_instance_status(port)

            status_color = "🟢" if status["status"] == "ONLINE" else "🔴"

            print(f"\n{status_color} {name.upper()} (Port {port}) - {purpose}")
            print(f"   Status: {status['status']}")
            print(f"   Queue: {status['running']} running, {status['pending']} pending")
            print(f"   VRAM: {status['vram_gb']} GB")

        print("\n" + "=" * 80)
        print("Press Ctrl+C to exit")

        time.sleep(5)

if __name__ == "__main__":
    INSTANCES = {
        "primary": {"port": 8188, "purpose": "I2V generation"},
        "secondary": {"port": 8189, "purpose": "Upscaling"},
        "backup": {"port": 8190, "purpose": "Standby"}
    }

    try:
        monitor_instances(INSTANCES)
    except KeyboardInterrupt:
        print("\n\nMonitoring stopped")

---

Recovery Procedures

Procedure 1: Soft Recovery (No Restart)

def soft_recovery(port=8188):
    """Attempt recovery without restarting"""
    url = f"http://127.0.0.1:{port}"

    # 1. Clear queue
    print("[Recovery] Clearing queue...")
    try:
        requests.post(f"{url}/queue", json={"clear": True})
    except:
        pass

    # 2. Free memory
    print("[Recovery] Requesting garbage collection...")
    try:
        requests.post(f"{url}/free", json={"unload_models": True})
    except:
        pass

    # 3. Wait and check
    time.sleep(10)
    is_healthy, _ = check_comfyui_health(url)

    return is_healthy

Procedure 2: Hard Recovery (Restart)

def hard_recovery(port=8188, comfyui_path="E:/ComfyUI-Easy-Install/ComfyUI"):
    """Full restart recovery"""
    print("[Recovery] Hard recovery initiated...")

    try:
        restart_comfyui(comfyui_path, port)
        return True
    except Exception as e:
        print(f"[Recovery] Hard recovery failed: {e}")
        return False

Procedure 3: Emergency Failover

def emergency_failover(failed_port, backup_ports=[8189, 8190]):
    """Switch to backup instance"""
    print(f"[Failover] Port {failed_port} has failed")

    for backup_port in backup_ports:
        is_healthy, _ = check_comfyui_health(f"http://127.0.0.1:{backup_port}")

        if is_healthy:
            print(f"[Failover] Switching to port {backup_port}")
            return backup_port

    print("[Failover] No backup instances available!")
    return None

---

Best Practices

1. Always monitor health - Check every 30-60 seconds 2. Save queue state before restart - Prevent work loss 3. Keep backup instance ready - Hot standby on different port 4. Log all restart events - For debugging patterns 5. Use graceful termination - Give ComfyUI time to save state 6. Test failover regularly - Don't wait for emergencies 7. Monitor VRAM usage - Memory leaks are common 8. Set restart thresholds - Auto-restart after N failures

---

Troubleshooting

ComfyUI won't start

• Check if port is already in use: netstat -ano | findstr :8188
• Verify Python environment is correct
• Check CUDA/ROCm drivers are installed
• Review startup logs for errors

Instance keeps crashing

• Check VRAM usage - may be OOM
• Review workflows for memory leaks
• Update to latest ComfyUI version
• Check for corrupted models

Queue stalls but process is running

• Workflow may have infinite loop
• Model download may be stuck
• Custom node may be hanging
• Try soft recovery first, then hard recovery

Failover not working

• Verify backup instance is actually running
• Check network/firewall settings
• Ensure backup has same models loaded
• Test failover mechanism regularly