Computer Vision Pipeline

Computer Vision Pipeline

Expert in building production-ready computer vision systems for object detection, tracking, and video analysis.

When to Use

✅ Use for:

• Drone footage analysis (archaeological surveys, conservation)
• Wildlife monitoring and tracking
• Real-time object detection systems
• Video preprocessing and analysis
• Custom model training and inference
• Multi-object tracking (MOT)

❌ NOT for:

• Simple image filters (use Pillow/PIL)
• Photo editing (use Photoshop/GIMP)
• Face recognition APIs (use AWS Rekognition)
• Basic OCR (use Tesseract)

---

Technology Selection

Object Detection Models

Timeline:

• 2015: Faster R-CNN (two-stage detection)
• 2016: YOLO v1 (one-stage, real-time)
• 2020: YOLOv5 (PyTorch, production-ready)
• 2023: YOLOv8 (state-of-the-art)
• 2024: YOLOv8 is industry standard for real-time

Decision tree:

Need real-time (>30 FPS)? → YOLOv8
Need highest accuracy? → Detectron2 Mask R-CNN
Need mobile deployment? → YOLOv8-nano or EfficientDet
Need instance segmentation? → Detectron2 or YOLOv8-seg
Need custom objects? → Fine-tune YOLOv8

---

Common Anti-Patterns

Anti-Pattern 1: Not Preprocessing Frames Before Detection

Novice thinking: "Just run detection on raw video frames"

Problem: Poor detection accuracy, wasted GPU cycles.

Wrong approach:

# ❌ No preprocessing - poor results
import cv2
from ultralytics import YOLO

model = YOLO('yolov8n.pt')
video = cv2.VideoCapture('drone_footage.mp4')

while True:
    ret, frame = video.read()
    if not ret:
        break

    # Raw frame detection - no normalization, no resizing
    results = model(frame)
    # Poor accuracy, slow inference

Why wrong:

• Video resolution too high (4K = 8.3 megapixels per frame)
• No normalization (pixel values 0-255 instead of 0-1)
• Aspect ratio not maintained
• GPU memory overflow on high-res frames

Correct approach:

# ✅ Proper preprocessing pipeline
import cv2
import numpy as np
from ultralytics import YOLO

model = YOLO('yolov8n.pt')
video = cv2.VideoCapture('drone_footage.mp4')

# Model expects 640x640 input
TARGET_SIZE = 640

def preprocess_frame(frame):
    # Resize while maintaining aspect ratio
    h, w = frame.shape[:2]
    scale = TARGET_SIZE / max(h, w)
    new_w, new_h = int(w * scale), int(h * scale)

    resized = cv2.resize(frame, (new_w, new_h), interpolation=cv2.INTER_LINEAR)

    # Pad to square
    pad_w = (TARGET_SIZE - new_w) // 2
    pad_h = (TARGET_SIZE - new_h) // 2

    padded = cv2.copyMakeBorder(
        resized,
        pad_h, TARGET_SIZE - new_h - pad_h,
        pad_w, TARGET_SIZE - new_w - pad_w,
        cv2.BORDER_CONSTANT,
        value=(114, 114, 114)  # Gray padding
    )

    # Normalize to 0-1 (if model expects it)
    # normalized = padded.astype(np.float32) / 255.0

    return padded, scale

while True:
    ret, frame = video.read()
    if not ret:
        break

    preprocessed, scale = preprocess_frame(frame)
    results = model(preprocessed)

    # Scale bounding boxes back to original coordinates
    for box in results[0].boxes:
        x1, y1, x2, y2 = box.xyxy[0]
        x1, y1, x2, y2 = x1/scale, y1/scale, x2/scale, y2/scale

Performance comparison:

• Raw 4K frames: 5 FPS, 72% mAP
• Preprocessed 640x640: 45 FPS, 89% mAP

Timeline context:

• 2015: Manual preprocessing required
• 2020: YOLOv5 added auto-resize
• 2023: YOLOv8 has smart preprocessing but explicit control is better

---

Anti-Pattern 2: Processing Every Frame in Video

Novice thinking: "Run detection on every single frame"

Problem: 99% of frames are redundant, wasting compute.

Wrong approach:

# ❌ Process every frame (30 FPS video = 1800 frames/min)
import cv2
from ultralytics import YOLO

model = YOLO('yolov8n.pt')
video = cv2.VideoCapture('drone_footage.mp4')

detections = []

while True:
    ret, frame = video.read()
    if not ret:
        break

    # Run detection on EVERY frame
    results = model(frame)
    detections.append(results)

# 10-minute video = 18,000 inferences (15 minutes on GPU)

Why wrong:

• Adjacent frames are nearly identical
• Wasting 95% of compute on duplicate work
• Slow processing time
• Massive storage for results

Correct approach 1: Frame sampling

# ✅ Sample every Nth frame
import cv2
from ultralytics import YOLO

model = YOLO('yolov8n.pt')
video = cv2.VideoCapture('drone_footage.mp4')

SAMPLE_RATE = 30  # Process 1 frame per second (if 30 FPS video)

frame_count = 0
detections = []

while True:
    ret, frame = video.read()
    if not ret:
        break

    frame_count += 1

    # Only process every 30th frame
    if frame_count % SAMPLE_RATE == 0:
        results = model(frame)
        detections.append({
            'frame': frame_count,
            'timestamp': frame_count / 30.0,
            'results': results
        })

# 10-minute video = 600 inferences (30 seconds on GPU)

Correct approach 2: Adaptive sampling with scene change detection

# ✅ Only process when scene changes significantly
import cv2
import numpy as np
from ultralytics import YOLO

model = YOLO('yolov8n.pt')
video = cv2.VideoCapture('drone_footage.mp4')

def scene_changed(prev_frame, curr_frame, threshold=0.3):
    """Detect scene change using histogram comparison"""
    if prev_frame is None:
        return True

    # Convert to grayscale
    prev_gray = cv2.cvtColor(prev_frame, cv2.COLOR_BGR2GRAY)
    curr_gray = cv2.cvtColor(curr_frame, cv2.COLOR_BGR2GRAY)

    # Calculate histograms
    prev_hist = cv2.calcHist([prev_gray], [0], None, [256], [0, 256])
    curr_hist = cv2.calcHist([curr_gray], [0], None, [256], [0, 256])

    # Compare histograms
    correlation = cv2.compareHist(prev_hist, curr_hist, cv2.HISTCMP_CORREL)

    return correlation < (1 - threshold)

prev_frame = None
detections = []

while True:
    ret, frame = video.read()
    if not ret:
        break

    # Only run detection if scene changed
    if scene_changed(prev_frame, frame):
        results = model(frame)
        detections.append(results)

    prev_frame = frame.copy()

# Adapts to video content - static shots skip frames, action scenes process more

Savings:

• Every frame: 18,000 inferences
• Sample 1 FPS: 600 inferences (97% reduction)
• Adaptive: ~1,200 inferences (93% reduction)

---

Anti-Pattern 3: Not Using Batch Inference

Novice thinking: "Process one image at a time"

Problem: GPU sits idle 80% of the time waiting for data.

Wrong approach:

# ❌ Sequential processing - GPU underutilized
import cv2
from ultralytics import YOLO
import time

model = YOLO('yolov8n.pt')

# 100 images to process
image_paths = [f'frame_{i:04d}.jpg' for i in range(100)]

start = time.time()

for path in image_paths:
    frame = cv2.imread(path)
    results = model(frame)  # Process one at a time
    # GPU utilization: ~20%

elapsed = time.time() - start
print(f"Processed {len(image_paths)} images in {elapsed:.2f}s")
# Output: 45 seconds

Why wrong:

• GPU has to wait for CPU to load each image
• No parallelization
• GPU utilization ~20%
• Slow throughput

Correct approach:

# ✅ Batch inference - GPU fully utilized
import cv2
from ultralytics import YOLO
import time

model = YOLO('yolov8n.pt')

image_paths = [f'frame_{i:04d}.jpg' for i in range(100)]

BATCH_SIZE = 16  # Process 16 images at once

start = time.time()

for i in range(0, len(image_paths), BATCH_SIZE):
    batch_paths = image_paths[i:i+BATCH_SIZE]

    # Load batch
    frames = [cv2.imread(path) for path in batch_paths]

    # Batch inference (single GPU call)
    results = model(frames)  # Pass list of images
    # GPU utilization: ~85%

elapsed = time.time() - start
print(f"Processed {len(image_paths)} images in {elapsed:.2f}s")
# Output: 8 seconds (5.6x faster!)

Performance comparison:

Batch size tuning:

# Find optimal batch size for your GPU
import torch

def find_optimal_batch_size(model, image_size=(640, 640)):
    for batch_size in [1, 2, 4, 8, 16, 32, 64]:
        try:
            dummy_input = torch.randn(batch_size, 3, *image_size).cuda()

            start = time.time()
            with torch.no_grad():
                _ = model(dummy_input)
            elapsed = time.time() - start

            throughput = batch_size / elapsed
            print(f"Batch {batch_size}: {throughput:.1f} img/s")
        except RuntimeError as e:
            print(f"Batch {batch_size}: OOM (out of memory)")
            break

# Find optimal batch size before production
find_optimal_batch_size(model)

---

Anti-Pattern 4: Ignoring Non-Maximum Suppression (NMS) Tuning

Problem: Duplicate detections, missed objects, slow post-processing.

Wrong approach:

# ❌ Use default NMS settings for everything
from ultralytics import YOLO

model = YOLO('yolov8n.pt')

# Default settings (iou_threshold=0.45, conf_threshold=0.25)
results = model('crowded_scene.jpg')

# Result: 50 bounding boxes, 30 are duplicates!

Why wrong:

• Default IoU=0.45 is too permissive for dense objects
• Default conf=0.25 includes low-quality detections
• No adaptation to use case

Correct approach:

# ✅ Tune NMS for your use case
from ultralytics import YOLO

model = YOLO('yolov8n.pt')

# Sparse objects (dolphins in ocean)
sparse_results = model(
    'ocean_footage.jpg',
    iou=0.5,    # Higher IoU = allow closer boxes
    conf=0.4    # Higher confidence = fewer false positives
)

# Dense objects (crowd, flock of birds)
dense_results = model(
    'crowded_scene.jpg',
    iou=0.3,    # Lower IoU = suppress more duplicates
    conf=0.5    # Higher confidence = filter noise
)

# High precision needed (legal evidence)
precise_results = model(
    'evidence.jpg',
    iou=0.5,
    conf=0.7,   # Very high confidence
    max_det=50  # Limit max detections
)

NMS parameter guide:

---

Anti-Pattern 5: No Tracking Between Frames

Novice thinking: "Run detection on each frame independently"

Problem: Can't count unique objects, track movement, or build trajectories.

Wrong approach:

# ❌ Independent frame detection - no object identity
from ultralytics import YOLO
import cv2

model = YOLO('yolov8n.pt')
video = cv2.VideoCapture('dolphins.mp4')

detections = []

while True:
    ret, frame = video.read()
    if not ret:
        break

    results = model(frame)
    detections.append(results)

# Result: Can't tell if frame 10 dolphin is same as frame 20 dolphin
# Can't count unique dolphins
# Can't track trajectories

Why wrong:

• No object identity across frames
• Can't count unique objects
• Can't analyze movement patterns
• Can't build trajectories

Correct approach: Use tracking (ByteTrack)

# ✅ Multi-object tracking with ByteTrack
from ultralytics import YOLO
import cv2

# YOLO with tracking
model = YOLO('yolov8n.pt')
video = cv2.VideoCapture('dolphins.mp4')

# Track objects across frames
tracks = {}

while True:
    ret, frame = video.read()
    if not ret:
        break

    # Run detection + tracking
    results = model.track(
        frame,
        persist=True,     # Maintain IDs across frames
        tracker='bytetrack.yaml'  # ByteTrack algorithm
    )

    # Each detection now has persistent ID
    for box in results[0].boxes:
        track_id = int(box.id[0])  # Unique ID across frames
        x1, y1, x2, y2 = box.xyxy[0]

        # Store trajectory
        if track_id not in tracks:
            tracks[track_id] = []

        tracks[track_id].append({
            'frame': len(tracks[track_id]),
            'bbox': (x1, y1, x2, y2),
            'conf': box.conf[0]
        })

# Now we can analyze:
print(f"Unique dolphins detected: {len(tracks)}")

# Trajectory analysis
for track_id, trajectory in tracks.items():
    if len(trajectory) > 30:  # Only long tracks
        print(f"Dolphin {track_id} appeared in {len(trajectory)} frames")
        # Calculate movement, speed, etc.

Tracking benefits:

• Count unique objects (not just detections per frame)
• Build trajectories and movement patterns
• Analyze behavior over time
• Filter out brief false positives

Tracking algorithms:

---

Production Checklist

□ Preprocess frames (resize, pad, normalize)
□ Sample frames intelligently (1 FPS or scene change detection)
□ Use batch inference (16-32 images per batch)
□ Tune NMS thresholds for your use case
□ Implement tracking if analyzing video
□ Log inference time and GPU utilization
□ Handle edge cases (empty frames, corrupted video)
□ Save results in structured format (JSON, CSV)
□ Visualize detections for debugging
□ Benchmark on representative data

---

When to Use vs Avoid

---

References

• /references/yolo-guide.md - YOLOv8 setup, training, inference patterns
• /references/video-processing.md - Frame extraction, scene detection, optimization
• /references/tracking-algorithms.md - ByteTrack, SORT, DeepSORT comparison

Scripts

• scripts/video_analyzer.py - Extract frames, run detection, generate timeline
• scripts/model_trainer.py - Fine-tune YOLO on custom dataset, export weights

---

This skill guides: Computer vision | Object detection | Video analysis | YOLO | Tracking | Drone footage | Wildlife monitoring

Multi-Object Tracking Algorithms

Comprehensive guide to tracking algorithms for maintaining object identity across video frames.

---

Why Tracking Matters

Without Tracking:

Frame 1: Detected 3 dolphins
Frame 2: Detected 3 dolphins
Frame 3: Detected 2 dolphins

Question: Are these the same dolphins? Which one left?

With Tracking:

Frame 1: Dolphin #1, #2, #3
Frame 2: Dolphin #1, #2, #3
Frame 3: Dolphin #1, #3 (Dolphin #2 disappeared)

Answer: Dolphin #2 left the scene

---

Algorithm Comparison

Key Metrics:

• Speed: Frames per second (higher = faster)
• Robustness: Handling ID switches
• Occlusion: Tracking through overlaps
• Re-ID: Re-identifying after long absence

---

SORT (Simple Online and Realtime Tracking)

Algorithm Overview

How it works: 1. Detect objects in frame (YOLO, etc.) 2. Associate detections with existing tracks using IoU 3. Update tracks with Kalman filter 4. Remove tracks that haven't been seen for N frames

Strengths:

• Extremely fast (260 FPS)
• Simple to implement
• Works well for sparse scenes

Weaknesses:

• Many ID switches in crowded scenes
• Poor occlusion handling
• No appearance-based matching

---

SORT Implementation

from sort import Sort

# Initialize tracker
tracker = Sort(
    max_age=30,        # Max frames to keep track without detection
    min_hits=3,        # Min detections before track is confirmed
    iou_threshold=0.3  # IoU threshold for matching
)

# Process video
video = cv2.VideoCapture('video.mp4')
while True:
    ret, frame = video.read()
    if not ret:
        break

    # Run detector
    results = yolo_model(frame)

    # Convert to SORT format: [x1, y1, x2, y2, confidence]
    detections = []
    for box in results[0].boxes:
        x1, y1, x2, y2 = box.xyxy[0].cpu().numpy()
        conf = float(box.conf[0])
        detections.append([x1, y1, x2, y2, conf])

    # Update tracker
    tracks = tracker.update(np.array(detections))

    # tracks: [x1, y1, x2, y2, track_id]
    for track in tracks:
        x1, y1, x2, y2, track_id = track
        cv2.rectangle(frame, (int(x1), int(y1)), (int(x2), int(y2)), (0, 255, 0), 2)
        cv2.putText(frame, f'ID {int(track_id)}', (int(x1), int(y1)-10),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)

Installation:

pip install filterpy scikit-learn
git clone https://github.com/abewley/sort.git

---

DeepSORT (Deep Simple Online and Realtime Tracking)

Algorithm Overview

How it works: 1. Detect objects (YOLO) 2. Extract appearance features (deep CNN) 3. Associate using IoU + appearance similarity 4. Update with Kalman filter 5. Re-identify objects after long absence

Strengths:

• Excellent robustness in crowded scenes
• Re-identification after occlusion
• Appearance-based matching reduces ID switches

Weaknesses:

• Slower than SORT (40 FPS vs 260 FPS)
• Requires pre-trained re-identification model
• Higher computational cost

---

DeepSORT Implementation

from deep_sort_realtime.deepsort_tracker import DeepSort

# Initialize tracker
tracker = DeepSort(
    max_age=30,              # Max frames without detection
    n_init=3,                # Min detections before confirmed
    max_iou_distance=0.7,    # IoU threshold
    max_cosine_distance=0.3, # Appearance similarity threshold
    embedder="mobilenet",    # Feature extractor (mobilenet, resnet50, etc.)
    half=True,               # Use FP16 for speed
    bgr=True                 # Input is BGR (OpenCV default)
)

# Process video
video = cv2.VideoCapture('video.mp4')
while True:
    ret, frame = video.read()
    if not ret:
        break

    # Run detector
    results = yolo_model(frame)

    # Convert to DeepSORT format: ([x1, y1, w, h], confidence, class)
    detections = []
    for box in results[0].boxes:
        x1, y1, x2, y2 = box.xyxy[0].cpu().numpy()
        w, h = x2 - x1, y2 - y1
        conf = float(box.conf[0])
        cls = int(box.cls[0])

        detections.append(([x1, y1, w, h], conf, cls))

    # Update tracker
    tracks = tracker.update_tracks(detections, frame=frame)

    # Draw tracks
    for track in tracks:
        if not track.is_confirmed():
            continue

        track_id = track.track_id
        ltrb = track.to_ltrb()  # [left, top, right, bottom]

        cv2.rectangle(frame, (int(ltrb[0]), int(ltrb[1])),
                      (int(ltrb[2]), int(ltrb[3])), (0, 255, 0), 2)
        cv2.putText(frame, f'ID {track_id}', (int(ltrb[0]), int(ltrb[1])-10),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)

Installation:

pip install deep-sort-realtime

---

ByteTrack

Algorithm Overview

How it works: 1. Detect objects with YOLO 2. Separate detections into high-confidence and low-confidence 3. Match high-confidence detections first (like SORT) 4. Use low-confidence detections to recover occluded objects 5. Update with Kalman filter

Key Innovation: Uses low-confidence detections that other trackers ignore

Strengths:

• Fast (150 FPS) - 3.75x faster than DeepSORT
• Excellent occlusion handling
• No appearance model needed (no extra GPU memory)
• SOTA performance on MOT benchmarks

Weaknesses:

• No re-identification after long absence
• Requires tuning confidence thresholds

---

ByteTrack Implementation

from ultralytics import YOLO

# YOLOv8 has ByteTrack built-in!
model = YOLO('yolov8n.pt')

# Track with ByteTrack
results = model.track(
    'video.mp4',
    tracker='bytetrack.yaml',  # Use ByteTrack
    conf=0.3,                  # Detection confidence (low for ByteTrack)
    iou=0.5,                   # IoU threshold for NMS
    persist=True,              # Persist tracks across frames
    verbose=False
)

# Process results
for result in results:
    boxes = result.boxes

    if boxes is not None and boxes.id is not None:
        for box in boxes:
            x1, y1, x2, y2 = box.xyxy[0].cpu().numpy()
            track_id = int(box.id[0])
            conf = float(box.conf[0])
            cls = int(box.cls[0])

            print(f'Track {track_id}: {result.names[cls]} ({conf:.2f})')

Custom ByteTrack Config (bytetrack.yaml):

tracker_type: bytetrack
track_high_thresh: 0.5    # High confidence threshold
track_low_thresh: 0.1     # Low confidence threshold (key!)
new_track_thresh: 0.6     # Threshold for new track
track_buffer: 30          # Max frames without detection
match_thresh: 0.8         # Matching threshold

---

BotSORT

Algorithm Overview

How it works: 1. ByteTrack foundation (high + low confidence) 2. Add camera motion compensation 3. Add appearance-based re-identification 4. Use more sophisticated motion model

Strengths:

• Best overall accuracy (SOTA on MOT17/20)
• Excellent occlusion handling (from ByteTrack)
• Re-identification (from DeepSORT)
• Camera motion compensation (unique)

Weaknesses:

• Slower than ByteTrack (45 FPS)
• More complex to tune
• Requires appearance model

---

BotSORT Implementation

from ultralytics import YOLO

model = YOLO('yolov8n.pt')

# Track with BotSORT
results = model.track(
    'video.mp4',
    tracker='botsort.yaml',  # Use BotSORT
    conf=0.3,
    persist=True
)

# Process same as ByteTrack example above

Custom BotSORT Config (botsort.yaml):

tracker_type: botsort
track_high_thresh: 0.5
track_low_thresh: 0.1
new_track_thresh: 0.6
track_buffer: 30
match_thresh: 0.8
proximity_thresh: 0.5     # For appearance matching
appearance_thresh: 0.25   # Appearance similarity
cmc_method: sparseOptFlow # Camera motion compensation (sparseOptFlow, orb, ecc)

---

Performance Comparison

Speed Benchmarks

Tested on 1080p video, 30 FPS, 20 objects per frame (NVIDIA RTX 3080):

Key Insight: Tracking overhead is minimal for SORT/ByteTrack, significant for DeepSORT/BotSORT

---

Accuracy Benchmarks

MOT17 Dataset (Multiple Object Tracking benchmark):

Metrics:

• MOTA: Multi-Object Tracking Accuracy (higher = better)
• IDF1: ID F1 Score (higher = better, measures ID consistency)
• ID Switches: Number of times IDs change (lower = better)

Winner: BotSORT (best overall), ByteTrack (best speed/accuracy)

---

Use Case Recommendations

Wildlife Monitoring (Dolphins, Birds, etc.)

Best Choice: ByteTrack

Why:

• Animals move smoothly (Kalman filter works well)
• Occlusions are common (ByteTrack excels)
• No need for re-ID (animals don't leave and return)
• Speed allows real-time processing

Config:

tracker_type: bytetrack
track_high_thresh: 0.4    # Lower for animals (harder to detect)
track_low_thresh: 0.1
track_buffer: 60          # Longer buffer (animals move slower)
match_thresh: 0.7         # Lower threshold (animal appearance varies)

---

Crowded Indoor Scenes (Retail, Security)

Best Choice: BotSORT or DeepSORT

Why:

• Many occlusions (need appearance model)
• People leave and return (need re-ID)
• Camera is stationary (can use camera motion compensation)

Config (BotSORT):

tracker_type: botsort
track_high_thresh: 0.5
track_low_thresh: 0.1
track_buffer: 30
appearance_thresh: 0.25   # Strict appearance matching
cmc_method: sparseOptFlow # Compensate for camera jitter

---

Drone Footage (Archaeological Surveys, Inspection)

Best Choice: ByteTrack with custom config

Why:

• Camera moves (simpler motion model better)
• Objects may be small/low confidence
• Speed important for large footage volumes
• No re-ID needed (continuous tracking)

Config:

tracker_type: bytetrack
track_high_thresh: 0.3    # Very low (objects are small)
track_low_thresh: 0.05    # Extremely low (catch faint objects)
track_buffer: 90          # Long buffer (objects move slowly relative to camera)
match_thresh: 0.6         # Lenient matching (camera motion)

---

Sports Tracking (Soccer, Basketball)

Best Choice: BotSORT

Why:

• Fast, erratic motion
• Frequent occlusions (players overlap)
• Need re-ID (players leave/enter frame)
• Camera pans/zooms (camera motion compensation helps)

Config:

tracker_type: botsort
track_high_thresh: 0.5
track_low_thresh: 0.1
track_buffer: 20          # Short buffer (fast action)
appearance_thresh: 0.3
cmc_method: ecc           # Best for sports (handles zoom)

---

Real-Time Applications (Edge Devices, Webcams)

Best Choice: SORT

Why:

• Fastest option (260 FPS)
• Low memory footprint
• Good enough for simple scenes

Config:

tracker = Sort(
    max_age=15,       # Short buffer for real-time
    min_hits=2,       # Quick confirmation
    iou_threshold=0.3
)

---

Common Issues and Solutions

Issue 1: Too Many ID Switches

Symptoms: Same object gets new ID every few frames

Causes:

• Detection confidence too low
• Match threshold too high
• Track buffer too short

Solutions:

# Increase detection confidence
conf: 0.5  # Instead of 0.3

# Lower match threshold (more lenient)
match_thresh: 0.6  # Instead of 0.8

# Longer track buffer
track_buffer: 60  # Instead of 30

---

Issue 2: Lost Tracks During Occlusion

Symptoms: Object disappears behind another, gets new ID when reappearing

Cause: Tracker doesn't use low-confidence detections

Solutions: 1. Use ByteTrack or BotSORT (designed for occlusion) 2. Lower track_low_thresh:

   track_low_thresh: 0.05  # Catch low-confidence detections

3. Increase track_buffer:

   track_buffer: 90  # Keep track alive longer

---

Issue 3: Tracks Not Re-identified After Long Absence

Symptoms: Object leaves frame, returns later with new ID

Cause: SORT/ByteTrack don't support re-identification

Solution: Switch to DeepSORT or BotSORT

---

Issue 4: Slow Tracking Speed

Symptoms: Tracking overhead dominates inference time

Causes:

• Using appearance model (DeepSORT/BotSORT)
• Too many tracks
• Too many detections

Solutions: 1. Switch to ByteTrack or SORT 2. Increase detection confidence:

   conf: 0.5  # Fewer detections = faster tracking

3. Use FP16 for appearance model:

   tracker = DeepSort(half=True)  # 2x faster

---

Advanced Techniques

1. Multi-Camera Tracking

Track objects across multiple camera views:

from deep_sort_realtime.deepsort_tracker import DeepSort

# Shared appearance database
global_tracker = DeepSort(
    max_cosine_distance=0.2,  # Strict appearance matching
    embedder="resnet50"       # Better features for cross-camera
)

# Camera 1
tracks_cam1 = global_tracker.update_tracks(detections_cam1, frame_cam1)

# Camera 2 (shares appearance database with cam 1)
tracks_cam2 = global_tracker.update_tracks(detections_cam2, frame_cam2)

# Match tracks by appearance
for t1 in tracks_cam1:
    for t2 in tracks_cam2:
        if appearance_similarity(t1, t2) > 0.8:
            print(f"Same object: Cam1 ID {t1.track_id} = Cam2 ID {t2.track_id}")

---

2. Track Smoothing

Reduce jittery bounding boxes with moving average:

from collections import deque

class TrackSmoother:
    def __init__(self, window_size=5):
        self.tracks = {}  # track_id -> deque of boxes
        self.window_size = window_size

    def smooth(self, track_id, box):
        """Smooth bounding box with moving average"""
        if track_id not in self.tracks:
            self.tracks[track_id] = deque(maxlen=self.window_size)

        self.tracks[track_id].append(box)

        # Average boxes
        boxes = np.array(self.tracks[track_id])
        smoothed = boxes.mean(axis=0)

        return smoothed

# Usage
smoother = TrackSmoother(window_size=5)

for track in tracks:
    x1, y1, x2, y2, track_id = track
    smoothed_box = smoother.smooth(int(track_id), [x1, y1, x2, y2])

---

3. Track Validation

Filter out false positive tracks:

def validate_track(track_history, min_length=10, min_movement=50):
    """
    Validate track is real (not false positive)

    Args:
        track_history: List of (x, y) center points
        min_length: Minimum track length
        min_movement: Minimum total movement (pixels)

    Returns:
        True if valid track
    """
    if len(track_history) < min_length:
        return False

    # Calculate total movement
    total_movement = 0
    for i in range(1, len(track_history)):
        dx = track_history[i][0] - track_history[i-1][0]
        dy = track_history[i][1] - track_history[i-1][1]
        total_movement += np.sqrt(dx**2 + dy**2)

    return total_movement >= min_movement

# Usage
track_histories = {}  # track_id -> list of (x, y)

for track in tracks:
    x1, y1, x2, y2, track_id = track
    center_x, center_y = (x1 + x2) / 2, (y1 + y2) / 2

    if track_id not in track_histories:
        track_histories[track_id] = []

    track_histories[track_id].append((center_x, center_y))

    # Validate
    if validate_track(track_histories[track_id]):
        # Draw only valid tracks
        cv2.rectangle(frame, (int(x1), int(y1)), (int(x2), int(y2)), (0, 255, 0), 2)

---

Resources

• ByteTrack Paper
• DeepSORT Paper
• BotSORT Paper
• SORT Implementation
• DeepSORT Implementation
• YOLOv8 Tracking
• MOT Challenge - Tracking benchmarks

Video Processing for Computer Vision

Efficient video frame extraction, preprocessing, and scene detection for object detection pipelines.

---

Frame Extraction with FFmpeg

Basic Frame Extraction

# Extract every 30th frame (1 FPS for 30 FPS video)
ffmpeg -i video.mp4 -vf "select='not(mod(n\,30))'" -vsync vfr frames/frame_%06d.jpg

# Extract at specific FPS
ffmpeg -i video.mp4 -vf fps=1 frames/frame_%06d.jpg

# Extract with quality control
ffmpeg -i video.mp4 -vf fps=1 -q:v 2 frames/frame_%06d.jpg
# q:v range: 1 (best) to 31 (worst)

Resolution and Aspect Ratio

# Resize to 640x640 (for YOLO)
ffmpeg -i video.mp4 -vf "fps=1,scale=640:640:force_original_aspect_ratio=decrease,pad=640:640:(ow-iw)/2:(oh-ih)/2:color=gray" frames/frame_%06d.jpg

# Maintain aspect ratio with padding
ffmpeg -i video.mp4 -vf "fps=1,scale=640:-1" frames/frame_%06d.jpg

Explanation:

• scale=640:640:force_original_aspect_ratio=decrease - Shrink to fit within 640x640
• pad=640:640:(ow-iw)/2:(oh-ih)/2 - Center padding to make square
• color=gray - Gray padding (114,114,114 matches YOLO default)

---

Scene Change Detection

FFmpeg Scene Detection

# Extract keyframes only (scene changes)
ffmpeg -i video.mp4 -vf "select='gt(scene,0.3)',showinfo" -vsync vfr frames/scene_%06d.jpg

# Adjust sensitivity (0.0 = all frames, 1.0 = major changes only)
ffmpeg -i video.mp4 -vf "select='gt(scene,0.4)'" -vsync vfr frames/scene_%06d.jpg

Scene threshold guide:

• 0.1 - Very sensitive (every small change)
• 0.3 - Moderate (good for drone footage)
• 0.5 - Conservative (only major scene changes)

---

Python Scene Detection with OpenCV

import cv2
import numpy as np
from typing import List, Tuple

def detect_scene_changes(
    video_path: str,
    threshold: float = 0.3,
    min_frame_gap: int = 10
) -> List[int]:
    """
    Detect scene changes using histogram comparison

    Args:
        video_path: Path to video file
        threshold: Scene change threshold (0.0-1.0)
        min_frame_gap: Minimum frames between scene changes

    Returns:
        List of frame numbers where scenes change
    """
    video = cv2.VideoCapture(video_path)

    scene_frames = []
    prev_hist = None
    frame_count = 0
    last_scene_frame = -min_frame_gap

    while True:
        ret, frame = video.read()
        if not ret:
            break

        # Convert to grayscale
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

        # Calculate histogram
        hist = cv2.calcHist([gray], [0], None, [256], [0, 256])
        cv2.normalize(hist, hist)

        if prev_hist is not None:
            # Compare histograms
            correlation = cv2.compareHist(
                prev_hist,
                hist,
                cv2.HISTCMP_CORREL
            )

            # Scene change detected
            if correlation < (1 - threshold):
                # Respect minimum gap
                if frame_count - last_scene_frame >= min_frame_gap:
                    scene_frames.append(frame_count)
                    last_scene_frame = frame_count

        prev_hist = hist
        frame_count += 1

    video.release()
    return scene_frames

---

Advanced: Structural Similarity (SSIM)

from skimage.metrics import structural_similarity as ssim

def detect_scenes_ssim(
    video_path: str,
    threshold: float = 0.7
) -> List[int]:
    """
    Detect scene changes using SSIM

    More accurate than histogram, but slower
    """
    video = cv2.VideoCapture(video_path)

    scene_frames = []
    prev_frame = None
    frame_count = 0

    while True:
        ret, frame = video.read()
        if not ret:
            break

        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

        if prev_frame is not None:
            # Calculate SSIM
            score = ssim(prev_frame, gray)

            # Low SSIM = scene change
            if score < threshold:
                scene_frames.append(frame_count)

        prev_frame = gray
        frame_count += 1

    video.release()
    return scene_frames

SSIM vs Histogram:

• Histogram: Fast (200 FPS), good for gross changes
• SSIM: Slower (50 FPS), better for subtle changes
• Use histogram for drone footage, wildlife
• Use SSIM for indoor scenes, dialogue cuts

---

Memory-Efficient Streaming

Problem: Loading Entire Video

# ❌ WRONG: Loads entire video into memory
video = cv2.VideoCapture('large_video.mp4')
frames = []
while True:
    ret, frame = video.read()
    if not ret:
        break
    frames.append(frame)  # 4K frame = 32 MB!

# 30 seconds of 4K @ 30 FPS = 28 GB RAM

---

Solution: Batch Processing

# ✅ CORRECT: Process in batches
def process_video_batched(
    video_path: str,
    model,
    batch_size: int = 16,
    sample_rate: int = 30
):
    """Process video in batches to limit memory usage"""
    video = cv2.VideoCapture(video_path)

    batch = []
    frame_count = 0

    while True:
        ret, frame = video.read()
        if not ret:
            # Process final batch
            if batch:
                yield model(batch)
            break

        frame_count += 1

        # Sample frames
        if frame_count % sample_rate != 0:
            continue

        # Preprocess
        processed = preprocess_frame(frame)
        batch.append(processed)

        # Process batch when full
        if len(batch) >= batch_size:
            results = model(batch)
            yield results
            batch = []  # Clear memory

    video.release()

# Usage
for batch_results in process_video_batched('video.mp4', yolo_model):
    # Process results immediately
    save_results(batch_results)

Memory savings:

• Batch of 16 frames @ 640x640 = 31 MB
• vs 900 frames @ 4K = 28 GB
• 900x less memory

---

Video Codec Optimization

Choosing the Right Codec

For CV pipelines: Use H.264 for storage, extract frames for processing

---

Re-encoding for Speed

# Re-encode to H.264 for faster seeking
ffmpeg -i input.mp4 -c:v libx264 -preset ultrafast -crf 23 output.mp4

# Preset options:
# - ultrafast: Fastest encoding, larger files
# - fast: Good balance
# - medium: Default
# - slow: Better compression

# CRF (quality):
# - 0: Lossless (huge files)
# - 18-23: High quality (visually lossless)
# - 28: Acceptable quality
# - 51: Worst quality

---

Preprocessing Pipeline

Complete Pipeline

import cv2
import numpy as np

class VideoPreprocessor:
    """Complete preprocessing pipeline for CV"""

    def __init__(
        self,
        target_size: int = 640,
        normalize: bool = True,
        enhance_contrast: bool = False
    ):
        self.target_size = target_size
        self.normalize = normalize
        self.enhance_contrast = enhance_contrast

    def preprocess_frame(self, frame: np.ndarray) -> np.ndarray:
        """
        Full preprocessing pipeline

        1. Resize to target size
        2. Pad to square
        3. Enhance contrast (optional)
        4. Normalize (optional)
        """
        # 1. Resize
        h, w = frame.shape[:2]
        scale = self.target_size / max(h, w)
        new_w, new_h = int(w * scale), int(h * scale)

        resized = cv2.resize(
            frame,
            (new_w, new_h),
            interpolation=cv2.INTER_LINEAR
        )

        # 2. Pad to square
        pad_w = (self.target_size - new_w) // 2
        pad_h = (self.target_size - new_h) // 2

        padded = cv2.copyMakeBorder(
            resized,
            pad_h, self.target_size - new_h - pad_h,
            pad_w, self.target_size - new_w - pad_w,
            cv2.BORDER_CONSTANT,
            value=(114, 114, 114)  # Gray padding
        )

        # 3. Enhance contrast (optional)
        if self.enhance_contrast:
            lab = cv2.cvtColor(padded, cv2.COLOR_BGR2LAB)
            l, a, b = cv2.split(lab)
            clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
            l = clahe.apply(l)
            padded = cv2.merge([l, a, b])
            padded = cv2.cvtColor(padded, cv2.COLOR_LAB2BGR)

        # 4. Normalize (optional)
        if self.normalize:
            padded = padded.astype(np.float32) / 255.0

        return padded

---

FFmpeg Advanced Filters

Multi-Stage Filtering

# Extract, resize, denoise, and sharpen
ffmpeg -i video.mp4 \
  -vf "fps=1,scale=640:640:force_original_aspect_ratio=decrease,pad=640:640:(ow-iw)/2:(oh-ih)/2,hqdn3d=4:3:6:4.5,unsharp=5:5:1.0:5:5:0.0" \
  -q:v 2 \
  frames/frame_%06d.jpg

Filter breakdown:

• fps=1 - 1 frame per second
• scale=640:640:force_original_aspect_ratio=decrease - Resize
• pad=640:640:(ow-iw)/2:(oh-ih)/2 - Center padding
• hqdn3d=4:3:6:4.5 - Denoise (luma:chroma:luma_temporal:chroma_temporal)
• unsharp=5:5:1.0:5:5:0.0 - Sharpen (luma only)

---

Extracting Specific Time Ranges

# Extract frames from 1:30 to 2:00
ffmpeg -i video.mp4 -ss 00:01:30 -to 00:02:00 -vf fps=1 frames/frame_%06d.jpg

# Extract frames starting at 5:00 for 30 seconds
ffmpeg -i video.mp4 -ss 00:05:00 -t 30 -vf fps=1 frames/frame_%06d.jpg

---

Performance Benchmarks

Frame Extraction Speed

Tested on 10-minute 4K drone footage (30 FPS, 18,000 frames):

Winner: FFmpeg with select filter (6x faster than Python)

---

Scene Detection Performance

10-minute video, detecting scene changes:

Winner: FFmpeg scene filter (fast + accurate)

---

Best Practices

1. Always sample frames

• Use fps=1 for general use
• Use scene detection for narrative content
• Process every frame only for critical applications

2. Resize before detection

• YOLO expects 640x640
• Resizing during extraction is 3x faster than after

3. Use FFmpeg for extraction

• 6x faster than Python
• Better quality control
• GPU acceleration available

4. Batch process frames

• Load 16-32 frames at once
• Process batch together
• Clear memory between batches

5. Choose codec wisely

• H.264 for general use
• ProRes for frame-accurate seeking
• Avoid H.265 for extraction (slow to decode)

---

Common Pitfalls

Pitfall 1: Extracting Every Frame

# ❌ WRONG: 18,000 frames for 10-minute video
ffmpeg -i video.mp4 frames/frame_%06d.jpg

Why it's wrong:

• 18,000 inferences (slow, expensive)
• Adjacent frames are nearly identical
• Wasting GPU cycles on duplicate information

Solution:

# ✅ CORRECT: 600 frames (97% reduction)
ffmpeg -i video.mp4 -vf fps=1 frames/frame_%06d.jpg

---

Pitfall 2: Not Preprocessing

# ❌ WRONG: Extract at original resolution
ffmpeg -i 4k_video.mp4 -vf fps=1 frames/frame_%06d.jpg
# Then resize in Python (slow)

Why it's wrong:

• Large files (32 MB per 4K frame)
• Slower I/O
• Extra preprocessing step

Solution:

# ✅ CORRECT: Resize during extraction
ffmpeg -i 4k_video.mp4 -vf "fps=1,scale=640:640:force_original_aspect_ratio=decrease,pad=640:640:(ow-iw)/2:(oh-ih)/2" frames/frame_%06d.jpg

---

Pitfall 3: Poor Quality Settings

# ❌ WRONG: Default quality (artifacts)
ffmpeg -i video.mp4 -vf fps=1 frames/frame_%06d.jpg

Why it's wrong:

• JPEG compression artifacts hurt detection accuracy
• Default quality varies by FFmpeg version

Solution:

# ✅ CORRECT: Explicit quality setting
ffmpeg -i video.mp4 -vf fps=1 -q:v 2 frames/frame_%06d.jpg
# q:v 2 = very high quality

---

Resources

• FFmpeg Filters Documentation
• OpenCV Video Processing
• Scene Detection Library (PySceneDetect)

YOLOv8 Guide

Complete guide to YOLOv8 for object detection: setup, training, inference, and optimization.

Installation

# Install ultralytics (includes YOLOv8)
pip install ultralytics

# Verify installation
yolo version

# Install additional dependencies
pip install opencv-python numpy torch torchvision

Latest version: ultralytics 8.1.20 (Jan 2024)

---

Model Variants

Naming:

• n = nano (smallest)
• s = small
• m = medium
• l = large
• x = extra large

Speed measured on: NVIDIA T4 GPU, batch size 1, image size 640x640

---

Basic Inference

Load Pre-trained Model

from ultralytics import YOLO

# Load model
model = YOLO('yolov8n.pt')  # nano model

# Run inference on single image
results = model('image.jpg')

# Access results
for result in results:
    boxes = result.boxes  # Bounding boxes
    masks = result.masks  # Segmentation masks (if using seg model)
    probs = result.probs  # Classification probabilities

Process Results

import cv2

results = model('image.jpg')

for result in results:
    # Get bounding boxes
    boxes = result.boxes

    for box in boxes:
        # Coordinates
        x1, y1, x2, y2 = box.xyxy[0]  # Box coordinates

        # Metadata
        conf = box.conf[0]  # Confidence
        cls = box.cls[0]    # Class ID
        label = result.names[int(cls)]  # Class name

        print(f"{label} {conf:.2f} at ({x1}, {y1}, {x2}, {y2})")

        # Draw on image
        img = result.orig_img
        cv2.rectangle(img, (int(x1), int(y1)), (int(x2), int(y2)), (0, 255, 0), 2)
        cv2.putText(img, f'{label} {conf:.2f}', (int(x1), int(y1)-10),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)

    cv2.imwrite('output.jpg', img)

---

Inference Options

Confidence and IoU Thresholds

results = model(
    'image.jpg',
    conf=0.5,    # Confidence threshold (default: 0.25)
    iou=0.7      # IoU threshold for NMS (default: 0.7)
)

Image Size

results = model(
    'image.jpg',
    imgsz=640    # Image size (default: 640)
                  # Can be single int or tuple (height, width)
)

Device Selection

# Use GPU
results = model('image.jpg', device=0)  # GPU 0

# Use CPU
results = model('image.jpg', device='cpu')

# Multiple GPUs
results = model('image.jpg', device=[0, 1])  # GPUs 0 and 1

Max Detections

results = model(
    'image.jpg',
    max_det=100  # Maximum detections per image (default: 300)
)

---

Batch Inference

# List of images
image_paths = ['img1.jpg', 'img2.jpg', 'img3.jpg']
results = model(image_paths)

# Process results
for i, result in enumerate(results):
    print(f"Image {i}: {len(result.boxes)} detections")

---

Training Custom Models

Dataset Format

YOLO requires this directory structure:

dataset/
├── data.yaml
├── images/
│   ├── train/
│   │   ├── img001.jpg
│   │   └── img002.jpg
│   └── val/
│       ├── img101.jpg
│       └── img102.jpg
└── labels/
    ├── train/
    │   ├── img001.txt
    │   └── img002.txt
    └── val/
        ├── img101.txt
        └── img102.txt

data.yaml:

path: /path/to/dataset
train: images/train
val: images/val

names:
  0: dolphin
  1: whale
  2: shark

Label format (one line per object):

<class_id> <x_center> <y_center> <width> <height>

All values normalized to [0, 1].

Example (img001.txt):

0 0.5 0.5 0.3 0.4
1 0.7 0.3 0.2 0.2

---

Train Model

from ultralytics import YOLO

# Load pre-trained model
model = YOLO('yolov8n.pt')

# Train
results = model.train(
    data='data.yaml',
    epochs=100,
    imgsz=640,
    batch=16,
    name='dolphin_detector',
    device=0
)

Training parameters:

• epochs: Number of training epochs
• imgsz: Image size (640, 1280, etc.)
• batch: Batch size (reduce if OOM errors)
• patience: Early stopping patience (default: 50)
• lr0: Initial learning rate (default: 0.01)
• lrf: Final learning rate (default: 0.01)
• momentum: SGD momentum (default: 0.937)
• weight_decay: Optimizer weight decay (default: 0.0005)
• warmup_epochs: Warmup epochs (default: 3.0)
• save: Save checkpoints (default: True)
• device: GPU device (0, 1, ..., 'cpu')

---

Resume Training

# Resume from last checkpoint
model = YOLO('runs/detect/dolphin_detector/weights/last.pt')
model.train(resume=True)

---

Augmentation

YOLO automatically applies augmentations. You can customize:

model.train(
    data='data.yaml',
    epochs=100,
    # Augmentation parameters
    hsv_h=0.015,       # HSV-Hue augmentation
    hsv_s=0.7,         # HSV-Saturation augmentation
    hsv_v=0.4,         # HSV-Value augmentation
    degrees=0.0,       # Rotation (+/- deg)
    translate=0.1,     # Translation (+/- fraction)
    scale=0.5,         # Scale (+/- gain)
    shear=0.0,         # Shear (+/- deg)
    perspective=0.0,   # Perspective (+/- fraction)
    flipud=0.0,        # Flip up-down (probability)
    fliplr=0.5,        # Flip left-right (probability)
    mosaic=1.0,        # Mosaic augmentation (probability)
    mixup=0.0          # MixUp augmentation (probability)
)

---

Validation

# Validate trained model
model = YOLO('runs/detect/dolphin_detector/weights/best.pt')

metrics = model.val(data='data.yaml')

# Access metrics
print(f"mAP50: {metrics.box.map50:.4f}")
print(f"mAP50-95: {metrics.box.map:.4f}")
print(f"Precision: {metrics.box.mp:.4f}")
print(f"Recall: {metrics.box.mr:.4f}")

# Per-class metrics
for i, ap in enumerate(metrics.box.ap50):
    print(f"Class {i} AP50: {ap:.4f}")

---

Export

ONNX (Recommended for Production)

model = YOLO('best.pt')

# Export to ONNX
model.export(format='onnx', imgsz=640)

# Use exported model
onnx_model = YOLO('best.onnx')
results = onnx_model('image.jpg')

Benefits:

• Faster inference (~2x)
• Smaller file size
• Cross-platform compatibility

Other Formats

# TorchScript
model.export(format='torchscript')

# CoreML (for iOS)
model.export(format='coreml')

# TensorFlow Lite (for mobile)
model.export(format='tflite')

# TensorFlow.js (for web)
model.export(format='tfjs')

---

CLI Usage

Inference

# Single image
yolo detect predict model=yolov8n.pt source=image.jpg

# Video
yolo detect predict model=yolov8n.pt source=video.mp4

# Webcam
yolo detect predict model=yolov8n.pt source=0

# Directory
yolo detect predict model=yolov8n.pt source=./images/

# With options
yolo detect predict model=yolov8n.pt source=image.jpg conf=0.5 iou=0.7 save=true

Training

yolo detect train data=data.yaml model=yolov8n.pt epochs=100 imgsz=640

Validation

yolo detect val model=best.pt data=data.yaml

---

Optimization Tips

1. Mixed Precision Training (FP16)

model.train(
    data='data.yaml',
    epochs=100,
    amp=True  # Automatic Mixed Precision (faster, less memory)
)

Benefits:

• 2x faster training
• 50% less GPU memory
• Minimal accuracy loss

---

2. Optimal Batch Size

# Find max batch size for your GPU
for batch_size in [4, 8, 16, 32, 64]:
    try:
        model.train(data='data.yaml', epochs=1, batch=batch_size)
        print(f"Batch {batch_size}: OK")
    except RuntimeError as e:
        print(f"Batch {batch_size}: OOM")
        break

---

3. Freeze Layers

# Freeze backbone layers (faster convergence for similar data)
model.train(
    data='data.yaml',
    epochs=100,
    freeze=10  # Freeze first 10 layers
)

---

4. Multi-GPU Training

# Use all GPUs
model.train(
    data='data.yaml',
    epochs=100,
    device=[0, 1, 2, 3]  # Use GPUs 0-3
)

---

Common Issues

Out of Memory (OOM)

Solution 1: Reduce batch size

model.train(batch=8)  # Instead of 16

Solution 2: Reduce image size

model.train(imgsz=416)  # Instead of 640

Solution 3: Use gradient accumulation

# Simulate larger batch size
model.train(batch=4, accumulate=4)  # Effective batch size = 16

---

Slow Training

Solution 1: Use smaller model

model = YOLO('yolov8n.pt')  # Instead of yolov8x.pt

Solution 2: Enable AMP

model.train(amp=True)

Solution 3: Use multiple workers

model.train(workers=8)  # More data loading workers

---

Poor Accuracy

Solution 1: Train longer

model.train(epochs=300, patience=100)

Solution 2: Increase image size

model.train(imgsz=1280)  # Higher resolution

Solution 3: More data augmentation

model.train(mosaic=1.0, mixup=0.15, copy_paste=0.3)

Solution 4: Use larger model

model = YOLO('yolov8l.pt')  # Instead of yolov8n.pt

---

Benchmarking

from ultralytics.utils.benchmarks import benchmark

# Benchmark model
benchmark(model='yolov8n.pt', imgsz=640, half=False, device=0)

Output:

Model          Format    Size (MB)  mAP50-95  Inference (ms)
yolov8n        PyTorch   6.2        37.3      80.2
yolov8n        ONNX      12.4       37.3      42.1  (1.9x faster)
yolov8n        TensorRT  13.1       37.3      28.5  (2.8x faster)

---

Resources

• YOLOv8 Documentation
• Ultralytics GitHub
• Model Zoo
• Training Tips