Collage Layout Expert

Collage & Layout Composition Expert

Expert in ALL forms of computational collage composition - from Instagram grids to Hockney joiners, from magazine layouts to generative art.

When to Use This Skill

✅ Use for:

• Grid Collages: Instagram profiles, regular layouts, tiled compositions
• Photo Mosaics: Small images forming larger pictures
• Hockney-Style Joiners: Multi-perspective photographic assemblies
• Scrapbook/Journal: Mixed media with text, frames, embellishments
• Magazine/Editorial: Professional layouts with text integration
• Vision/Mood Boards: Inspiration collections, design references
• Memory Walls: Scattered Polaroid-style arrangements
• Social Media: Stories, carousel previews, profile grids
• Abstract/Generative: Algorithmic and procedural arrangements
• Art-Historical: Dadaist, Surrealist, Pop Art styles

❌ Do NOT use for:

• Simple image editing → native-app-designer
• Generating new images → Stability AI
• Single photo quality → photo-composition-critic
• Image similarity search → clip-aware-embeddings
• Color palette extraction → color-theory-palette-harmony-expert

Expert vs Novice Shibboleths

Decision Tree: Choosing a Style

What's the purpose?

• Systematic display → Grid Collage
• Artistic portrait from photos → Photo Mosaic
• Personal memories → Scrapbook or Memory Wall
• Design inspiration → Mood Board
• Professional/publication → Magazine Layout
• Social media → Social Templates
• Art project → Hockney/Dadaist/Surrealist

What's the vibe?

• Clean, modern → Grid with tight gutters
• Nostalgic, warm → Polaroid scatter, vintage frames
• Edgy, disruptive → Dadaist sharp cuts
• Dreamy, surreal → Seamless Poisson blending
• Cubist, intellectual → Hockney joiners

Core Algorithms (Summary)

→ See references/algorithms.md for full implementations.

Anti-Patterns to Avoid

1. Ignoring Visual Weight

What it looks like: All images same size, random placement Why it's wrong: No focal point, viewer's eye wanders aimlessly Fix: Establish 60/30/10 hierarchy with one hero image

2. Over-Saturating the Canvas

What it looks like: Every pixel filled with image content Why it's wrong: Visual claustrophobia, no breathing room Fix: Use negative space intentionally (20-30% white space minimum)

3. Linear FFT for Color Matching

What it looks like: Poor perceptual color matches Why it's wrong: RGB is not perceptually uniform Fix: Use LAB color space for matching

4. Seamless Hockney Joiners

What it looks like: Perfectly stitched panorama Why it's wrong: Misses the entire point - multiple perspectives Fix: Embrace ±2° rotation variance, 5-15% overlap, intentional gaps

5. Global Poisson Blending

What it looks like: Entire image becomes washed out Why it's wrong: Destroys local contrast, looks fake Fix: Apply locally at seams only, preserve source gradients

6. Reusing Mosaic Tiles

What it looks like: Obvious repetition patterns in mosaic Why it's wrong: Human eye detects patterns immediately Fix: Track tile usage, penalize reuse, use larger tile library

MCP Integrations

Performance Targets

References

→ references/collage-types.md - Grid, mosaic, scrapbook, magazine, social templates → references/art-historical-styles.md - Hockney, Dadaist, Surrealist, Rauschenberg → references/algorithms.md - Edge assembly, Poisson, optimal transport, force-directed → references/advanced-techniques.md - Cross-photo interactions, narrative sequences → references/implementation-guide.md - Metal shaders, Core ML, performance

Integrates With

• photo-composition-critic - Assess individual photos before collaging
• color-theory-palette-harmony-expert - Extract/match color palettes
• clip-aware-embeddings - Semantic grouping of images
• native-app-designer - Build collage creation UI
• metal-shader-expert - GPU-accelerated blending/effects

---

Remember: Great collages tell stories through arrangement. Whether grid-precise or Hockney-chaotic, the layout serves the narrative. Master both the math and the art.

Changelog

All notable changes to the collage-layout-expert skill will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[2.0.0] - 2025-11-26

Changed

• BREAKING: Refactored from single 1761-line file to modular structure
• Reduced SKILL.md from 1761 lines to 287 lines (84% reduction)
• Moved detailed implementations to /references/ directory
• Updated frontmatter from custom YAML to standard allowed-tools format
• Simplified description with proper NOT clause and activation keywords

Added

• When to Use This Skill section with clear scope boundaries
• Do NOT use for section with skill alternatives
• MCP Integrations section (Firecrawl, Stability AI)
• Created /references/hockney-technique.md:
• David Hockney's joiners technique (1982-1985)
• Historical context and artistic intent
• Computational implementation parameters
• Art historical references (Rauschenberg, Höch, Baldessari, Rosler)
• Style implementations dictionary
• Contemporary trends (2025)
• Created /references/line-detection.md:
• Algorithm comparison table (EDLines, LSD, Hough, LB-LSD, LETR)
• EDLines algorithm overview and performance benchmarks
• LSD and Hough implementation examples
• When to use each algorithm guide
• Created /references/edge-assembly.md:
• EdgeDescriptor dataclass
• Edge compatibility scoring function
• Angle and position alignment algorithms
• Greedy edge growth algorithm
• Edge urgency heuristics
• Performance optimizations:
• Hierarchical clustering (50x speedup)
• Multi-scale matching (10x speedup)
• Caching good pairs (1.5x speedup)
• Pruning generic edges (2-3x speedup)
• Backtracking strategy
• Created /references/mathematical-foundations.md:
• Optimal transport for color harmonization
• Wasserstein distance and Sinkhorn algorithm
• Affine approximation for real-time
• LAB color space rationale
• Poisson blending for seamless junctions
• Jacobi iteration solver (Python and Metal)
• Energy function formulation
• Semantic, geometric, and aesthetic energy components
• User mode presets (Coherent, Balanced, Chaotic)
• Aesthetic principles (Rule of thirds, visual weight, balance, golden ratio, negative space)
• Created /references/advanced-techniques.md:
• Cross-photo interactions (gesture-response, pointing, gaze, passing)
• InteractionDetector class
• Negative space awareness and matching
• Multi-layer compositing
• Narrative sequences (journey, day_in_life, emotion_arc)
• Simulated annealing for photo swapping
• Genetic algorithms concept
• CSP formulation concept
• Created /references/implementation-guide.md:
• Metal shader pipeline (edge extraction, line detection, histogram, Poisson)
• Core ML integration (MobileSAM, CLIP, MediaPipe)
• HNSW database indexing
• Performance targets table (Mac M2, iPhone 15 Pro)
• Memory management strategies
• Algorithm selection guide
• Error handling patterns
• Testing strategies
• Python dependencies

Removed

• Custom YAML frontmatter format (tools, triggers, integrates_with, python_dependencies)
• 1500+ lines of detailed implementations (moved to references)

Improved

• Progressive disclosure: essential concepts in SKILL.md, full code in references
• Quick reference tables for algorithms and performance
• Cross-references to related skills (native-app-designer, clip-aware-embeddings, photo-composition-critic, color-theory-palette-harmony-expert)

[1.0.0] - 2024-XX-XX

Added

• Initial collage-layout-expert skill
• David Hockney's joiners technique documentation
• Line detection algorithms (EDLines, LSD, Hough)
• Edge-based assembly strategy with greedy growth
• Advanced collage concepts:
• Cross-photo interactions
• Negative space awareness
• Multi-layer compositing
• Narrative sequences
• Mathematical foundations:
• Optimal transport (Wasserstein, Sinkhorn)
• Poisson blending
• Energy function optimization
• Aesthetic principles from art history
• Art historical references (Hockney, Rauschenberg, Höch, Baldessari, Rosler)
• Practical implementation guidance:
• Metal shader pipeline
• Core ML integration
• Database indexing (HNSW)
• Performance optimization strategies
• Common patterns and best practices
• Advanced techniques (simulated annealing, genetic algorithms)

Advanced Collage Techniques

Cross-Photo Interactions

Concept: Photos "talk" to each other across boundaries.

Types of Interactions

1. Gesture-Response Pairs:

   Photo A (left): Person waving to the right →
   Photo B (right): Person waving to the left ←
   Result: Two people greeting each other

2. Pointing Interactions:

   Photo A: Person pointing right →
   Photo B: Interesting object/scene
   Result: Person pointing at the object

3. Gaze Direction:

   Photo A: Person looking right →
   Photo B: Beautiful landscape
   Result: Person admiring the view

4. Passing Objects:

   Photo A (top): Hands reaching down ↓
   Photo B (bottom): Hands reaching up ↑
   Result: Handing something between photos

Implementation

class InteractionDetector:
    def __init__(self):
        self.pose_estimator = load_pose_model()
        self.action_classifier = load_action_model()

    def find_interaction_pairs(self, photo1, photo2, edge_pair):
        """Find natural interactions across photo boundary."""
        people1 = self.detect_people(photo1)
        people2 = self.detect_people(photo2)

        interactions = []
        for p1 in people1:
            if not self.is_near_edge(p1, edge_pair[0]):
                continue

            gesture1 = self.detect_gesture(photo1, p1.bbox)

            for p2 in people2:
                if not self.is_near_edge(p2, edge_pair[1]):
                    continue

                gesture2 = self.detect_gesture(photo2, p2.bbox)
                score = self.score_interaction(gesture1, gesture2)

                if score > 0.5:
                    interactions.append({
                        'person1': p1,
                        'person2': p2,
                        'type': self.classify_interaction(gesture1, gesture2),
                        'score': score
                    })

        return interactions

    def score_interaction(self, gesture1, gesture2):
        """Natural interaction pairs."""
        NATURAL_PAIRS = {
            ('waving', 'waving'): 0.9,
            ('waving', 'looking'): 0.8,
            ('pointing', 'looking'): 0.85,
            ('reaching', 'reaching'): 0.7,
            ('throwing', 'catching'): 0.95,
            ('looking_right', 'looking_left'): 0.7,
        }

        key = (gesture1['gesture'], gesture2['gesture'])
        base_score = NATURAL_PAIRS.get(key, 0.3)

        if self.directions_align(gesture1, gesture2):
            base_score += 0.1

        return min(1.0, base_score)

---

Negative Space Awareness

The Insight: Empty space is as important as filled space.

class NegativeSpaceAnalyzer:
    def analyze_negative_space(self, photo, subject_mask):
        """Analyze quality and distribution of negative space."""
        h, w = photo.shape[:2]
        negative_mask = 1 - subject_mask

        breathing_room = {
            'top': negative_mask[:h//3, :].mean(),
            'bottom': negative_mask[2*h//3:, :].mean(),
            'left': negative_mask[:, :w//3].mean(),
            'right': negative_mask[:, 2*w//3:].mean(),
            'overall': negative_mask.mean()
        }

        background = photo * negative_mask[..., None]
        bg_variance = np.var(background)
        quality = 1.0 - min(1.0, bg_variance / 1000.0)

        return {
            'distribution': breathing_room,
            'quality': quality,
            'total_ratio': breathing_room['overall']
        }

    def match_negative_space(self, analysis1, analysis2):
        """Find complementary negative space patterns."""
        # Subject on left + Subject on right = good pair
        if (analysis1['distribution']['left'] < 0.3 and
            analysis2['distribution']['right'] < 0.3):
            return 'left_right_pair', 0.9

        if (analysis1['distribution']['bottom'] < 0.3 and
            analysis2['distribution']['top'] < 0.3):
            return 'top_bottom_pair', 0.9

        if (analysis1['distribution']['right'] > 0.6 and
            analysis2['distribution']['right'] > 0.6):
            return 'right_stack', 0.7

        return None, 0.0

Use Case:

Photo A: Person on left, empty beach on right
Photo B: Sunset on right, empty ocean on left

Composite: Person (from A) on left + Sunset (from B) on right
Result: Person appears to be watching the sunset

---

Multi-Layer Compositing

Concept: Create depth through foreground/midground/background layers.

class LayeredCollage:
    def create_layered_composition(self, photos):
        """Build composition with depth layers."""

        background_photos = self.select_backgrounds(photos)
        midground_photos = self.select_midgrounds(photos)
        foreground_photos = self.select_foregrounds(photos)

        layers = {
            'background': self.create_background_layer(background_photos),
            'midground': self.create_midground_layer(midground_photos),
            'foreground': self.create_foreground_layer(foreground_photos)
        }

        canvas = self.composite_layers(layers)
        return canvas

    def select_backgrounds(self, photos):
        """Select photos suitable for background layer."""
        candidates = []

        for photo in photos:
            score = 0.0

            if self.contains_sky(photo):
                score += 0.5
            if self.is_landscape_oriented(photo):
                score += 0.3

            depth = self.estimate_depth(photo)
            if depth.mean() > 0.7:
                score += 0.2

            if score > 0.5:
                candidates.append((photo, score))

        return [p for p, s in sorted(candidates, key=lambda x: -x[1])]

---

Narrative Sequences

Concept: Tell a story across the collage.

class NarrativeCollageBuilder:
    def build_story_collage(self, photos, story_type='journey'):
        """Build collage that tells a story."""

        if story_type == 'journey':
            # Start → Travel → Arrive → Experience → Depart
            segments = self.segment_by_story_arc(photos)
            layout = self.create_flow_layout(segments)

        elif story_type == 'day_in_life':
            # Morning → Midday → Evening → Night
            segments = self.segment_by_time_of_day(photos)
            layout = self.create_temporal_gradient_layout(segments)

        elif story_type == 'emotion_arc':
            # Calm → Excitement → Joy → Reflection
            segments = self.segment_by_emotion(photos)
            layout = self.create_emotional_flow_layout(segments)

        return layout

    def segment_by_story_arc(self, photos):
        """Cluster photos into narrative segments."""
        features = []
        for photo in photos:
            feat = np.concatenate([
                photo.clip_embedding,
                self.encode_location(photo.gps),
                self.encode_time(photo.timestamp)
            ])
            features.append(feat)

        segments = self.hierarchical_cluster(features, n_clusters=5)

        segments = sorted(segments,
                         key=lambda s: np.mean([p.timestamp for p in s]))

        return {
            'beginning': segments[0],
            'rising': segments[1],
            'climax': segments[2],
            'falling': segments[3],
            'end': segments[4]
        }

---

Simulated Annealing for Photo Swapping

When to Use: User explicitly wants to explore alternative arrangements, or initial assembly has suboptimal global aesthetics.

What It Does: Randomly swaps photos in the existing collage and accepts swaps that improve the global energy function.

def refine_with_simulated_annealing(canvas, max_iters=10000):
    """
    Refine existing collage by swapping photos.
    NOTE: This is a refinement, NOT the primary assembly algorithm.
    """
    T = 10.0
    T_min = 0.01
    cooling_rate = 0.95

    current_energy = compute_total_energy(canvas)
    best_canvas = canvas.copy()
    best_energy = current_energy

    for iteration in range(max_iters):
        canvas_new = canvas.copy()
        i, j = random.sample(range(len(canvas.shards)), 2)
        canvas_new.swap_shards(i, j)

        new_energy = compute_total_energy(canvas_new)
        delta_E = new_energy - current_energy

        if delta_E < 0:
            canvas = canvas_new
            current_energy = new_energy
        else:
            acceptance_prob = np.exp(-delta_E / T)
            if np.random.random() < acceptance_prob:
                canvas = canvas_new
                current_energy = new_energy

        if current_energy < best_energy:
            best_canvas = canvas.copy()
            best_energy = current_energy

        T = max(T_min, T * cooling_rate)

    return best_canvas

Performance:

• Time: 5-15 seconds for 50 photos
• Quality gain: 5-10% improvement in global aesthetics
• Diminishing returns after 1000-2000 iterations

When NOT to Use:

• Interactive editing (too slow)
• Initial assembly (use greedy edge growth)
• User wants predictable results (stochastic)

---

Genetic Algorithms for Layout Evolution

Concept: Maintain population of collages, breed and mutate to explore layout space.

Operations:

• Crossover: Swap regions between two parent collages
• Mutation: Random perturbations (rotate, scale, move shards)
• Selection: Keep top-scoring collages, discard poor ones

Performance: Even slower than simulated annealing, typically for offline rendering.

---

Constraint Satisfaction Problem (CSP) Formulation

Concept: Define collage assembly as constraint satisfaction problem.

Constraints:

• Edge compatibility > threshold
• No overlaps (or controlled overlaps for Hockney)
• Minimum global aesthetics score
• Semantic coherence within range

Listed as alternative strategy, not recommended for MVP.

Core Collage Algorithms

Mathematical and computational techniques for collage composition.

---

Edge-Based Assembly (Hockney/Joiners)

def edge_compatibility(edge1, edge2):
    """Score how well two edges can connect (0-1)."""
    return (
        0.30 * line_continuation_score +
        0.15 * curve_flow_score +
        0.25 * color_harmony_score +
        0.20 * semantic_coherence +  # CLIP similarity
        0.10 * complexity_balance
    )

Edge Extraction

def extract_edges(image, edge_position='right'):
    """
    Extract edge strip from image for compatibility scoring.
    """
    edge_width = 20  # pixels

    if edge_position == 'right':
        return image[:, -edge_width:]
    elif edge_position == 'left':
        return image[:, :edge_width]
    elif edge_position == 'top':
        return image[:edge_width, :]
    elif edge_position == 'bottom':
        return image[-edge_width:, :]

Line Continuation Score

def line_continuation_score(edge1, edge2):
    """
    Score how well lines continue across edge boundary.
    Uses Hough line detection.
    """
    # Detect lines in both edges
    lines1 = cv2.HoughLinesP(edge1, 1, np.pi/180, 50)
    lines2 = cv2.HoughLinesP(edge2, 1, np.pi/180, 50)

    if lines1 is None or lines2 is None:
        return 0.5  # Neutral score

    # Find lines that approach the boundary
    boundary_lines1 = [l for l in lines1 if approaches_boundary(l, 'right')]
    boundary_lines2 = [l for l in lines2 if approaches_boundary(l, 'left')]

    # Score angle continuity
    score = 0
    for l1 in boundary_lines1:
        for l2 in boundary_lines2:
            angle_diff = abs(get_angle(l1) - get_angle(l2))
            score += max(0, 1 - angle_diff / 45)  # Within 45° = good

    return min(1.0, score / max(len(boundary_lines1), 1))

---

Poisson Blending (Seamless Transitions)

Preserves gradients from source while matching boundary conditions.

def poisson_blend(source, target, mask, center):
    """
    Seamless clone using OpenCV's implementation.
    """
    # Ensure mask is binary
    mask = (mask > 127).astype(np.uint8) * 255

    # NORMAL_CLONE preserves source gradients
    # MIXED_CLONE preserves stronger gradient from either
    result = cv2.seamlessClone(
        source, target, mask, center,
        cv2.NORMAL_CLONE  # or cv2.MIXED_CLONE
    )

    return result

When to Use Each Mode

• NORMAL_CLONE: Standard seamless blending, preserves source fully
• MIXED_CLONE: Preserves dominant gradients (good for textured backgrounds)
• MONOCHROME_TRANSFER: Transfers lighting only, not color

Performance Notes

• GPU-parallelizable with Jacobi iteration
• ~20ms for 512×512 on modern GPU
• ~100ms for 1080p on CPU

---

Optimal Transport (Color Harmonization)

Wasserstein distance measures "effort" to transform color distributions.

def color_transfer_optimal_transport(source, target):
    """
    Transfer color distribution from target to source
    using optimal transport.
    """
    # Convert to LAB for perceptual uniformity
    source_lab = cv2.cvtColor(source, cv2.COLOR_BGR2LAB)
    target_lab = cv2.cvtColor(target, cv2.COLOR_BGR2LAB)

    # Compute means and covariances
    source_mean, source_std = compute_stats(source_lab)
    target_mean, target_std = compute_stats(target_lab)

    # Affine transformation
    result = (source_lab - source_mean) * (target_std / source_std) + target_mean

    return cv2.cvtColor(result.astype(np.uint8), cv2.COLOR_LAB2BGR)

def compute_stats(lab_image):
    """Compute per-channel mean and std."""
    mean = np.mean(lab_image, axis=(0, 1))
    std = np.std(lab_image, axis=(0, 1))
    std = np.where(std == 0, 1, std)  # Avoid division by zero
    return mean, std

Sinkhorn Algorithm (Full Optimal Transport)

def sinkhorn_color_transfer(source, target, reg=0.01, iterations=100):
    """
    More accurate but slower color transfer using Sinkhorn.
    """
    import ot  # POT library

    # Flatten and sample colors
    source_colors = source.reshape(-1, 3).astype(float)
    target_colors = target.reshape(-1, 3).astype(float)

    # Sample for speed (full image too slow)
    n_samples = 1000
    source_sample = source_colors[np.random.choice(len(source_colors), n_samples)]
    target_sample = target_colors[np.random.choice(len(target_colors), n_samples)]

    # Compute cost matrix (Euclidean distance in LAB)
    M = ot.dist(source_sample, target_sample, metric='euclidean')

    # Sinkhorn transport
    T = ot.sinkhorn(
        np.ones(n_samples) / n_samples,
        np.ones(n_samples) / n_samples,
        M, reg
    )

    # Apply transport (simplified - full implementation more complex)
    return source  # Placeholder

---

Force-Directed Layout (Organic Scatter)

def force_directed_layout(images, canvas_size, iterations=100):
    """
    Organic layout using physics simulation.
    """
    # Initialize random positions
    for img in images:
        img.position = np.random.rand(2) * canvas_size
        img.velocity = np.zeros(2)

    canvas_center = np.array(canvas_size) / 2

    for _ in range(iterations):
        for img in images:
            force = np.zeros(2)

            # Repulsion from other images
            for other in images:
                if img != other:
                    diff = img.position - other.position
                    dist = np.linalg.norm(diff)
                    if dist < 1:
                        dist = 1
                    # Inverse square repulsion
                    force += diff / (dist ** 2) * 100

            # Attraction to center (prevent drift)
            center_diff = canvas_center - img.position
            force += center_diff * 0.01

            # Boundary repulsion
            for i in range(2):
                if img.position[i] < 50:
                    force[i] += 10
                if img.position[i] > canvas_size[i] - 50:
                    force[i] -= 10

            # Apply force with damping
            img.velocity = img.velocity * 0.9 + force * 0.1
            img.position += img.velocity

    return images

Collision Avoidance

def check_overlap(img1, img2):
    """Check if two positioned images overlap."""
    r1 = img1.get_rect()  # (x, y, w, h)
    r2 = img2.get_rect()

    return (r1.x < r2.x + r2.w and
            r1.x + r1.w > r2.x and
            r1.y < r2.y + r2.h and
            r1.y + r1.h > r2.y)

def resolve_overlap(img1, img2):
    """Push overlapping images apart."""
    diff = img1.position - img2.position
    dist = np.linalg.norm(diff)
    if dist < 1:
        diff = np.random.rand(2) - 0.5
        dist = np.linalg.norm(diff)

    # Move each image half the overlap distance
    overlap = get_overlap_distance(img1, img2)
    push = diff / dist * overlap / 2

    img1.position += push
    img2.position -= push

---

Bin Packing (Tight Grid Layout)

def guillotine_pack(images, canvas_width):
    """
    Pack images using guillotine algorithm.
    Returns positions for each image.
    """
    # Sort by height (tallest first)
    sorted_images = sorted(images, key=lambda x: -x.height)

    # Initialize free rectangles
    free_rects = [(0, 0, canvas_width, float('inf'))]
    positions = []

    for img in sorted_images:
        # Find best fit
        best_rect = None
        best_score = float('inf')

        for rect in free_rects:
            if rect[2] >= img.width and rect[3] >= img.height:
                score = rect[2] * rect[3]  # Area
                if score < best_score:
                    best_score = score
                    best_rect = rect

        if best_rect:
            # Place image
            positions.append((img, best_rect[0], best_rect[1]))

            # Split remaining space (guillotine cut)
            free_rects.remove(best_rect)
            # Right split
            if best_rect[2] - img.width > 0:
                free_rects.append((
                    best_rect[0] + img.width,
                    best_rect[1],
                    best_rect[2] - img.width,
                    img.height
                ))
            # Bottom split
            if best_rect[3] - img.height > 0:
                free_rects.append((
                    best_rect[0],
                    best_rect[1] + img.height,
                    best_rect[2],
                    best_rect[3] - img.height
                ))

    return positions

---

Performance Benchmarks

Art-Historical Collage Styles

Detailed implementations for historically-inspired collage techniques.

---

David Hockney's Joiners (1982-1985)

Hockney created photographs with "perspectival sophistication of Cubist paintings":

HOCKNEY_JOINER_STYLE = {
    'overlap': 0.1,              # 5-15% overlap between photos
    'rotation_variance': 2.0,    # ±2° rotation per photo
    'perspective_shift': True,   # Multiple viewpoints
    'grid_irregularity': 0.15,   # 10-15% positional offset
    'border_style': 'polaroid',  # White borders (optional)
    'allow_gaps': True,          # Intentional negative space
}

Key Innovations:

• Multiple perspectives simultaneously (vs. single camera viewpoint)
• Temporal dimension (same scene, different moments)
• Viewer's eye "constructs" the scene (active participation)
• Embraces imperfection (overlaps, gaps, misalignments)

Implementation Notes

The magic of Hockney joiners comes from: 1. Intentional misalignment - Don't perfect-stitch 2. Visible seams - Borders are part of the aesthetic 3. Multiple focal lengths - Shift attention through zoom variation 4. Temporal narrative - Capture same scene at different moments

---

Dadaist Photomontage (Hannah Höch, 1920s)

DADAIST_STYLE = {
    'layout': 'chaotic',
    'semantic_mismatch': True,  # Intentionally incongruous elements
    'sharp_cutouts': True,      # No feathering
    'scale_absurdity': True,    # Giant heads, tiny bodies
    'political_commentary': True,
}

Key Characteristics

• Sharp edges: No smooth transitions
• Jarring juxtapositions: Unrelated elements collide
• Scale distortion: Size relationships defied
• Found imagery: Newspapers, magazines, advertisements
• Political/social critique: Visual commentary

Implementation Approach

def dadaist_cutout(image, subject_mask):
    """
    Create Dadaist-style sharp cutout
    """
    # Hard edge, no feathering
    mask = subject_mask.astype(np.uint8) * 255

    # Optional: Add slight paper-tear effect to edges
    if random.random() > 0.7:
        kernel = np.ones((2, 2), np.uint8)
        mask = cv2.erode(mask, kernel, iterations=1)

    return cv2.bitwise_and(image, image, mask=mask)

---

Pop Art Combines (Rauschenberg, 1950s-60s)

RAUSCHENBERG_STYLE = {
    'layout': 'layered',
    'blend_modes': ['multiply', 'screen', 'overlay'],
    'found_imagery': True,      # Newspaper, ads, photos
    'paint_integration': True,  # Mix photo + paint texture
    'silkscreen_effect': True,
}

Key Characteristics

• Layered composition: Images stacked, semi-transparent
• Mixed media: Photography + painting + found objects
• Urban detritus: Street imagery, commercial artifacts
• Color registration errors: Intentional offset for silkscreen look

Silkscreen Effect

def silkscreen_effect(image, color_shift=(5, 3)):
    """
    Create Warhol/Rauschenberg silkscreen look
    """
    # Split channels
    b, g, r = cv2.split(image)

    # Offset each channel slightly
    rows, cols = b.shape
    M_r = np.float32([[1, 0, color_shift[0]], [0, 1, color_shift[1]]])
    M_b = np.float32([[1, 0, -color_shift[0]], [0, 1, -color_shift[1]]])

    r = cv2.warpAffine(r, M_r, (cols, rows))
    b = cv2.warpAffine(b, M_b, (cols, rows))

    # Recombine
    return cv2.merge([b, g, r])

---

Surrealist Assemblage

SURREALIST_STYLE = {
    'dreamlike_transitions': True,
    'impossible_juxtaposition': True,
    'seamless_blend': True,     # Unlike Dada's sharp cuts
    'perspective_manipulation': True,
}

Key Characteristics

• Dreamlike logic: Elements connected by unconscious association
• Seamless integration: Smooth blending (unlike Dada)
• Scale manipulation: Objects in impossible sizes
• Perspective warping: Shared vanishing points across disparate elements

Implementation Approach

def surrealist_blend(foreground, background, mask):
    """
    Seamless surrealist integration using Poisson blending
    """
    # Find center of foreground object
    M = cv2.moments(mask)
    cx = int(M['m10'] / M['m00'])
    cy = int(M['m01'] / M['m00'])

    # Seamless clone
    result = cv2.seamlessClone(
        foreground,
        background,
        mask,
        (cx, cy),
        cv2.NORMAL_CLONE
    )

    return result

---

Constructivist Montage (Rodchenko, 1920s)

CONSTRUCTIVIST_STYLE = {
    'layout': 'dynamic_diagonal',
    'typography_integration': True,
    'bold_geometry': True,
    'limited_palette': ['red', 'black', 'white'],
    'propaganda_aesthetic': True,
}

Key Characteristics

• Strong diagonals: Dynamic composition
• Bold typography: Text as design element
• Limited color palette: Red, black, white dominant
• Geometric shapes: Circles, triangles, bars as framing
• Worker imagery: Industrial, heroic subjects

---

Choosing a Historical Style

Collage Types & Techniques

Detailed code examples and configurations for each collage style.

---

1. Grid Collages

Use for: Instagram profiles, product showcases, team photos, systematic displays.

GRID_STYLES = {
    'uniform': {
        'rows': 3, 'cols': 3,
        'gap': 4,  # pixels
        'aspect': '1:1',
    },
    'masonry': {
        'columns': 3,
        'gap': 8,
        'variable_height': True,  # Pinterest-style
    },
    'mixed_grid': {
        'hero_size': 2,  # 2x2 for main image
        'small_count': 5,
        'layout': 'L_shape',  # or 'corner', 'split'
    },
}

Key considerations:

• Consistent color temperature across images
• Visual flow (Z-pattern or F-pattern for reading)
• One hero image as anchor; others support

---

2. Photo Mosaics

Use for: Tribute images, corporate displays, artistic recreations.

def create_photo_mosaic(target_image, tile_images, tile_size=32):
    """
    Each tile_image replaces a region of target_image
    based on average color matching.
    """
    # 1. Compute average color of each tile
    tile_colors = [avg_color(img) for img in tile_images]

    # 2. Build k-d tree for fast lookup
    color_tree = KDTree(tile_colors)

    # 3. For each grid cell in target
    for y in range(0, target.height, tile_size):
        for x in range(0, target.width, tile_size):
            region_color = avg_color(target[y:y+tile_size, x:x+tile_size])
            best_tile_idx = color_tree.query(region_color)
            place_tile(tile_images[best_tile_idx], x, y)

Expert tips:

• Tile size 20-40px for viewing distance balance
• Use LAB color space for perceptual matching
• Avoid repetition: track tile usage, penalize reuse
• Consider edge detection for structural preservation

---

3. Scrapbook & Digital Journal

Use for: Personal memories, travel journals, baby books, wedding albums.

SCRAPBOOK_ELEMENTS = {
    'photos': {'rotation_variance': (-5, 5), 'drop_shadow': True},
    'frames': ['polaroid', 'vintage', 'tape_corners', 'washi_tape'],
    'text': {'fonts': ['handwritten', 'typewriter', 'label_maker']},
    'embellishments': ['stickers', 'stamps', 'doodles', 'tickets'],
    'backgrounds': ['paper_texture', 'cork_board', 'fabric'],
}

Layer order (back to front): 1. Background texture/paper 2. Decorative elements (washi tape, ribbons) 3. Photos with frames/borders 4. Text blocks and labels 5. Small embellishments (stickers, stamps)

---

4. Magazine & Editorial Layouts

Use for: Professional publications, marketing materials, portfolios.

EDITORIAL_GRIDS = {
    '3_column': {'cols': 3, 'gutter': 20, 'margin': 40},
    '12_column': {'cols': 12, 'gutter': 16, 'margin': 48},  # Flexible
    'modular': {'rows': 6, 'cols': 6, 'baseline': 24},
}

# Text-image relationships
WRAP_STYLES = ['square', 'tight', 'through', 'top_bottom']

Typography integration:

• Headlines: contrast with imagery, never compete
• Body text: respect image boundaries, maintain gutter
• Pull quotes: can overlap images with proper contrast
• Captions: anchor to relevant image

---

5. Vision Boards & Mood Boards

Use for: Design inspiration, goal visualization, brand development.

MOOD_BOARD_LAYOUT = {
    'style': 'organic_cluster',  # or 'grid', 'radial', 'timeline'
    'overlap': 0.15,
    'rotation_range': (-8, 8),
    'scale_variation': (0.7, 1.3),
    'anchor_image': 'largest',  # Central focal point
    'color_coherence': 0.8,  # How matched colors should be
}

Curation principles:

• 60/30/10 rule: dominant/secondary/accent
• Mix scales: wide shots + details + textures
• Include non-photo elements: swatches, type samples, textures

---

6. Memory Walls & Polaroid Layouts

Use for: Nostalgic displays, event walls, family galleries.

POLAROID_STYLE = {
    'border': {'top': 8, 'sides': 8, 'bottom': 24},  # Classic Polaroid
    'caption_font': 'permanent_marker',
    'scatter': {
        'rotation': (-15, 15),
        'overlap_allowed': True,
        'pin_style': 'pushpin',  # or 'tape', 'clip', 'magnet'
    },
}

Arrangement algorithms:

• Force-directed: Images repel like particles, settle naturally
• Gravity clustering: Images fall toward anchor points
• Chronological spiral: Time-based arrangement outward

---

7. Social Media Collages

Use for: Instagram stories, carousel covers, Pinterest pins.

SOCIAL_TEMPLATES = {
    'instagram_story': {'width': 1080, 'height': 1920, 'safe_zone': 100},
    'instagram_post': {'width': 1080, 'height': 1080},
    'instagram_carousel': {'count': 10, 'continuity': True},  # Seamless swipe
    'pinterest_pin': {'width': 1000, 'height': 1500},
    'twitter_card': {'width': 1200, 'height': 628},
}

Platform-specific tips:

• Instagram: Avoid text in top/bottom 15% (UI overlap)
• Carousel: Create visual continuity across swipes
• Pinterest: Vertical images, text overlay in top third

Edge-Based Assembly Strategy

Core Concept: "Edge-First" Composition

The Insight: Photos connect at their edges, not by timestamp or random placement.

Edge Descriptor

@dataclass
class EdgeDescriptor:
    photo_id: UUID
    side: str  # 'top', 'bottom', 'left', 'right'
    region: np.ndarray  # 10% strip along edge

    # Geometric features
    lines: List[Line]              # Lines intersecting this edge
    curves: List[Curve]            # Curves at edge
    dominant_angle: float          # -90° to 90°
    complexity: float              # 0-1 (busy vs. clean)

    # Color features
    colors: ColorPalette           # 3-5 dominant colors in LAB
    gradient_direction: str        # 'lighter', 'darker', 'neutral'
    temperature: str               # 'warm', 'cool', 'neutral'

    # Semantic features
    clip_embedding: np.ndarray     # 512-dim CLIP of edge region
    detected_objects: List[str]    # ['sky', 'water', 'person_partial']

    # Match preferences
    blendability: float            # 0-1 (how well can this edge blend?)
    wants_continuation: bool       # Is something cut off?

---

Edge Compatibility Scoring

def edge_compatibility(edge1, edge2):
    """
    Score how well two edges can connect (0-1, higher = better).
    """
    scores = {}

    # GEOMETRIC: Lines/curves flow across boundary
    scores['line_continuation'] = (
        angle_alignment(edge1.lines, edge2.lines) * 0.4 +
        position_alignment(edge1.lines, edge2.lines) * 0.3 +
        multiple_line_bonus(edge1.lines, edge2.lines) * 0.3
    )

    scores['curve_flow'] = (
        tangent_match(edge1.curves, edge2.curves) * 0.5 +
        curvature_naturalness(edge1.curves, edge2.curves) * 0.5
    )

    # COLOR: Harmonious or complementary
    scores['color_harmony'] = compute_color_harmony(
        edge1.colors, edge2.colors, mode='edge_regions'
    )

    # SEMANTIC: Related content (CLIP similarity)
    scores['semantic_coherence'] = cosine_similarity(
        edge1.clip_embedding, edge2.clip_embedding
    )

    # BALANCE: Complexity contrast
    complexity_diff = abs(edge1.complexity - edge2.complexity)
    scores['complexity_balance'] = 1.0 - min(1.0, complexity_diff / 0.5)

    # Weighted combination
    return (
        0.30 * scores['line_continuation'] +
        0.15 * scores['curve_flow'] +
        0.25 * scores['color_harmony'] +
        0.20 * scores['semantic_coherence'] +
        0.10 * scores['complexity_balance']
    )

---

Angle Alignment

def angle_alignment(lines1, lines2, tolerance=15.0):
    """
    Check if dominant angles of two edge regions align.
    tolerance: degrees (15° is forgiving, 5° is strict)
    """
    if not lines1 or not lines2:
        return 0.0

    # Weighted average by line length and strength
    angle1 = weighted_average_angle(lines1)
    angle2 = weighted_average_angle(lines2)

    # Angular difference (accounting for ±180° equivalence)
    diff = abs(angle1 - angle2)
    diff = min(diff, 180 - diff)  # Handle wraparound

    # Score: 1.0 if perfect, 0.0 if > tolerance
    return max(0.0, 1.0 - diff / tolerance)

def weighted_average_angle(lines):
    """Calculate dominant angle weighted by line properties."""
    weights = [line.length * line.strength for line in lines]
    angles = [line.angle for line in lines]
    return np.average(angles, weights=weights)

---

Position Alignment

def position_alignment(lines1, lines2, edge_pair):
    """
    Check if lines align positionally across boundary.

    Example: For right edge of photo A and left edge of photo B,
             do horizontal lines have matching y-coordinates?
    """
    edge_type = edge_pair  # ('right', 'left') or ('top', 'bottom')

    if edge_type in [('right', 'left'), ('left', 'right')]:
        coord_dim = 'y'
    else:
        coord_dim = 'x'

    relevant_lines1 = filter_lines_by_orientation(lines1, edge_type[0])
    relevant_lines2 = filter_lines_by_orientation(lines2, edge_type[1])

    if not relevant_lines1 or not relevant_lines2:
        return 0.0

    coords1 = [get_boundary_coord(line, edge_type[0], coord_dim) for line in relevant_lines1]
    coords2 = [get_boundary_coord(line, edge_type[1], coord_dim) for line in relevant_lines2]

    # Find closest pairs and compute alignment score
    min_distances = []
    for c1 in coords1:
        min_dist = min(abs(c1 - c2) for c2 in coords2)
        min_distances.append(min_dist)

    avg_misalignment = np.mean(min_distances)

    # Score: 1.0 if perfect (<5px), 0.0 if terrible (>50px)
    return max(0.0, 1.0 - avg_misalignment / 50.0)

---

Assembly Algorithm: Greedy Edge Growth

def assemble_collage_greedy(seed_photo, photo_database, target_size=(10, 10)):
    """
    Build collage by iteratively adding photos to best-matching edges.
    """
    # 1. SEED SELECTION
    canvas = Canvas(target_size)
    canvas.place_photo(seed_photo, position='center', locked=True)

    # Priority queue of open edges (scored by "urgency")
    open_edges = PriorityQueue()
    for edge in seed_photo.edges:
        urgency = compute_edge_urgency(edge)
        open_edges.push(edge, priority=urgency)

    # 2. ITERATIVE GROWTH
    while canvas.coverage < 0.8 and not open_edges.empty():
        current_edge = open_edges.pop()

        # Query k best matches from database
        candidates = photo_database.find_compatible_edges(
            query_edge=current_edge,
            k=20,
            filters={
                'aspect_ratio': current_edge.compatible_aspect_ratios,
                'min_compatibility': 0.4
            }
        )

        # Try candidates in order of compatibility
        placed = False
        for candidate_photo in candidates:
            if canvas.would_overlap(candidate_photo):
                continue

            local_fit = edge_compatibility(current_edge, candidate_photo.opposite_edge)
            global_aesthetics = canvas.score_global_aesthetics_with(candidate_photo)

            if local_fit > 0.5 and global_aesthetics > 0.6:
                canvas.place_photo(candidate_photo, adjacent_to=current_edge)

                for new_edge in candidate_photo.new_open_edges:
                    urgency = compute_edge_urgency(new_edge)
                    open_edges.push(new_edge, priority=urgency)

                placed = True
                break

        if not placed:
            current_edge.relaxed = True
            open_edges.push(current_edge, priority=0.5)

    # 3. BOUNDARY REFINEMENT
    canvas.refine_boundaries(
        crop_for_alignment=True,
        blend_overlaps=True,
        inpaint_gaps=True,
        color_grade_globally=True
    )

    return canvas.render()

---

Edge Urgency Heuristic

def compute_edge_urgency(edge):
    """
    Determine which edges should be filled first.
    Higher urgency = fill sooner
    """
    urgency = 0.0

    # Strong lines → high urgency (want to continue them)
    if edge.has_strong_lines():
        urgency += 0.5

    # Cut-off objects → very high urgency (want completion)
    if edge.wants_continuation:
        urgency += 0.7

    # High aesthetic quality → high urgency
    urgency += edge.photo.aesthetic_score * 0.3

    # Central position → higher urgency (build from center out)
    distance_from_center = edge.distance_to_canvas_center()
    urgency += (1.0 - distance_from_center) * 0.2

    return urgency

---

Practical Optimizations

1. Hierarchical Clustering

Concept: Group similar photos into clusters, search within clusters first.

class PhotoDatabase:
    def __init__(self, photos):
        self.clusters = self._cluster_photos_hierarchically(photos)

    def _cluster_photos_hierarchically(self, photos):
        """
        Group photos into ~50-100 clusters using CLIP embeddings.
        Benefits: 50x speedup in matching
        """
        embeddings = np.array([p.clip_embedding for p in photos])

        from sklearn.cluster import AgglomerativeClustering
        clustering = AgglomerativeClustering(
            n_clusters=min(100, len(photos) // 100),
            metric='cosine',
            linkage='average'
        )
        labels = clustering.fit_predict(embeddings)

        clusters = {}
        for photo, label in zip(photos, labels):
            clusters.setdefault(label, []).append(photo)

        return clusters

2. Multi-Scale Matching

def find_matches_multiscale(query_edge, database):
    """
    Progressive refinement: fast coarse search, slow precise refinement.

    Total: 50ms instead of 500ms for all-full-res
    """
    # Stage 1: Coarse search on thumbnails
    candidates_coarse = database.search_thumbnails(
        query_edge.thumbnail_embedding, k=100
    )

    # Stage 2: Geometric filtering
    candidates_filtered = [
        c for c in candidates_coarse
        if abs(c.dominant_angle - query_edge.dominant_angle) < 30
    ]

    # Stage 3: Full-resolution scoring (top 20 only)
    candidates_scored = []
    for c in candidates_filtered[:20]:
        score = edge_compatibility_fullres(query_edge, c)
        candidates_scored.append((score, c))

    candidates_scored.sort(reverse=True, key=lambda x: x[0])
    return [c for score, c in candidates_scored[:10]]

3. Caching Good Pairs

class PairCache:
    """Learn from experience: which edges work well together?"""
    def __init__(self):
        self.successful_pairs = {}
        self.usage_counts = {}

    def record_success(self, edge1, edge2, score):
        pair_key = (edge1.id, edge2.id)
        self.successful_pairs[pair_key] = score
        self.usage_counts[pair_key] = self.usage_counts.get(pair_key, 0) + 1

    def boost_known_pairs(self, candidates, query_edge):
        for c in candidates:
            pair_key = (query_edge.id, c.edge_id)
            if pair_key in self.successful_pairs:
                boost = self.successful_pairs[pair_key] * 0.2
                boost += np.log1p(self.usage_counts[pair_key]) * 0.1
                c.score += boost
        return sorted(candidates, key=lambda c: c.score, reverse=True)

4. Pruning Generic Edges

def is_edge_generic(edge):
    """
    Generic edges (plain sky, solid colors) don't need expensive matching.
    """
    if edge.complexity < 0.2 and edge.blendability > 0.8:
        if len(edge.lines) < 2 and len(edge.colors.colors) <= 2:
            return True
    return False

5. Backtracking

def assemble_with_backtracking(seed, database, target_size):
    """Greedy growth with backtracking for difficult cases."""
    canvas = Canvas(target_size)
    canvas.place_photo(seed, position='center')

    history = []
    max_backtracks = 5

    while canvas.coverage < 0.8:
        edge = canvas.best_open_edge()
        candidates = database.find_compatible_edges(edge, k=20)

        placed = False
        for candidate in candidates:
            if canvas.can_place(candidate):
                canvas.place_photo(candidate, adjacent_to=edge)
                history.append((candidate, edge))
                placed = True
                break

        if not placed and len(history) > 0 and max_backtracks > 0:
            canvas.undo(history.pop())
            canvas.undo(history.pop())
            max_backtracks -= 1
            continue

        if not placed:
            edge.mark_skipped()

    return canvas

---

Performance Impact

Combined: 10-photo collage in 0.5-2 seconds instead of 50-200 seconds.

David Hockney's Joiners Technique (1982-1985)

Historical Context

Origins (1982):

• Curator Alain Sayag invited Hockney to Centre Pompidou (Paris) photography exhibition
• Breakthrough: Overcome photography's limitation of single perspective + frozen moment
• Started with Polaroid instant prints, creating grid-like compositions
• Later evolved to 35mm commercially processed prints with organic shapes

Technique Characteristics

Phase 1 - Grid Joiners (1982)

┌─────┬─────┬─────┐
│ POL │ POL │ POL │  ← Polaroid grid
├─────┼─────┼─────┤    Multiple viewpoints
│ POL │ POL │ POL │    Slight overlaps (~5-15%)
├─────┼─────┼─────┤    Subtle misalignments
│ POL │ POL │ POL │    Capturing time + space
└─────┴─────┴─────┘

Phase 2 - Organic Joiners (1984-1985)

• Compositions "took on a shape of their own"
• Less rigid structure, more painterly
• Influenced by Cubist paintings (Picasso, Braque)
• Intentional rotation variance (±2-3°)
• Grid irregularity (~10-15% positional variance)

Artistic Intent

Hockney's Goal: Create photographs with "perspectival sophistication of Cubist paintings"

Key Innovations: 1. Multiple perspectives simultaneously (vs. single camera viewpoint) 2. Temporal dimension (same scene, different moments) 3. Viewer's eye "constructs" the scene (active participation) 4. Embraces imperfection (overlaps, gaps, misalignments)

Computational Implementation

HOCKNEY_JOINER_STYLE = {
    'overlap': 0.1,                    # 5-15% overlap between photos
    'rotation_variance': 2.0,          # ±2° rotation per photo
    'perspective_shift': True,         # Multiple viewpoints
    'grid_irregularity': 0.15,         # 10-15% positional offset
    'border_style': 'polaroid',        # White borders (optional)
    'allow_gaps': True,                # Intentional negative space
}

Modern Interpretation:

• Hockney's manual Polaroid placement → Edge-based algorithmic assembly
• Visual intuition → CLIP semantic matching + geometric compatibility
• Trial-and-error → Greedy edge growth with intelligent optimizations
• Days/weeks per piece → Seconds to minutes with GPU acceleration

---

Art Historical References

Photographers and Artists to Study

1. David Hockney (1937-present)

• Joiners series (1982-1985)
• Cubist-inspired multiple perspectives
• Polaroid and 35mm collages
• Key works: "Pearblossom Hwy.", "The Scrabble Game"

2. Robert Rauschenberg (1925-2008)

• Combines (1950s-1960s)
• Mixed media: photos + painting + objects
• Layering and transparency
• Abstract + representational

3. Hannah Höch (1889-1978)

• Dada photomontage (1920s)
• Cut-and-paste magazine photos
• Juxtapose unrelated subjects
• Political/social commentary

4. John Baldessari (1931-2020)

• Conceptual photography
• Colored dots over faces
• Text + image combinations
• Systematic rules (all red objects, all circles)

5. Martha Rosler (1943-present)

• Critical photomontage
• "House Beautiful: Bringing the War Home" series
• Political commentary through juxtaposition

Style Implementations

ARTISTIC_STYLES = {
    'hockney_joiner': {
        'layout': 'irregular_grid',
        'overlap': (0.05, 0.15),
        'rotation_variance': (-3, 3),
        'scale_variance': (0.95, 1.05),
        'perspective_shift': True,
        'border': 'polaroid',  # White borders
        'allow_gaps': True,
    },

    'rauschenberg_combine': {
        'layout': 'layered',
        'overlap': (0.2, 0.5),
        'blend_modes': ['multiply', 'screen', 'overlay'],
        'texture_overlay': True,
        'abstract_elements': True,
    },

    'hoch_photomontage': {
        'layout': 'chaotic',
        'semantic_mismatch': True,  # Intentional surrealism
        'sharp_cutouts': True,
        'juxtaposition': 'unexpected',
    },

    'baldessari_conceptual': {
        'layout': 'systematic',
        'color_dots_on_faces': True,
        'thematic_constraints': True,  # e.g., "all blue objects"
        'text_overlay': True,
    },
}

Contemporary Trends (2025)

1. Maximalist

• Dense, abundant, ornate
• 15-30+ photos overlapping
• Nature horror vacui (fear of empty space)

2. Y2K Revival

• Early 2000s aesthetic
• Glitchy effects, chromatic aberration
• Metallic, holographic elements

3. Nostalgic Analog

• Film grain, light leaks
• Vintage color grading
• Polaroid borders, film strip edges

4. Brutalist

• Raw, unpolished
• Exposed grid structures
• Monochrome, high contrast

Practical Implementation Guide

Metal Shader Pipeline

1. Edge Extraction

kernel void extract_edge_region(
    texture2d<float, access::read> image [[texture(0)]],
    texture2d<float, access::write> edge_region [[texture(1)]],
    constant EdgeParams& params [[buffer(0)]],
    uint2 gid [[thread_position_in_grid]]
) {
    // Extract 10% strip along specified edge
    // ...
}

2. Line Detection (EDLines on GPU)

// Multi-pass: gradient → edge chains → line fitting
kernel void compute_gradients(...);
kernel void extract_edge_chains(...);
kernel void fit_line_segments(...);

3. Color Histogram

kernel void build_lab_histogram(
    texture2d<float, access::read> lab_image [[texture(0)]],
    device atomic_uint* histogram [[buffer(0)]],
    uint2 gid [[thread_position_in_grid]]
) {
    float3 lab = lab_image.read(gid).rgb;

    // Quantize to bins (8×8×8)
    uint l_bin = uint(lab.x / 100.0 * 8.0);
    uint a_bin = uint((lab.y + 128.0) / 256.0 * 8.0);
    uint b_bin = uint((lab.z + 128.0) / 256.0 * 8.0);

    uint bin_index = l_bin * 64 + a_bin * 8 + b_bin;
    atomic_fetch_add_explicit(&histogram[bin_index], 1, memory_order_relaxed);
}

4. Poisson Blending

kernel void poisson_jacobi_iteration(...);  // 50 iterations

---

Core ML Integration

Models Needed

1. MobileSAM (segmentation) - 5M params 2. CLIP ViT-B/32 (embeddings) - 150M params 3. MediaPipe Pose (gesture detection) - 3M params

Conversion

import coremltools as ct

# Convert PyTorch → Core ML
traced_model = torch.jit.trace(model, example_input)
mlmodel = ct.convert(traced_model, inputs=[...])
mlmodel.save("model.mlpackage")

---

Database Indexing

HNSW for CLIP embeddings

import hnswlib

# Initialize index
dim = 512  # CLIP dimension
index = hnswlib.Index(space='cosine', dim=dim)
index.init_index(max_elements=10000, ef_construction=200, M=16)

# Add embeddings
for i, embedding in enumerate(clip_embeddings):
    index.add_items(embedding, i)

# Query
k = 50
labels, distances = index.knn_query(query_embedding, k=k)

---

Performance Targets

---

Memory Management

Texture Compression

let descriptor = MTLTextureDescriptor()
descriptor.pixelFormat = .bc7_rgbaUnorm  // 6:1 compression

Lazy Loading

// Store only feature vectors in memory
// Load textures on-demand from disk
class ShardDatabase {
    var features: [UUID: ShardFeatures]  // In memory
    var texturePaths: [UUID: URL]        // On disk

    func loadTexture(id: UUID) -> MTLTexture {
        // Load PNG from disk when needed
    }
}

---

Algorithm Selection Guide

Line Detection

Layout Strategy

• Greedy Edge Growth (MVP, Phase 4): Primary algorithm
• Hierarchical Clustering: Essential optimization (50x speedup)
• Multi-Scale Matching: Progressive refinement (10x speedup)
• Simulated Annealing (Phase 6): Optional refinement
• Hockney Joiner Style: User explicitly requests

Color Harmonization

• Optimal Transport: Always use (mathematically principled)
• Affine Approximation: Real-time preview (fast)
• Full Sinkhorn: Final render (accurate)

Blending

• Poisson: Seamless photographic junctions
• Alpha Feathering: Simple overlaps, soft edges
• Diffusion Inpainting: Poor-quality junctions (expensive)

---

Error Handling

def place_shard_safe(canvas, shard, position):
    if canvas.would_overlap(shard, position):
        raise PlacementError("Overlap detected")

    if canvas.is_out_of_bounds(shard, position):
        raise PlacementError("Out of bounds")

    compatibility = canvas.check_neighbor_compatibility(shard, position)
    if compatibility < 0.3:
        logger.warning(f"Low compatibility: {compatibility:.2f}")

    canvas.place(shard, position)

---

Testing Strategies

def test_edge_alignment():
    """Verify lines align across boundaries."""
    photo1 = load_test_photo("horizon_left.jpg")
    photo2 = load_test_photo("horizon_right.jpg")

    edge1 = extract_edge_descriptor(photo1, 'right')
    edge2 = extract_edge_descriptor(photo2, 'left')

    assert len(edge1.lines) >= 1
    assert len(edge2.lines) >= 1

    angle_diff = abs(edge1.dominant_angle - edge2.dominant_angle)
    assert angle_diff < 5.0

def test_hockney_style():
    """Verify Hockney characteristics are present."""
    collage = create_collage(photos, style='hockney_joiner')

    positions = [s.position for s in collage.shards]
    irregularity = compute_grid_irregularity(positions)
    assert 0.1 < irregularity < 0.2

    rotations = [s.rotation for s in collage.shards]
    assert np.std(rotations) > 1.0

    overlaps = count_overlaps(collage)
    assert overlaps > 0

---

Python Dependencies

pip install opencv-python numpy scipy scikit-image transformers pot hnswlib

Line Detection Algorithms (State of the Art)

Algorithm Comparison (2025)

EDLines: Optimal Choice for Collage Assembly

Why EDLines for Edge-Based Collage Assembly:

1. Speed: 10x faster than LSD (critical for interactive generation) 2. Accuracy: Produces precise line segments with false detection control 3. No parameter tuning: Works out-of-box (vs. Hough's many parameters) 4. Edge-based: Aligns perfectly with "edge-first assembly" approach 5. Real-time: Suitable for live preview as users adjust parameters

EDLines Algorithm Overview

1. Edge Detection (Edge Drawing algorithm)
   - Fast gradient-based edge extraction
   - Produces clean edge chains (not noisy pixel maps)

2. Line Segment Fitting
   - Fit line segments to edge chains
   - Use least-squares fitting with error threshold
   - Validate line segments (reject false detections)

3. Output
   - List of line segments: [(x1, y1, x2, y2, angle, length, strength), ...]
   - Angle in degrees (-90 to 90)
   - Strength from gradient magnitude

Performance Benchmarks

• 1024×1024 image: ~10-15ms on M2 GPU
• 4K image: ~40-50ms on M2 GPU
• iPhone 15 Pro: ~20-30ms (1024×1024)

---

LSD (Line Segment Detector)

Use when: You need maximum accuracy over speed (e.g., final high-res render)

Characteristics:

• Gradient grouping approach
• Built-in false detection control (Helmholtz principle)
• Parameter-free (adaptive thresholds)
• Produces sub-pixel accurate line segments

Implementation

import cv2

# OpenCV includes LSD
lsd = cv2.createLineSegmentDetector(0)  # 0 = LSD_REFINE_NONE
lines, width, prec, nfa = lsd.detect(gray_image)

# lines: Nx1x4 array of [x1, y1, x2, y2]
# width: line widths
# nfa: Number of False Alarms (lower = more confident)

---

Hough Transform

Use when: Detecting specific geometric patterns (circles, ellipses) or teaching/legacy contexts

Classical Hough

import cv2
import numpy as np

# 1. Preprocess: Edge detection
edges = cv2.Canny(gray_image, 50, 150)

# 2. Hough Transform
lines = cv2.HoughLines(edges, rho=1, theta=np.pi/180, threshold=100)

# 3. Convert from (ρ, θ) to (x1, y1, x2, y2)
for rho, theta in lines:
    a, b = np.cos(theta), np.sin(theta)
    x0, y0 = a * rho, b * rho
    x1 = int(x0 + 1000 * (-b))
    y1 = int(y0 + 1000 * (a))
    x2 = int(x0 - 1000 * (-b))
    y2 = int(y0 - 1000 * (a))

Probabilistic Hough (faster variant)

lines = cv2.HoughLinesP(
    edges,
    rho=1,
    theta=np.pi/180,
    threshold=50,
    minLineLength=30,
    maxLineGap=10
)
# Returns line segments directly: [(x1, y1, x2, y2), ...]

---

When to Use Each Algorithm

Mathematical Foundations

Optimal Transport for Color Harmonization

Problem: Harmonize shard colors with global palette without destroying local structure.

Wasserstein Distance (Earth Mover's Distance)

W₂(μ, ν)² = inf{γ ∈ Π(μ,ν)} ∫∫ ‖x - y‖² dγ(x,y)

Where:

• μ = shard's color distribution (LAB histogram)
• ν = target/global distribution
• γ = transport plan (how to move color mass)

Sinkhorn Algorithm (entropy-regularized)

def sinkhorn_optimal_transport(source_hist, target_hist, epsilon=0.1, max_iters=100):
    """
    Compute optimal transport plan using Sinkhorn iterations.
    epsilon: regularization strength (smaller = closer to true OT)
    """
    # Cost matrix: squared distances in LAB space
    C = compute_cost_matrix_lab(source_hist.bins, target_hist.bins)

    # Kernel matrix
    K = np.exp(-C / epsilon)

    # Initialize
    u = np.ones(len(source_hist))
    v = np.ones(len(target_hist))

    # Iterate (converges exponentially fast)
    for _ in range(max_iters):
        u = source_hist.weights / (K @ v)
        v = target_hist.weights / (K.T @ u)

    # Optimal transport plan
    gamma = np.diag(u) @ K @ np.diag(v)

    return gamma  # gamma[i,j] = mass to move from bin i to bin j

Affine Approximation (for real-time)

def fit_affine_color_transform(source_hist, target_hist):
    """
    Approximate optimal transport as affine transform in LAB space.
    Returns: (M, b) where transformed_color = M @ color + b
    """
    # 1. Compute OT plan
    gamma = sinkhorn_optimal_transport(source_hist, target_hist)

    # 2. Sample points from distributions
    source_samples = source_hist.sample(n=256)
    target_samples = target_hist.sample(n=256)

    # 3. Weighted least squares
    X = source_samples  # Nx3 (L, a, b)
    Y = target_samples  # Mx3

    M = (Y.T @ gamma @ X.T) @ np.linalg.inv(X.T @ gamma.T @ X)
    b = target_hist.mean() - M @ source_hist.mean()

    return M, b

Why LAB Space

• Perceptually uniform: Euclidean distance ≈ perceived color difference
• Separates luminance from chrominance: L (lightness), a (green-red), b (blue-yellow)
• Better blending: Avoids hue shifts that occur in RGB

---

Poisson Blending for Seamless Junctions

Problem: Blend overlapping halos without visible seams.

Poisson Equation

∇²f = div(g)  in Ω
f = T         on ∂Ω

Where:

• f = unknown blended image
• g = guidance field (gradients from source images)
• Ω = blend region (halo intersection)
• ∂Ω = boundary (fixed to target values)

Discrete Form (pixel grid)

# For each interior pixel (i, j):
4·f[i,j] - f[i-1,j] - f[i+1,j] - f[i,j-1] - f[i,j+1] = div(g)[i,j]

Jacobi Iteration Solver

def poisson_blend_jacobi(source, target, mask, max_iters=50):
    """
    Solve Poisson equation using Jacobi iteration.
    Perfect for GPU parallelization (Metal shader).
    """
    # Compute guidance field (source gradients)
    gx = np.gradient(source, axis=1)
    gy = np.gradient(source, axis=0)

    # Divergence of guidance field
    div_g = np.gradient(gx, axis=1) + np.gradient(gy, axis=0)

    # Initialize solution with target
    f = target.copy()
    f_new = f.copy()

    # Iterate
    for iteration in range(max_iters):
        for i in range(1, mask.shape[0] - 1):
            for j in range(1, mask.shape[1] - 1):
                if mask[i, j]:  # Interior pixel
                    f_new[i, j] = 0.25 * (
                        f[i-1, j] + f[i+1, j] +
                        f[i, j-1] + f[i, j+1] +
                        div_g[i, j]
                    )
                # else: boundary pixel, keep f_new[i,j] = target[i,j]

        f = f_new.copy()

    return f

Metal Implementation (GPU acceleration)

kernel void poisson_jacobi_step(
    texture2d<float, access::read> f_current [[texture(0)]],
    texture2d<float, access::read> divergence [[texture(1)]],
    texture2d<float, access::write> f_next [[texture(2)]],
    texture2d<uint, access::read> mask [[texture(3)]],
    uint2 gid [[thread_position_in_grid]]
) {
    if (mask.read(gid).r == 0) {
        // Boundary: keep original
        f_next.write(f_current.read(gid), gid);
        return;
    }

    // Interior: Jacobi update
    float left  = f_current.read(gid + uint2(-1,  0)).r;
    float right = f_current.read(gid + uint2( 1,  0)).r;
    float down  = f_current.read(gid + uint2( 0, -1)).r;
    float up    = f_current.read(gid + uint2( 0,  1)).r;
    float div   = divergence.read(gid).r;

    float f_new = 0.25 * (left + right + down + up + div);

    f_next.write(float4(f_new, 0, 0, 0), gid);
}

Performance: ~20ms for 512×512 image on M2 GPU (50 iterations)

---

Energy Function for Composition Optimization

Total Energy

E(C) = α·E_semantic(C) + β·E_geometric(C) + γ·E_aesthetic(C)

1. Semantic Energy (CLIP similarity)

def compute_semantic_energy(canvas):
    """Reward semantically coherent adjacencies."""
    energy = 0.0

    for (i, j) in canvas.adjacent_pairs():
        similarity = cosine_similarity(
            canvas.shards[i].clip_embedding,
            canvas.shards[j].clip_embedding
        )
        energy -= similarity  # Negative: higher similarity → lower energy

    return energy / len(canvas.adjacent_pairs())

2. Geometric Energy (boundary compatibility)

def compute_geometric_energy(canvas):
    """Penalize geometric incompatibilities at junctions."""
    energy = 0.0

    for (i, j) in canvas.adjacent_pairs():
        # Tangent angle mismatch
        angle_diff = abs(canvas.tangent_angle[i] - canvas.tangent_angle[j])
        angle_diff = min(angle_diff, 180 - angle_diff)
        energy += (angle_diff / 180.0) ** 2

        # Curvature mismatch
        curv_diff = abs(canvas.curvature[i] - canvas.curvature[j])
        energy += curv_diff ** 2

    return energy / len(canvas.adjacent_pairs())

3. Aesthetic Energy (composition principles)

def compute_aesthetic_energy(canvas):
    """Classical aesthetic principles."""

    # Balance: visual weight distribution
    weights = [compute_visual_weight(s) for s in canvas.shards]
    quadrants = canvas.divide_into_quadrants()
    quadrant_weights = [sum(weights[s] for s in q) for q in quadrants]
    balance = np.var(quadrant_weights)

    # Symmetry
    symmetry = compute_symmetry(canvas)

    # Density variance
    density_grid = canvas.compute_density_grid(grid_size=10)
    density_variance = np.var(density_grid)

    # Rule of thirds
    thirds_score = compute_rule_of_thirds_score(canvas)

    return (
        0.3 * balance +
        0.2 * (1 - symmetry) +
        0.3 * density_variance +
        0.2 * (1 - thirds_score)
    )

Typical Weight Values

• α = 1.0: Semantic coherence is primary
• β = 0.5: Geometry important but secondary
• γ = 0.3: Aesthetics are subtle refinements

User Modes

• "Coherent": α=1.5, β=0.8, γ=0.2 (prioritize meaning)
• "Balanced": α=1.0, β=0.5, γ=0.3 (default)
• "Chaotic": α=0.2, β=0.1, γ=0.7 (prioritize aesthetics, allow surprises)

---

Aesthetic Principles

Rule of Thirds

def compute_rule_of_thirds_score(canvas):
    """Score how well composition follows rule of thirds."""
    thirds_points = [
        (1/3, 1/3), (1/3, 2/3),
        (2/3, 1/3), (2/3, 2/3)
    ]

    salient_shards = [s for s in canvas.shards if s.salience > 0.7]

    if not salient_shards:
        return 0.5

    scores = []
    for shard in salient_shards:
        center = shard.center_normalized()
        distances = [
            np.linalg.norm(np.array(center) - np.array(tp))
            for tp in thirds_points
        ]
        min_distance = min(distances)
        score = max(0.0, 1.0 - min_distance / 0.5)
        scores.append(score * shard.salience)

    return np.mean(scores)

Visual Weight

def compute_visual_weight(shard):
    """
    Visual weight considers:
    - Area (larger = heavier)
    - Contrast (higher contrast = heavier)
    - Color saturation (vibrant = heavier)
    - Semantic importance (faces = heavier)
    """
    weight = shard.area / 10000.0
    weight *= (1 + shard.contrast)
    weight *= (1 + shard.saturation)

    if shard.contains_face:
        weight *= 1.5

    return weight

Balance Score

def compute_balance(canvas):
    """Check if visual weight is distributed evenly."""
    quadrants = canvas.divide_into_quadrants()
    weights = [
        sum(compute_visual_weight(s) for s in q)
        for q in quadrants
    ]

    variance = np.var(weights)
    return max(0.0, 1.0 - variance / 10.0)

Golden Ratio

def check_golden_ratio(canvas):
    """Bonus if composition exhibits φ ≈ 1.618 proportions."""
    phi = (1 + np.sqrt(5)) / 2  # 1.618...

    aspect_ratio = canvas.width / canvas.height
    aspect_score = np.exp(-abs(aspect_ratio - phi))

    return aspect_score

Negative Space Quality

def compute_negative_space_quality(canvas):
    """
    High-quality negative space:
    - Exists (at least 20% of canvas)
    - Is simple/clean (low variance)
    - Is strategically placed
    """
    coverage = canvas.compute_coverage()
    negative_ratio = 1.0 - coverage

    if negative_ratio < 0.2:
        return 0.0  # Too crowded
    if negative_ratio > 0.6:
        return 0.0  # Too sparse

    negative_regions = canvas.extract_negative_space()
    simplicity = 1.0 - np.mean([np.var(r) for r in negative_regions])

    return simplicity