Data Visualization

Data Visualization Best Practices

Expert guidance for creating publication-quality scientific visualizations, avoiding common pitfalls, and optimizing figure clarity.

When to Use This Skill

• Creating figures for scientific publications
• Debugging misleading or distorted visualizations
• Optimizing figure layouts and element sizes
• Choosing appropriate plot types for data characteristics
• Ensuring statistical annotations fit properly
• Generating images for sharing with Claude or other AI tools

Supporting Files

This skill is organized into focused reference files. Load them as needed:

• [pitfalls-and-troubleshooting.md](pitfalls-and-troubleshooting.md) - Log-scale distortion, coordinate transform bugs, outlier handling philosophy, axis range optimization, float year labels
• [chart-recipes.md](chart-recipes.md) - Code recipes for temporal trends, boxplots, scatter plots, category proportions, stacked area charts, sample size legends, the dual-approach for outlier handling in publication figures
• [color-palettes.md](color-palettes.md) - Okabe-Ito palette, Paul Tol palette, sequential/diverging schemes, colorblind-safe implementation in matplotlib/seaborn
• [claude-image-constraints.md](claude-image-constraints.md) - Claude API 8000px limit, safe figure size presets, resize helpers, Jupyter notebook oversized image fixes
• [figure-descriptions.md](figure-descriptions.md) - Templates for writing publication-quality figure descriptions with proper statistical reporting
• [itol-reference.md](itol-reference.md) - iTOL dataset formats (DATASET_STYLE, DATASET_BINARY, DATASET_COLORSTRIP), species name synchronization, troubleshooting
• [journal-requirements.md](journal-requirements.md) - Journal-specific figure specs (Nature, Science, Cell, PLOS, ACS, IEEE, Elsevier, BMC): dimensions, DPI, formats, panel labeling, file naming

Assets (importable in notebooks/scripts)

• `assets/publication.mplstyle` - General publication style: plt.style.use('path/to/publication.mplstyle')
• `assets/nature.mplstyle` - Nature journal style (89mm single column, 7pt fonts, 600 DPI)
• `assets/presentation.mplstyle` - Larger fonts/lines for posters and slides
• `assets/color_palettes.py` - Importable palette definitions (Okabe-Ito, Wong, Paul Tol), apply_palette() helper, DNA base colors

Scripts (helper utilities)

• `scripts/figure_export.py` - save_publication_figure(), save_for_journal(), check_figure_size() - export in multiple formats with journal-specific DPI/format settings
• `scripts/style_presets.py` - apply_publication_style(), configure_for_journal(), set_color_palette() - one-command journal configuration

Core Principles

1. Always Check Log-Scale Plots

KDE-based plots (violin, ridge) on log axes produce distorted shapes. Use boxplots or log-transform data first, then plot on linear axes. See pitfalls-and-troubleshooting.md for details.

2. Show All Data First, Filter Later

Default to showing ALL data points in initial visualizations (showfliers=True). Outliers may be biologically meaningful. Only filter after review with domain expert, and always document exclusions.

3. Use Colorblind-Safe Palettes

Use Okabe-Ito palette (recommended by Nature) for categorical data. Combine color with marker shapes for redundancy. Never use red-green combinations. See color-palettes.md for hex codes and implementation.

4. Respect Claude's Image Size Limit

Images shared with Claude must not exceed 8000 pixels in either dimension. Use safe figure size presets and the save_figure() helper. See claude-image-constraints.md.

5. Position Annotations Carefully

Use pure data coordinates or ax.transAxes (0-1 range) for text positioning. Never mix coordinate systems (e.g., ax.get_xaxis_transform() with data-scale y-values). See pitfalls-and-troubleshooting.md.

Chart Selection Quick Guide

Publication Figure Checklist

Before Creating

• [ ] Choose colorblind-safe palette (Okabe-Ito recommended)
• [ ] Plan figure dimensions within Claude's 8000px limit
• [ ] Decide on panel layout (side-by-side vs stacked)

During Creation

• [ ] Include sample sizes in legends: Category (n=123)
• [ ] Use integer year labels: ax.xaxis.set_major_locator(plt.MaxNLocator(integer=True))
• [ ] Set explicit axis limits when adding annotations
• [ ] Reduce element sizes for dense data (s=25, alpha=0.5)
• [ ] Use bbox_inches='tight' when saving

After Creation

• [ ] Verify image dimensions (max 7999x7999 for Claude)
• [ ] Check annotations are within plot bounds
• [ ] Test colorblind accessibility
• [ ] Save at 300 DPI minimum for publication

For Temporal Analyses

• [ ] Create both all-data and cleaned versions
• [ ] Calculate statistics on FULL dataset (not cleaned)
• [ ] Document outlier removal method and retention rate
• [ ] Use clear file naming: figure.png vs figure_clean.png

Quick Reference: Safe Figure Sizes (300 DPI)

FIG_SIZES = {
    'single_column': (3.5, 4),      # 1050x1200 px
    'double_column': (7, 5),        # 2100x1500 px
    'full_page': (7, 9),            # 2100x2700 px
    'poster': (20, 15),             # 6000x4500 px
    'max_claude': (26, 26),         # 7800x7800 px
}

Quick Reference: Okabe-Ito Colors (3 Categories)

colors = {
    'Category_A': '#0072B2',    # Blue
    'Category_B': '#E69F00',    # Orange
    'Category_C': '#CC79A7'     # Reddish Purple
}

Best Practices Summary

1. Always check log-scale plots - Verify KDE-based plots against histograms 2. Test element sizes - Regenerate with different sizes for optimal clarity 3. Explicit axis limits - Don't rely on auto-limits when annotations are added 4. Consistent styling - Use seaborn context and style for publication consistency 5. High DPI - Save at 300 DPI minimum (dpi=300, bbox_inches='tight') 6. Optimize axis ranges - Zoom to data range when distributions are compressed 7. Check image dimensions - Verify size before sharing with Claude (max 7999x7999) 8. Set size constraints - Use safe figure sizes when generating images programmatically 9. Temporal trends with outliers - Create both cleaned (publication) and full (verification) versions 10. Include sample sizes - Always show n= in legends for comparative figures

References

• Matplotlib documentation: https://matplotlib.org/
• Seaborn visualization: https://seaborn.pydata.org/
• iTOL documentation: https://itol.embl.de/help.cgi
• Okabe-Ito palette: https://jfly.uni-koeln.de/color/
• ColorBrewer: https://colorbrewer2.org

Chart Recipes

Code recipes for common scientific visualization patterns.

Publication Figure Refinement

Element Sizing for Clarity

When figures are cluttered or elements overlap:

# Point sizes: Reduce for dense data
ax.scatter(..., s=25, alpha=0.5)  # Down from s=60

# Line widths: Thinner lines reduce visual clutter
ax.plot(..., linewidth=1.5)  # Down from 2.5

# Text sizes: Prevent overlap
ax.text(..., fontsize=8)  # Down from 10

# Error bar cap sizes
ax.errorbar(..., capsize=5)  # Standard readable size

P-value and Annotation Positioning

Problem: Statistical annotations (p-values, significance stars) often placed outside plot bounds

Solution: Position relative to data range with explicit limits

# Calculate data range
y_max = max([d.max() for d in data_list])
y_min = min([d.min() for d in data_list])

# Position annotations within plot
y_pos = y_max * 0.92  # 92% of max, not 105% which goes outside

ax.text(x_pos, y_pos, 'p < 0.001***', ha='center', fontsize=9)

# Set explicit limits with headroom
ax.set_ylim(y_min * 0.95 if y_min > 0 else -5, y_max * 1.05)

Panel Layout Testing

Test both orientations to find clearest presentation:

# Side-by-side (good for comparing distributions)
fig, axes = plt.subplots(1, 2, figsize=(16, 7))

# Stacked vertically (good for larger individual panels)
fig, axes = plt.subplots(2, 1, figsize=(10, 14))

Decision criteria:

• Side-by-side: Better for direct left-right comparison
• Stacked: Better when each panel needs more space
• Let user feedback guide the choice

Adding Sample Sizes to Legends

Why Sample Sizes Matter: Readers need to assess statistical power at a glance. Include sample sizes directly in legend labels for scientific figures.

Pattern 1: Simple Legend with Sample Sizes

# Calculate sample sizes once
category_sizes = df.groupby('category').size().to_dict()

# Use in scatter plot legend
for category in categories:
    data = df[df['category'] == category]
    ax.scatter(data['x'], data['y'],
              label=f"{category} (n={category_sizes[category]})")

ax.legend(loc='best')

Pattern 2: Custom Legend for Complex Plots

When you have multiple marker types (e.g., technology + category), create custom legend:

from matplotlib.lines import Line2D

# Calculate sizes
category_sizes = df.groupby('category').size().to_dict()

# Create custom legend elements
custom_lines = [
    Line2D([0], [0], color=colors['Cat1'], marker='o', linestyle='', markersize=8),
    Line2D([0], [0], color=colors['Cat2'], marker='o', linestyle='', markersize=8),
]

custom_labels = [
    f"Category 1 (n={category_sizes['Cat1']})",
    f"Category 2 (n={category_sizes['Cat2']})",
]

ax.legend(custom_lines, custom_labels, loc='best', fontsize=9)

Pattern 3: Multi-Panel Figures - Show Legend Once

For 2x3 or similar grids, show legend only in first subplot:

# Calculate once, use in all panels
category_sizes = df.groupby('category').size().to_dict()

for idx, metric in enumerate(metrics):
    ax = axes[idx]

    for category in categories:
        # Only add label for first subplot
        if idx == 0:
            label_text = f"{category} (n={category_sizes[category]})"
        else:
            label_text = ''

        ax.scatter(..., label=label_text)

    if idx == 0:
        ax.legend(loc='best')

Best Practices:

• Calculate sizes once at the top (efficient, avoids repeated computation)
• Use consistent format: Category Name (n=123)
• For small panels, use fontsize=7-9
• Consider ncol=1 for vertical layout if space allows
• Place sample sizes in legend OR as text annotations, not both

Publication Standards:

• Nature/Science: Strongly recommended for all comparative figures
• PLOS: Required for sample size transparency
• Cell: Expected in methods or figure legends
• General guideline: Always include when comparing groups

Example: Temporal Analysis with Categories

# Temporal trends by category
category_sizes = df.groupby('category').size().to_dict()

fig, ax = plt.subplots(figsize=(10, 6))

for category in ['Phased+Dual', 'Phased+Single', 'Pri/alt+Single']:
    data = df[df['category'] == category]
    ax.scatter(data['year'], data['quality_metric'],
              color=colors[category],
              label=f"{category} (n={category_sizes[category]})",
              alpha=0.6, s=40)

ax.set_xlabel('Year')
ax.set_ylabel('Assembly Quality')
ax.legend(loc='best', fontsize=9)

Visualizing Category Proportions Over Time

Use Case: Show how the relative proportions of categories have changed over time.

Dual-Panel Approach: Proportions + Absolute Counts

Show both relative and absolute trends using side-by-side panels.

Left Panel: Stacked area chart (proportions sum to 100%) Right Panel: Stacked bar chart (shows actual sample sizes)

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# Calculate counts and proportions by year
year_category_counts = df.groupby(['year', 'category']).size().unstack(fill_value=0)
year_category_proportions = year_category_counts.div(
    year_category_counts.sum(axis=1), axis=0
) * 100

# Dual-panel figure
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(16, 6))

# Panel 1: Stacked area (proportions)
years = year_category_proportions.index
categories = ['Cat1', 'Cat2', 'Cat3']

# Calculate total sizes for legend
total_counts = df.groupby('category').size().to_dict()

bottom = np.zeros(len(years))
for category in categories:
    values = year_category_proportions[category].values
    ax1.fill_between(years, bottom, bottom + values,
                     label=f"{category} (n={total_counts[category]})",
                     color=colors[category], alpha=0.7)
    bottom += values

ax1.set_xlabel('Year', fontsize=12)
ax1.set_ylabel('Proportion (%)', fontsize=12)
ax1.set_title('Category Proportions Over Time', fontsize=14, fontweight='bold')
ax1.set_ylim(0, 100)
ax1.legend(loc='best', fontsize=10)
ax1.grid(axis='y', alpha=0.3)
ax1.xaxis.set_major_locator(plt.MaxNLocator(integer=True))

# Panel 2: Stacked bar (absolute counts)
year_category_counts[categories].plot(
    kind='bar', stacked=True, ax=ax2,
    color=[colors[c] for c in categories],
    width=0.7, edgecolor='black', linewidth=0.5
)

ax2.set_xlabel('Year', fontsize=12)
ax2.set_ylabel('Number of Samples', fontsize=12)
ax2.set_title('Absolute Counts by Category', fontsize=14, fontweight='bold')
ax2.legend(title='Category',
          labels=[f"{c} (n={total_counts[c]})" for c in categories],
          loc='upper left', fontsize=9)
ax2.grid(axis='y', alpha=0.3)
ax2.set_xticklabels([int(y) for y in years], rotation=0)

# Add totals on top of bars
for i, year in enumerate(years):
    total = year_category_counts.loc[year].sum()
    ax2.text(i, total + 2, str(int(total)),
            ha='center', va='bottom', fontsize=9, fontweight='bold')

plt.tight_layout()
plt.savefig('category_proportions.png', dpi=150, bbox_inches='tight')

Why Both Panels?

Proportions (Area Chart):

• Shows relative shifts in category usage
• Easy to see if one category is growing/declining
• Sums to 100% (intuitive interpretation)

Absolute Counts (Bar Chart):

• Shows actual sample sizes (statistical power)
• Reveals total data volume changes
• Helps interpret proportion changes (growing proportion of shrinking pie?)

Together: Complete picture of temporal trends

Styling Tips:

• Colors: Use colorblind-safe palette consistently across both panels
• Edge colors: Black edges on bars improve readability (linewidth=0.5)
• Totals: Add count labels above stacked bars for context
• X-axis: Integer years, not decimals (use MaxNLocator(integer=True))
• Legend: Include total sample sizes: Category (n=123)

When to Use:

• Tracking category adoption over time
• Showing methodology shifts in field
• Demonstrating changing experimental approaches
• Any temporal categorical composition analysis

Temporal Trends with Boxplots vs Scatter Plots

Challenge: Visualizing temporal trends with multiple categories and overlapping data points

Solution Progression: 1. Scatter plots - Initial approach, but overlapping points obscure distributions 2. Boxplots grouped by year - Better visibility of distributions, clear quartiles 3. Boxplots with outlier removal - Cleaner visualization for identifying trends

Why Boxplots for Temporal Analysis?

Advantages over scatter plots:

• Shows distribution at each timepoint (median, quartiles, outliers)
• Groups by year and category simultaneously
• Better visibility than overlapping scatter points
• Easier to spot temporal trends in median values
• Quantifies variability within each year

Example: Boxplots Grouped by Year and Category

import matplotlib.pyplot as plt
import numpy as np

# Prepare data grouped by year and category
years = df['year'].unique()
categories = ['Cat_A', 'Cat_B', 'Cat_C']

fig, axes = plt.subplots(2, 3, figsize=(15, 10))
metrics = ['metric1', 'metric2', 'metric3', 'metric4', 'metric5', 'metric6']

for idx, metric in enumerate(metrics):
    ax = axes.flatten()[idx]

    # Prepare boxplot data
    boxplot_data = []
    boxplot_positions = []
    boxplot_colors = []

    for year_idx, year in enumerate(years):
        for cat_idx, category in enumerate(categories):
            data = df[(df['year'] == year) & (df['category'] == category)][metric]
            if len(data) > 0:
                boxplot_data.append(data.values)
                # Position: year_idx * 4 + cat_idx (spacing between years)
                boxplot_positions.append(year_idx * 4 + cat_idx)
                boxplot_colors.append(colors[category])

    # Create boxplots
    bp = ax.boxplot(boxplot_data, positions=boxplot_positions, widths=0.6,
                   patch_artist=True, showfliers=True,
                   boxprops=dict(linewidth=1.5),
                   medianprops=dict(color='black', linewidth=2))

    # Color boxes by category
    for patch, color in zip(bp['boxes'], boxplot_colors):
        patch.set_facecolor(color)
        patch.set_alpha(0.7)

    # Format x-axis with years
    ax.set_xticks([i * 4 + 1 for i in range(len(years))])
    ax.set_xticklabels(years)
    ax.set_xlabel('Year')
    ax.set_ylabel(metric)

plt.tight_layout()

Positioning logic: year_idx * 4 + cat_idx creates groups of 3 categories per year with spacing between years.

Outlier Removal for Cleaner Visualization

When to remove outliers: When extreme values obscure trends in the bulk of the data.

IQR Method (1.5x multiplier):

# Remove outliers using IQR method
def remove_outliers_iqr(data, metric):
    """Remove outliers using IQR method (1.5x multiplier)"""
    q1 = data[metric].quantile(0.25)
    q3 = data[metric].quantile(0.75)
    iqr = q3 - q1
    upper_threshold = q3 + 1.5 * iqr
    lower_threshold = q1 - 1.5 * iqr

    clean_data = data[
        (data[metric] >= lower_threshold) &
        (data[metric] <= upper_threshold)
    ]

    # Document retention rate
    retention = len(clean_data) / len(data) * 100
    print(f"{metric}: Retained {len(clean_data)}/{len(data)} ({retention:.1f}%)")

    return clean_data

# Apply to all metrics
df_clean = df.copy()
for metric in metrics:
    df_clean = remove_outliers_iqr(df_clean, metric)

Generate Both Versions:

1. Figure with all data (outliers included) - Shows full distribution 2. Figure with clean data (outliers removed) - Better for trend visualization

# Figure 1: All data
plt.savefig('temporal_trends_all_data.png', dpi=150, bbox_inches='tight')

# Figure 2: Clean data (outliers removed)
# Use df_clean instead of df
plt.savefig('temporal_trends_clean.png', dpi=150, bbox_inches='tight')

Adding Regression Statistics

For the outlier-removed version, calculate R-squared for linear trends:

from scipy.stats import linregress

# Calculate R-squared for each metric x category combination
regression_stats = []

for metric in metrics:
    for category in categories:
        # Get data for this metric and category
        data = df_clean[df_clean['category'] == category]
        x = data['year'].values
        y = data[metric].values

        if len(x) > 2:  # Need at least 3 points for regression
            slope, intercept, r_value, p_value, std_err = linregress(x, y)
            r_squared = r_value ** 2

            regression_stats.append({
                'metric': metric,
                'category': category,
                'r_squared': r_squared,
                'p_value': p_value,
                'slope': slope,
                'n_points': len(x),
                'significant': p_value < 0.05
            })

# Save to CSV
import pandas as pd
stats_df = pd.DataFrame(regression_stats)
stats_df.to_csv('regression_statistics_clean.csv', index=False)

# Report significant trends
sig_trends = stats_df[stats_df['significant']]
print(f"\nSignificant temporal trends: {len(sig_trends)}/{len(regression_stats)}")

Interpretation Example:

From real temporal analysis (HiFi assemblies, 2021-2025):

• 18 metric x category combinations tested
• Only 1 showed strong significant trend: Pri/alt+Single Scaffold N50 (R-squared=0.937, p=0.007)
• Conclusion: Quality is methodology-determined, not temporally-dependent
• Validation: Pooling assemblies across years is valid for comparative analysis

Chart Type Selection Guide

When to use scatter plots:

• Small datasets (< 50 points per year)
• Want to show individual data points
• Continuous time variable (not discrete years)

When to use boxplots:

• Large datasets (overlapping points)
• Discrete timepoints (years, quarters)
• Want to emphasize distributions and quartiles
• Multiple categories to compare

When to remove outliers:

• For visualization clarity (generate both versions)
• For regression analysis (outliers distort trends)
• Don't remove for reporting distributions
• Don't remove without documenting retention rate

Documentation:

• Always report retention rate after outlier removal
• Generate both versions (all data + clean data)
• Document IQR multiplier used (1.5x is standard)
• Include R-squared statistics for clean data version

Temporal Trend Figures: The Dual Approach

The Challenge

Scatter plots showing temporal trends often have outliers that:

• Obscure the overall pattern
• Make regression lines hard to interpret
• Reduce visual clarity for reviewers

But removing outliers raises concerns about cherry-picking data.

The Dual Approach Solution

Create TWO versions for different purposes:

1. Figure with all data: For initial analysis and verification 2. Figure with cleaned data: For publication and presentation

Key requirement: Use different approaches for visualization vs. statistics

Implementation Pattern

import matplotlib.pyplot as plt
import seaborn as sns
from scipy.stats import spearmanr

def plot_temporal_trends(df, remove_outliers=False, output_suffix=''):
    """
    Plot temporal trends with optional outlier removal

    Parameters:
    -----------
    df : DataFrame with year and metric columns
    remove_outliers : bool, if True removes points beyond 1.5x IQR
    output_suffix : str, added to filename (e.g., '_clean')
    """

    if remove_outliers:
        # Define outliers by metric
        for metric in ['scaffold_n50', 'gap_density']:
            Q1 = df[metric].quantile(0.25)
            Q3 = df[metric].quantile(0.75)
            IQR = Q3 - Q1

            # Keep only points within 1.5x IQR
            df = df[
                (df[metric] >= Q1 - 1.5 * IQR) &
                (df[metric] <= Q3 + 1.5 * IQR)
            ]

    # Create scatter plot with regression line
    fig, axes = plt.subplots(2, 3, figsize=(15, 10))

    for ax, metric in zip(axes.flat, metrics):
        # Scatter plot by category
        for category in categories:
            subset = df[df['category'] == category]
            ax.scatter(subset['year'], subset[metric],
                      label=category, alpha=0.6)

            # Add regression line
            z = np.polyfit(subset['year'], subset[metric], 1)
            p = np.poly1d(z)
            ax.plot(subset['year'], p(subset['year']), '--')

            # Add statistics (calculated on FULL dataset elsewhere)
            rho, pval = spearmanr(subset['year'], subset[metric])
            ax.text(0.05, 0.95, f'rho={rho:.2f}, p={pval:.3f}',
                   transform=ax.transAxes, va='top')

        ax.set_xlabel('Year')
        ax.set_ylabel(metric)
        ax.legend()

    plt.tight_layout()
    plt.savefig(f'temporal_trends{output_suffix}.png', dpi=300)
    plt.close()

# Create both versions
plot_temporal_trends(df_full, remove_outliers=False, output_suffix='')
plot_temporal_trends(df_full, remove_outliers=True, output_suffix='_clean')

Calculate Statistics Separately

CRITICAL: Statistics should ALWAYS use the full dataset, even if figures show cleaned data.

# Statistics on FULL dataset (no outlier removal)
def calculate_temporal_statistics(df_full):
    """Calculate Spearman correlations on complete dataset"""
    results = []

    for category in categories:
        subset = df_full[df_full['category'] == category]
        for metric in metrics:
            rho, pval = spearmanr(subset['year'], subset[metric])
            results.append({
                'category': category,
                'metric': metric,
                'rho': rho,
                'p_value': pval,
                'n': len(subset)
            })

    return pd.DataFrame(results)

# Use FULL dataset
stats_df = calculate_temporal_statistics(df_full)
stats_df.to_csv('temporal_statistics_full_dataset.csv')

File Naming Convention

Use clear naming to distinguish versions:

temporal_trends.png              # All data (for analysis)
temporal_trends_clean.png        # Cleaned (for publication)
temporal_statistics.csv          # Stats from FULL dataset
regression_statistics_clean.csv  # Regression from cleaned figure

Documentation Requirements

If using cleaned figures with full-dataset statistics:

In figure caption:

Outliers removed for clarity (points beyond 1.5x IQR from quartiles).
\textbf{Note}: Statistical tests use Spearman correlation (rho) on the
full dataset (n=XXX) for conservative assessment; this figure shows
cleaned data for visual clarity only.

In Methods section:

\textbf{Visualization}: Scatter plots show cleaned data (outliers
beyond 1.5x IQR removed). Statistical tests use Spearman correlation
on complete dataset for conservative assessment.

When to Use This Approach

Use cleaned figures when:

• Many outliers obscure the main pattern
• Target audience needs quick visual interpretation (papers, talks)
• Statistics are robust to outliers (Spearman, Kendall)
• You document the dual approach explicitly

Keep all points when:

• Few outliers (< 5% of data)
• Outliers are scientifically interesting
• Sample size is small (n < 50)
• Using parametric statistics sensitive to outliers

Quality Check

Before finalizing:

# Verify outlier counts
n_total = len(df_full)
n_clean = len(df_clean)
n_removed = n_total - n_clean
pct_removed = 100 * n_removed / n_total

print(f"Total points: {n_total}")
print(f"Removed: {n_removed} ({pct_removed:.1f}%)")
print(f"Retained: {n_clean} ({100-pct_removed:.1f}%)")

# Should typically remove < 10% of points
assert pct_removed < 10, "Too many outliers removed!"

Image Size Constraints for Claude API

CRITICAL: When generating images to share with Claude (for review, debugging, etc.), images must not exceed 8000 pixels in either dimension.

Check Image Size Before Opening

Always verify image dimensions before trying to display them in Claude:

from PIL import Image

# Check dimensions
img = Image.open('figure.png')
print(f"Image size: {img.width}x{img.height}")

if img.width > 8000 or img.height > 8000:
    print(f"WARNING: Image too large for Claude API!")
    print(f"   Claude limit: 8000px max dimension")
    print(f"   Your image: {img.width}x{img.height}")

Set Size Constraints When Generating Figures

For matplotlib/seaborn figures:

import matplotlib.pyplot as plt

# Set figure size to stay under Claude's limits
# Rule of thumb: Keep figsize under (80, 80) at 100 DPI
# Or under (26, 26) at 300 DPI
fig, ax = plt.subplots(figsize=(16, 12))  # Safe: 1600x1200 at 100 DPI

# When saving, control DPI to stay under limits
# 7999px / 300 DPI = 26.6 inches max
# 7999px / 100 DPI = 79.9 inches max
plt.savefig('figure.png', dpi=300, bbox_inches='tight')  # Max ~26x26 inches

# For very large figures, use lower DPI
plt.savefig('large_figure.png', dpi=100, bbox_inches='tight')  # Max ~80x80 inches

Safe figure size presets:

# Publication quality (300 DPI) - fits Claude limit
FIG_SIZES = {
    'single_column': (3.5, 4),      # 1050x1200 px
    'double_column': (7, 5),        # 2100x1500 px
    'full_page': (7, 9),            # 2100x2700 px
    'poster': (20, 15),             # 6000x4500 px - safe for Claude
    'max_claude': (26, 26),         # 7800x7800 px - maximum safe size
}

fig, ax = plt.subplots(figsize=FIG_SIZES['double_column'])
plt.savefig('figure.png', dpi=300, bbox_inches='tight')

Resize Oversized Images

If you have an existing image that's too large:

from PIL import Image

def resize_for_claude(image_path, max_dim=7999, output_path=None):
    """
    Resize image to fit Claude's API constraints.

    Args:
        image_path: Path to input image
        max_dim: Maximum dimension (default 7999 for safety margin)
        output_path: Output path (default: adds '_resized' to filename)
    """
    img = Image.open(image_path)

    # Check if resize needed
    if img.width <= max_dim and img.height <= max_dim:
        print(f"Image OK: {img.width}x{img.height}")
        return image_path

    # Calculate new size preserving aspect ratio
    img.thumbnail((max_dim, max_dim), Image.Resampling.LANCZOS)

    # Save
    if output_path is None:
        base = image_path.rsplit('.', 1)[0]
        ext = image_path.rsplit('.', 1)[1]
        output_path = f"{base}_resized.{ext}"

    img.save(output_path)
    print(f"Resized: {image_path}")
    print(f"  Original: {Image.open(image_path).size}")
    print(f"  New: {img.size}")
    print(f"  Saved: {output_path}")

    return output_path

# Usage
resize_for_claude('large_figure.png')

Quick Checks

Bash one-liner to check size:

# Using ImageMagick
identify figure.png | grep -o '[0-9]*x[0-9]*'

# Check if oversized
python3 -c "from PIL import Image; img=Image.open('figure.png'); print(f'{img.width}x{img.height}'); exit(0 if img.width<=7999 and img.height<=7999 else 1)" && echo "OK" || echo "TOO LARGE"

Add to notebook imports:

# Standard imports for Claude-compatible figures
import matplotlib.pyplot as plt
import seaborn as sns
from PIL import Image

# Set global figure size limit
plt.rcParams['figure.max_open_warning'] = 50
MAX_CLAUDE_DIM = 7999  # Claude API limit: 8000px, use 7999 for safety

def save_figure(filename, dpi=300, **kwargs):
    """Save figure with Claude size constraint check."""
    plt.savefig(filename, dpi=dpi, bbox_inches='tight', **kwargs)

    # Verify size
    img = Image.open(filename)
    if img.width > MAX_CLAUDE_DIM or img.height > MAX_CLAUDE_DIM:
        print(f"WARNING: {filename} exceeds Claude limit!")
        print(f"   Size: {img.width}x{img.height} (max: {MAX_CLAUDE_DIM})")
        print(f"   Resizing...")
        img.thumbnail((MAX_CLAUDE_DIM, MAX_CLAUDE_DIM), Image.Resampling.LANCZOS)
        img.save(filename)
        print(f"   Resized to: {img.width}x{img.height}")
    else:
        print(f"Saved {filename}: {img.width}x{img.height}")

Common Scenarios

High-DPI screenshots from Retina displays:

• Retina screenshots are 2x pixel density
• A full-screen 4K monitor screenshot can be 7680x4320 (OK)
• A 5K monitor screenshot is 10240x5760 (TOO LARGE!)
• Solution: Resize before sharing or take partial screenshots

Multi-panel figures:

# Instead of one huge figure with many panels
fig, axes = plt.subplots(4, 4, figsize=(40, 40))  # Could be 12000x12000 px!

# Split into smaller figures
for i in range(4):
    fig, axes = plt.subplots(2, 2, figsize=(12, 12))  # 3600x3600 px - safe!
    # Plot subset of panels
    plt.savefig(f'figure_part{i}.png', dpi=300, bbox_inches='tight')

Error Recovery

If you get the error:

API Error: 400 ... image dimensions exceed max allowed size: 8000 pixels

The error is stuck in conversation history. To recover:

1. Skip the message: "Please ignore the oversized image in the previous message" 2. Resize and resend: Use resize_for_claude() function above 3. Use /safe-clear: Save context and start fresh (if command available)

Jupyter Notebook Image Size Issues

Oversized Images from Combined Output

Problem: Jupyter notebook saves figures as extremely tall images (e.g., 1541 x 42,011 pixels) that exceed the 8000 pixel limit.

Cause: When a cell generates both a figure AND text output (print statements, statistical results), Jupyter captures both as a single tall image. The text output is rendered as image pixels below the figure, creating a massive combined image.

Symptoms:

• Image dimensions like 1541 x 42,011 pixels (height >> 8000)
• Figure displays fine in notebook but won't display in Claude or other tools
• Error: "image dimensions exceed max allowed size: 8000 pixels"

Example of the problem:

# Cell that creates oversized image
fig, axes = plt.subplots(2, 3, figsize=(12, 8))

# ... plotting code ...
plt.tight_layout()
plt.savefig('figure.png', dpi=150, bbox_inches='tight')
plt.show()

# Text output after figure (PROBLEM!)
print("Statistical Results:")
print(f"Spearman correlation: rho={rho:.3f}, p={pval:.4f}")
# Multiple print statements create tall text output
# Jupyter combines this with figure into one 42K pixel tall image

Solution 1: Split into multiple figures

Instead of creating one large multi-panel figure, split into smaller figures:

# GOOD: Split 2x3 grid into two 1x3 grids
# Figure 1: First 3 panels
fig1, axes1 = plt.subplots(1, 3, figsize=(10, 3.5))
# ... plot first 3 panels ...
plt.savefig('figure_part1.png', dpi=150, bbox_inches='tight')
plt.show()

# Figure 2: Second 3 panels
fig2, axes2 = plt.subplots(1, 3, figsize=(10, 3.5))
# ... plot second 3 panels ...
plt.savefig('figure_part2.png', dpi=150, bbox_inches='tight')
plt.show()

Solution 2: Separate text output into different cell

Move print statements to a separate cell after the figure:

# Cell 1: Just the figure
fig, axes = plt.subplots(2, 3, figsize=(12, 8))
# ... plotting code ...
plt.savefig('figure.png', dpi=150, bbox_inches='tight')
plt.show()

# Cell 2: Text output (separate!)
print("Statistical Results:")
print(f"Spearman correlation: rho={rho:.3f}, p={pval:.4f}")

Solution 3: Suppress text output in figure cell

# Capture results without printing
results = []
for category in categories:
    rho, pval = stats.spearmanr(x, y)
    results.append({'category': category, 'rho': rho, 'pval': pval})

# Create figure (no print statements!)
fig, axes = plt.subplots(2, 3, figsize=(12, 8))
# ... plotting code ...
plt.savefig('figure.png', dpi=150, bbox_inches='tight')
plt.show()

# Display results in separate cell or as DataFrame
results_df = pd.DataFrame(results)

When to split figures:

• Multi-panel figures with many subplots (3+ rows x 2+ columns)
• Any figure where dimensions approach 8000 pixels
• When cell has significant text output after figure
• When total cell output height feels very long in notebook

Prevention:

• Use the save_figure() helper from jupyter-notebook skill (auto-checks size)
• Keep figure cells focused on visualization only
• Save statistical results to CSV files instead of printing
• Use separate markdown cells for result interpretation

Color Palettes for Scientific Figures

Reference for colorblind-safe, sequential, and diverging color schemes.

Colorblind-Friendly Palettes

Standard palette for dual comparisons:

COLORS = {
    'Group1': '#0173B2',  # Blue
    'Group2': '#DE8F05'   # Orange
}

Accessible to most common color vision deficiencies.

Comprehensive Colorblind-Safe Color Palettes

Problem: Poor Color Accessibility

Common issue: Default color schemes often use green-blue or red-green combinations that are indistinguishable for colorblind viewers (~8% of population).

Examples of problematic combinations:

• Green + Blue (similar for deuteranopia/protanopia)
• Red + Green (classic colorblindness issue)
• Light blue + Dark blue (insufficient contrast)

Okabe-Ito Palette (Recommended by Nature)

The gold standard for scientific figures, developed by Masataka Okabe and Kei Ito.

Complete 8-color palette (hex codes):

okabe_ito = {
    'orange': '#E69F00',
    'sky_blue': '#56B4E9',
    'bluish_green': '#009E73',
    'yellow': '#F0E442',
    'blue': '#0072B2',
    'vermillion': '#D55E00',
    'reddish_purple': '#CC79A7',
    'black': '#000000'
}

For 3 categories (maximum distinction):

# Best combination for 3 categories
category_colors = {
    'Category_A': '#0072B2',    # Blue
    'Category_B': '#E69F00',    # Orange
    'Category_C': '#CC79A7'     # Reddish Purple
}

Why this combination:

• Blue (cool) + Orange (warm) + Purple (neutral) = maximum perceptual separation
• Works for all types of colorblindness (deuteranopia, protanopia, tritanopia)
• Blue-orange is universally distinguishable
• No green-blue or red-green confusion

For 5+ categories, use additional colors from the palette:

five_colors = {
    'Cat_1': '#0072B2',    # Blue
    'Cat_2': '#E69F00',    # Orange
    'Cat_3': '#CC79A7',    # Reddish Purple
    'Cat_4': '#D55E00',    # Vermillion
    'Cat_5': '#F0E442'     # Yellow
}

Paul Tol's Bright Palette (Alternative)

Another scientifically validated option:

paul_tol_bright = {
    'blue': '#4477AA',
    'red': '#EE6677',
    'green': '#228833',
    'yellow': '#CCBB44',
    'cyan': '#66CCEE',
    'purple': '#AA3377',
    'grey': '#BBBBBB'
}

Implementation in Matplotlib/Seaborn

Set up colorblind-safe palette:

import matplotlib.pyplot as plt
import seaborn as sns

# Okabe-Ito colors for 3 categories
colors = ['#0072B2', '#E69F00', '#CC79A7']

# Apply to matplotlib
plt.rcParams['axes.prop_cycle'] = plt.cycler(color=colors)

# Or use directly in plots
fig, ax = plt.subplots()
for i, category in enumerate(['A', 'B', 'C']):
    ax.plot(x, y[i], color=colors[i], label=category)

For categorical plots (seaborn):

# Define palette dictionary
palette = {
    'Phased+Dual': '#0072B2',
    'Phased+Single': '#E69F00',
    'Pri/alt+Single': '#CC79A7'
}

# Use in seaborn
sns.boxplot(data=df, x='category', y='value', palette=palette)

Best Practices

1. Avoid red-green combinations - Most common colorblindness type 2. Use patterns/markers too - Combine color with shapes for redundancy 3. Test your figures - Use colorblindness simulators online 4. Document your palette - Add comment explaining choice 5. Be consistent - Use same colors for same categories across all figures

Example: Complete Figure Setup

# Okabe-Ito palette for 3 categories
category_colors = {
    'Method_A': '#0072B2',      # Blue (Okabe-Ito)
    'Method_B': '#E69F00',      # Orange (Okabe-Ito)
    'Method_C': '#CC79A7'       # Reddish Purple (Okabe-Ito)
}

# Also use different markers for redundancy
markers = {
    'Method_A': 'o',  # circle
    'Method_B': 's',  # square
    'Method_C': '^'   # triangle
}

# Plot with both color and marker distinction
for method in ['Method_A', 'Method_B', 'Method_C']:
    data = df[df['method'] == method]
    plt.scatter(data['x'], data['y'],
                color=category_colors[method],
                marker=markers[method],
                label=method, s=50, alpha=0.7)

plt.legend()
plt.title('Analysis Results (Colorblind-Safe)')

When to Use Which Palette

Okabe-Ito:

• Scientific publications (recommended by Nature)
• 3-8 categorical variables
• Need maximum accessibility
• Standard for academic figures

Paul Tol:

• Alternative when you want different aesthetics
• Good for presentations
• Widely used in Europe

Seaborn 'colorblind':

• Quick matplotlib/seaborn integration
• Based on similar principles
• Built-in convenience

Resources

• Okabe-Ito palette: https://jfly.uni-koeln.de/color/
• Paul Tol's schemes: https://personal.sron.nl/~pault/
• Colorblind simulator: https://www.color-blindness.com/coblis-color-blindness-simulator/
• Venngage guide: https://venngage.com/blog/color-blind-friendly-palette/

Real Example: VGP Assembly Analysis

# Before: Similar blues caused confusion
old_colors = {
    'Phased+Dual': '#1976D2',    # Dark blue
    'Phased+Single': '#4FC3F7',  # Light blue - TOO SIMILAR!
    'Pri/alt+Single': '#66BB6A'  # Green - confusing with blue
}

# After: Okabe-Ito palette with maximum distinction
new_colors = {
    'Phased+Dual': '#0072B2',      # Blue
    'Phased+Single': '#E69F00',    # Orange - DISTINCT!
    'Pri/alt+Single': '#CC79A7'    # Purple - DISTINCT!
}

This ensures all readers can distinguish categories regardless of color vision deficiency.

Sequential vs Diverging Palettes

Use sequential palettes for:

• Temporal progression (old to new)
• Intensity/magnitude (low to high)
• Single-direction trends

Best practice for temporal data:

• YlOrRd (Yellow-Orange-Red): Clear old to new progression
• Start: Light color (e.g., #ffffcc - light yellow)
• End: Dark color (e.g., #b10026 - dark red)
• Users intuitively understand light to dark as past to present

Avoid for temporal data:

• Diverging palettes (RdYlBu): Implies a meaningful midpoint
• Blue to Red: No clear temporal association
• Rainbow: Color order not intuitive

Example (assembly release years 2019-2025):

# Good: Sequential YlOrRd
gradient_colors = [
    '#ffffcc',  # 2019 - light yellow (oldest)
    '#ffeda0',  # 2020
    '#fed976',  # 2021
    '#feb24c',  # 2022
    '#fd8d3c',  # 2023
    '#fc4e2a',  # 2024
    '#b10026',  # 2025 - dark red (newest)
]

# Bad: Diverging RdYlBu (implies 2022 is special/central)

ColorBrewer sequential palettes:

• YlOrRd: Yellow-Orange-Red (temporal, intensity, heat)
• YlGn: Yellow-Green (growth, vegetation)
• PuBuGn: Purple-Blue-Green (water, depth)
• OrRd: Orange-Red (similar to YlOrRd, starts darker)

ColorBrewer resources:

• Website: https://colorbrewer2.org
• Python: from palettable.colorbrewer import sequential
• Matplotlib: plt.cm.YlOrRd or custom with hex codes

Scientific Figure Descriptions for Publications

Templates and guidelines for writing publication-quality figure descriptions.

Structure for Multi-Panel Figures

Opening Sentence: Overview + sample size + stratification

**[Analysis type] of [N] [unit]** across [timeframe/condition], stratified by
[categories]: Category1 (n=X), Category2 (n=Y), Category3 (n=Z).

Panel Descriptions: For each panel:

**[Metric Name]** (panel location): [Pattern observed]. [Statistical test]
(rho=[value], p=[value]) shows [interpretation]. [Biological/technical context].

Closing Interpretation: Synthesize findings

**Interpretation:** [Overall pattern]. [Comparison across categories].
[Methodological implications]. [Connection to study goals].

Example: Temporal Trends Figure

### Figure 4. Temporal trends in assembly quality metrics for HiFi-only assemblies (2021-2025)

**Temporal analysis of six assembly quality metrics across 268 HiFi assemblies**
spanning 2021-2025, stratified by assembly and curation method: Phased+Dual
(n=101, blue), Phased+Single (n=42, orange), and Pri/alt+Single (n=125, purple).
Each panel displays individual assembly measurements (points) with linear
regression trend lines (dashed) for each category. Trend significance was
assessed using Spearman correlation (alpha=0.05).

**Key Findings:**

**Scaffold N50** (upper left): Pri/alt+Single assemblies show significant
improvement over time (rho=0.32, p=2.7x10^-4), increasing from ~100 Mb to ~700 Mb,
while Phased assemblies remain stable at ~100-200 Mb. This suggests technological
improvements in single-assembly methods during the HiFi era.

**Gap Density** (upper middle): All HiFi assemblies collectively show decreasing
gap density over time (rho=-0.17, p=0.0057), indicating improved sequence continuity.

[Additional panels...]

**Interpretation:** Temporal trends are category-specific and metric-dependent.
Pri/alt+Single assemblies show quality improvements (N50, gap density) consistent
with technological advancement during 2021-2025. Phased assemblies remain stable
across most metrics, suggesting their quality is primarily methodology-determined.

Quantitative Details to Include

Always include:

• Sample sizes (n=X) for each group
• Statistical test used (Spearman, Mann-Whitney, etc.)
• Effect sizes (rho, r-squared, effect magnitude)
• p-values with scientific notation (p=2.7x10^-4)
• Temporal/spatial ranges (2021-2025, 100-700 Mb)
• Significance threshold (alpha=0.05)

Avoid:

• Vague terms ("improved", "changed") without quantification
• p-values without effect sizes
• Missing sample sizes
• Unspecified statistical methods

Adding to Jupyter Notebooks

import json

fig_description = {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
        "### Figure X. [Title]\n",
        "\n",
        "**[Opening with sample sizes]**\n",
        "\n",
        "**Key Findings:**\n",
        "\n",
        "**[Metric 1]**: [Statistical result]. [Interpretation].\n",
        "\n",
        "**Interpretation:** [Synthesis]."
    ]
}

# Insert after the plotting cell
nb['cells'].insert(plot_cell_idx + 1, fig_description)

iTOL (Interactive Tree of Life) Dataset Creation

Reference for creating annotation datasets for phylogenetic tree visualization in iTOL.

Overview

iTOL is a web-based tool for phylogenetic tree visualization. Creating annotation datasets requires specific formats and understanding format differences between legacy and modern approaches.

Key Format Types

1. DATASET_STYLE (Modern Format for Branch/Node Coloring)

Use for coloring individual terminal branches or nodes.

Critical requirements:

• Use SEPARATOR COMMA (not TAB)
• Format: species,branch,node,#color,width,style
• The three fields (branch/node/style) are all required even though only one is used

DATASET_STYLE
SEPARATOR COMMA

DATASET_LABEL,Terminal Branch Colors by Taxonomy
COLOR,#ff0000

DATA
Homo_sapiens,branch,node,#C084C0,2,normal
Mus_musculus,branch,node,#C084C0,2,normal

Common errors avoided:

• Using TAB separator causes "Invalid color definition" errors
• Using clade instead of individual species causes all branches to get same color
• Omitting required fields causes format errors

2. DATASET_BINARY (Presence/Absence Markers)

Use for adding symbols (checkmarks, stars, etc.) to specific species.

DATASET_BINARY
SEPARATOR TAB

DATASET_LABEL	Dual Curation

FIELD_SHAPES	6
FIELD_LABELS	Dual Curation
FIELD_COLORS	#FF0000

LEGEND_TITLE	Curation Status
LEGEND_SHAPES	6
LEGEND_COLORS	#FF0000
LEGEND_LABELS	Dual Curation

DATA
Homo_sapiens	1
Mus_musculus	1

Symbol codes:

• 1 = circle, 2 = square, 3 = diamond, 4 = triangle, 5 = filled square, 6 = checkmark

3. DATASET_COLORSTRIP (Colored Rectangles)

DATASET_COLORSTRIP
SEPARATOR TAB

DATASET_LABEL	Taxonomic Lineage

DATA
Homo_sapiens	#C084C0	Mammals
Mus_musculus	#C084C0	Mammals

Species Name Synchronization

Problem: Tree species names often differ from metadata due to: 1. TimeTree database replacements (standardization) 2. Spelling variants (e.g., Chiropotes_utahickae vs Chiropotes_utahicki) 3. Case differences (e.g., Alca_torda vs Alca_Torda) 4. Trailing spaces in CSV files

Solution workflow: 1. Export tree species list: grep -oE "[A-Z][a-z]+_[a-z]+" Tree.nwk | sort -u 2. Compare with metadata species list 3. Create replacement mapping JSON 4. Apply systematically to tree AND all annotation files 5. Document replacements for reproducibility

Best practice: Create separate versions:

• *_corrected.* - After TimeTree replacements
• *_final.* - After all name variant corrections

Handling Reference Species Added by Tree Builders

TimeTree and similar tools may add reference species not in your original dataset for:

• Phylogenetic completeness
• Temporal calibration
• Topological constraints

Document these additions: 1. Identify species in tree but not in metadata 2. Research their phylogenetic role 3. Create separate iTOL dataset to highlight them 4. Document why they were added

Example:

# Create dataset for reference species
timetree_additions = ["Species_one", "Species_two"]
# Use different symbol/color to distinguish from your species

Color Schemes for Taxonomy

Standard color palette for major vertebrate groups:

colors = {
    'Mammals': '#C084C0',       # Purple
    'Birds': '#FFD700',         # Gold
    'Reptiles': '#9370DB',      # Medium Purple
    'Amphibians': '#98D8C8',    # Turquoise
    'Fishes': '#87CEEB',        # Sky Blue
    'Invertebrates': '#8B4513'  # Brown
}

Troubleshooting iTOL Errors

File Organization Best Practice

phylo/
├── Tree.nwk                              # Original
├── Tree_corrected.nwk                    # After TimeTree replacements
├── Tree_final.nwk                        # After all name corrections
├── itol_branch_colors_final.txt          # Terminal branch colors
├── itol_taxonomic_colorstrip_final.txt   # Colored strips
├── itol_dual_curation_binary_final.txt   # Binary markers
├── itol_timetree_additions_final.txt     # Reference species markers
├── species_replacements.json             # TimeTree replacements
├── name_variant_replacements.json        # Spelling/case fixes
└── SPECIES_CORRECTIONS_SUMMARY.md        # Full documentation

Updating iTOL Config Color Schemes

When updating color schemes across multiple iTOL configuration files, colors appear in multiple locations with different syntax:

Files requiring updates (for 3-category example):

1. Colorstrip configs (itol_3category_colorstrip_UPDATED.txt):

• LEGEND_COLORS line: tab-separated hex values
• Individual species rows: species_name<tab>category<tab>#HEXCODE

2. Label configs (itol_3category_labels_UPDATED.txt):

• LEGEND_COLORS, line: comma-separated hex values
• DATA rows: species,label,label,#HEXCODE,1,normal

3. Branch color configs (itol_3category_branch_colors_UPDATED.txt):

• DATA rows: species<tab>branch<tab>#HEXCODE<tab>normal<tab>2

4. Binary highlight configs (one per category):

• COLOR line: single hex value
• LEGEND_COLORS line: single hex value
• FIELD_COLORS line: single hex value

Efficient Update Strategy:

Use Edit tool with replace_all=true for each old to new color mapping:

# Update all instances of old color across file
Edit(
    file_path="itol_3category_colorstrip_UPDATED.txt",
    old_string="#3498db",
    new_string="#FF8C00",
    replace_all=True
)

Typical color update sequence: 1. Map old to new colors (e.g., blue to orange, orange to green, green to blue) 2. Update all files with first mapping (old blue to new orange) 3. Update all files with second mapping (old orange to new green) 4. Update all files with third mapping (old green to new blue)

Files to update (for 3-category system):

• itol_3category_colorstrip_UPDATED.txt
• itol_3category_labels_UPDATED.txt
• itol_3category_branch_colors_UPDATED.txt
• itol_3category_phased_dual_binary_UPDATED.txt
• itol_3category_phased_single_binary_UPDATED.txt
• itol_3category_pri_alt_single_binary_UPDATED.txt

Verification:

• Grep for old hex codes to confirm all replaced
• Check LEGEND_COLORS lines match DATA row colors
• Verify binary files use correct category color

Common Color Scheme Examples:

VGP Curation 3-Category System:

COLORS = {
    'Phased+Dual': '#FF8C00',      # Dark orange
    'Phased+Single': '#50C878',    # Emerald green
    'Pri/alt+Single': '#4169E1'    # Royal blue
}

References

• iTOL documentation: https://itol.embl.de/help.cgi

Journal-Specific Figure Requirements

Overview

Different journals have specific technical requirements for figures. This reference compiles common requirements from major scientific publishers. Always check the specific journal's author guidelines for the most current requirements.

Nature Portfolio (Nature, Nature Methods, etc.)

Technical Specifications

• File formats:
• Vector: PDF, EPS, AI (preferred for graphs)
• Raster: TIFF, PNG (for images)
• Never: PowerPoint, Word, JPEG

• Resolution:
• Line art: 1000-1200 DPI
• Combination (line art + images): 600 DPI
• Photographs/microscopy: 300 DPI minimum

• Color space: RGB (Nature is digital-first)

• Dimensions:
• Single column: 89 mm (3.5 inches)
• 1.5 column: 120 mm (4.7 inches)
• Double column: 183 mm (7.2 inches)
• Maximum height: 247 mm (9.7 inches)

• Fonts:
• Arial or Helvetica (or similar sans-serif)
• Minimum 5-7 pt at final size
• Embed all fonts in PDF/EPS

Nature Specific Guidelines

• Panel labels: a, b, c (lowercase, bold) in top-left corner
• Scale bars required for microscopy images
• Gel images: Include molecular weight markers
• Cropping: Indicate with line breaks
• Statistics: Mark significance; define symbols in legend
• Source data: Required for all graphs

File Naming

Format: FirstAuthorLastName_FigureNumber.ext Example: Smith_Fig1.pdf

Science (AAAS)

Technical Specifications

• File formats:
• Vector: EPS, PDF (preferred)
• Raster: TIFF
• Acceptable: AI, PSD (Photoshop)

• Resolution:
• Line art: 1000 DPI minimum
• Photographs: 300 DPI minimum
• Combination: 600 DPI minimum

• Color space: RGB

• Dimensions:
• Single column: 5.5 cm (2.17 inches)
• 1.5 column: 12 cm (4.72 inches)
• Full width: 17.5 cm (6.89 inches)
• Maximum height: 23.3 cm (9.17 inches)

• Fonts:
• Helvetica (or Arial)
• 6-8 pt minimum at final size
• Consistent across all figures

Science Specific Guidelines

• Panel labels: (A), (B), (C) in parentheses
• Minimal text within figures (details in caption)
• High contrast for web and print
• Error bars required; define in caption
• Avoid excessive whitespace

File Naming

Format: Manuscript#_Fig#.ext Example: abn1234_Fig1.eps

Cell Press (Cell, Neuron, Molecular Cell, etc.)

Technical Specifications

• File formats:
• Vector: PDF, EPS (preferred for graphs/diagrams)
• Raster: TIFF (for photographs)

• Resolution:
• Line art: 1000 DPI
• Photographs: 300 DPI
• Combination: 600 DPI

• Color space: RGB

• Dimensions:
• Single column: 85 mm (3.35 inches)
• Double column: 178 mm (7.01 inches)
• Maximum height: 230 mm (9.06 inches)

• Fonts:
• Arial or Helvetica only
• 8-12 pt for axis labels
• 6-8 pt for tick labels

Cell Press Specific Guidelines

• Panel labels: (A), (B), (C) or A, B, C in top-left
• Related panels should match in size
• Scale bars mandatory for microscopy
• Western blots: Include molecular weight markers
• Arrows/arrowheads: 2 pt minimum width
• Line widths: 1-2 pt for data

PLOS (Public Library of Science)

Technical Specifications

• File formats:
• Vector: EPS, PDF (preferred)
• Raster: TIFF, PNG
• TIFF with LZW compression acceptable

• Resolution:
• Minimum 300 DPI at final size (all figure types)
• 600 DPI preferred for line art

• Color space: RGB

• Dimensions:
• Single column: 8.3 cm (3.27 inches)
• 1.5 column: 11.4 cm (4.49 inches)
• Double column: 17.3 cm (6.81 inches)
• Maximum height: 23.3 cm (9.17 inches)

• Fonts:
• Sans-serif preferred (Arial, Helvetica)
• 8-12 pt for labels at final size

PLOS Specific Guidelines

• Figures should be understandable without caption
• Color required only if adding information
• All figures convertible to grayscale
• Panel labels optional but recommended
• Open access: Figures must be CC-BY licensed
• Source data files encouraged

ACS (American Chemical Society)

Technical Specifications

• File formats:
• Preferred: TIFF, PDF, EPS
• Application files: AI, CDX (ChemDraw), CDL
• Acceptable: PNG (not for publication)

• Resolution:
• Minimum 300 DPI at final size
• 600 DPI for line art and chemical structures
• 1200 DPI for detailed structures

• Color space: RGB or CMYK (check specific journal)

• Dimensions:
• Single column: 3.25 inches (8.25 cm)
• Double column: 7 inches (17.78 cm)

• Fonts:
• Embedded fonts required
• Consistent sizing across figures

ACS Specific Guidelines

• Chemical structures: Use ChemDraw or equivalent
• Atom labels: 10-12 pt
• Bond thickness: 2 pt
• Panel labels: Lowercase bold (a, b, c)
• High contrast required (many ACS journals grayscale print)

Elsevier Journals (varies by journal)

Technical Specifications

• File formats:
• Vector: EPS, PDF
• Raster: TIFF, JPEG (only for photographs)

• Resolution:
• Line art: 1000 DPI minimum
• Photographs: 300 DPI minimum
• Combination: 600 DPI minimum

• Color space: RGB (for online); CMYK (for print journals)

• Dimensions: Vary by journal
• Common single column: 90 mm
• Common double column: 190 mm

• Fonts:
• Preferred: Arial, Times, Symbol
• Minimum 6 pt at final size

Elsevier Specific Guidelines

• Check individual journal guidelines (highly variable)
• Some journals charge for color in print
• Panel labels typically (A), (B), (C) or A, B, C
• Graphical abstract often required (separate from figures)

IEEE (Engineering/Computer Science)

Technical Specifications

• File formats:
• Vector: PDF, EPS (preferred)
• Raster: TIFF, PNG

• Resolution:
• Photographs/graphics: 300 DPI minimum at final size
• Line art: 600 DPI minimum

• Color space: RGB (online); CMYK (print)

• Dimensions:
• Single column: 3.5 inches (8.9 cm)
• Double column: 7.16 inches (18.2 cm)

• Fonts:
• Sans-serif preferred
• Minimum 8-10 pt at final size

IEEE Specific Guidelines

• Figures should be readable in black and white
• Color figures incur no charge (online publication)
• Panel labels: (a), (b), (c) in lowercase
• Captions below figures (not on separate page)
• Use IEEE graphics checker tool before submission

BMC (BioMed Central) - Open Access

Technical Specifications

• File formats:
• Any standard format accepted
• Preferred: TIFF, PDF, EPS, PNG

• Resolution:
• Minimum 600 DPI for line art
• Minimum 300 DPI for photographs

• Color space: RGB

• Dimensions:
• Flexible, but consider readability
• Maximum width typically 140 mm

• Fonts:
• Embedded and readable

BMC Specific Guidelines

• Open access: CC-BY license required
• Figure files uploaded separately
• Panel labels as appropriate for field
• Source data encouraged
• Accessibility important (colorblind-friendly)

Common Requirements Across Journals

Universal Best Practices

1. Never use JPEG for graphs/plots: Compression artifacts 2. Embed all fonts: In PDF/EPS files 3. Layer structure: Flatten images (merge layers in Photoshop) 4. RGB vs CMYK: Most journals now RGB (digital-first) 5. High resolution: Always better to start high, reduce if needed 6. Consistency: Same style across all figures in manuscript 7. File size: Balance quality with reasonable file sizes (typically <10 MB per figure)

Submitting Figures

• Initial submission: Lower resolution often acceptable (for review)
• Revision/acceptance: High-resolution required
• Separate files: Each figure as separate file
• File naming: Clear, systematic naming
• Supporting information: May have different requirements

Quick Reference Table

Checking Requirements

Before Submission Checklist

1. Read journal's author guidelines (figure section) 2. Check file format requirements 3. Verify resolution requirements 4. Confirm size specifications (width × height) 5. Check font requirements 6. Verify color space (RGB vs CMYK) 7. Check panel labeling style 8. Review supplementary materials requirements 9. Confirm file naming conventions 10. Check file size limits

Useful Tools

• ImageJ/Fiji: Check/adjust DPI
• Adobe Acrobat: Verify embedded fonts, check PDF properties
• GIMP: Free alternative to Photoshop for raster editing
• Inkscape: Free vector graphics editor

Resources

• Journal websites: Always check "Author Guidelines" or "Instructions for Authors"
• Publisher resources: Many provide templates and tools
• Format conversion: Use reputable tools; check output quality
• Help desks: Contact journal staff if unclear

Notes

• Requirements change periodically - always verify current guidelines
• Preprint servers (bioRxiv, arXiv) often have different requirements
• Conference proceedings may have separate requirements
• Some journals offer figure preparation services (often paid)
• Supplementary figures may have relaxed requirements compared to main text figures

Data Visualization Pitfalls and Troubleshooting

Reference material for common visualization mistakes, coordinate system issues, and debugging techniques.

Common Pitfalls with Log-Scale Plots

Violin Plots on Log Scales

Problem: Violin plots use Kernel Density Estimation (KDE) in linear space, then the axis is transformed to log scale. This causes severe visual distortion where the violin shape doesn't accurately represent the actual data distribution.

Symptoms:

• Smooth, blob-like violin shapes on log axes
• Visual representation suggests even distribution but histogram shows heavy clustering
• Particularly problematic with right-skewed data heavily concentrated in one region

Example of the problem:

# BAD: Violin plot on log scale
import matplotlib.pyplot as plt
import numpy as np

data = np.random.exponential(10, 1000)  # Right-skewed data
fig, ax = plt.subplots()
ax.violinplot([data])
ax.set_yscale('log')  # Distorts the violin shape!
# Result: Smooth violin that doesn't show the true concentration at low values

Solution 1: Use boxplots instead

# GOOD: Boxplot on log scale
ax.boxplot([data])
ax.set_yscale('log')  # Boxplot statistics remain meaningful

Why boxplots work: Boxplot statistics (median, quartiles, outliers) are calculated as specific values, not density estimates, so they remain meaningful on log scales.

Solution 2: Log-transform data first

# ALTERNATIVE: Log-transform data first, then use violin on linear axis
log_data = np.log10(data[data > 0])
ax.violinplot([log_data])
ax.set_ylabel('log10(Value)')
# Keep linear axis - violin now accurately represents log-space distribution

Solution 3: Use histograms with log axes

# GOOD: Histogram with log y-axis shows true frequency distribution
ax.hist(data, bins=30)
ax.set_xscale('log')  # Data axis
ax.set_yscale('log')  # Frequency axis - shows concentration clearly

Impact

This pitfall can lead to misleading figures in publications where the visual representation contradicts the actual data distribution. In our VGP curation analysis, this affected 4 different figures before correction.

Outlier Handling: Show First, Decide Later

Default stance: Show ALL data points in initial visualizations

# CORRECT - show all data
plt.boxplot(data, showfliers=True)

# AVOID initially - hides potentially important data
plt.boxplot(data, showfliers=False)

Rationale:

• Outliers may be biologically meaningful
• Filtering decisions should be informed by seeing complete data
• Easy to filter later, hard to know what you missed
• Patterns in outliers can reveal data quality issues

Workflow: 1. Generate figures with all data (showfliers=True) 2. Review with domain expert 3. Decide case-by-case if outliers should be excluded 4. Document rationale for any exclusions

Only filter outliers when:

• Technical artifact confirmed (e.g., processing error)
• Prevents seeing relevant patterns in bulk of data
• Documented and justified in methods
• Alternative view with all data provided in supplement

Example documentation:

# Remove known technical outlier
"""
Excluded assembly GCA_123456 from Figure 2 analysis:
- Scaffold N50 = 500 Gb (500x larger than genome size)
- Confirmed as assembly processing error in NCBI notes
- Other metrics for this assembly are valid and included in other figures
"""

Matplotlib Text Positioning Creates Empty Space

Problem: Saved figure has large area of empty white space above the actual plot, making the figure much taller than necessary.

Cause: Using transform=ax.get_xaxis_transform() with data coordinate y-values positions text far outside the plot bounds. The transform uses axis-relative coordinates (0-1) for x but data coordinates for y, so large y-values create huge positioning errors.

Symptoms:

• Empty white space at top (or bottom) of saved figure
• Text annotations not visible or way off the plot
• tight_layout() and pad_inches adjustments don't fix it
• Problem persists even after reducing figure size

Example of the problem:

# BAD: Mixing coordinate systems
for category in categories:
    data = df[df['category'] == category]
    ax.scatter(data['x'], data['y'])

    # Add significance marker
    y_max = data['y'].max()  # e.g., y_max = 2000000000 (2 billion)

    # PROBLEM: y_max is in data coordinates, transform expects 0-1!
    ax.text(0.5, y_max * 0.95, '***',
           transform=ax.get_xaxis_transform(),  # x in 0-1, y in data coords
           ha='center', fontsize=12)
    # This positions text at y = 1.9 billion in the mixed coordinate system!

plt.savefig('figure.png', dpi=150, bbox_inches='tight')
# Result: Massive empty space with text way above visible plot

Why this happens:

• ax.get_xaxis_transform() uses axis coordinates (0-1) for x, data coordinates for y
• y_max * 0.95 for scaffold N50 might be 1.9 billion (1.9e9)
• Transform interprets this as 1.9 billion axis units above the plot
• bbox_inches='tight' includes this invisible text, creating empty space

Solution 1: Position within data range (RECOMMENDED)

Calculate position within the actual data coordinate system:

# GOOD: Position within plot bounds using pure data coordinates
for category in categories:
    data = df[df['category'] == category]
    ax.scatter(data['x'], data['y'])

    # Get actual data range
    y_min, y_max = ax.get_ylim()

    # Position at 90% of the visible range
    y_pos = y_min + (y_max - y_min) * 0.90

    # Use data coordinates (no transform needed)
    ax.text(0.5, y_pos, '***',
           ha='center', va='center', fontsize=12)

Solution 2: Use axis transform correctly with 0-1 coordinates

If you want to use the transform, use 0-1 range for y:

# GOOD: Both x and y in 0-1 axis coordinates
ax.text(0.5, 0.90, '***',
       transform=ax.transAxes,  # Both x and y in 0-1 range
       ha='center', va='center', fontsize=12)

Solution 3: Use annotate with xycoords

# GOOD: Explicit coordinate specification
ax.annotate('***',
           xy=(x_pos, y_pos),          # Data coordinates
           xycoords='data',
           ha='center', va='center', fontsize=12)

Coordinate Transform Quick Reference:

When to use each approach:

• Data coordinates (no transform): Annotations tied to specific data points
• Axis coordinates (transAxes): Labels in fixed positions (e.g., panel letters)
• Mixed transforms: Advanced use only, requires careful coordinate scaling

Debugging tips:

• If empty space appears, check for text/annotation calls with large y-values
• Use ax.get_ylim() to verify reasonable y-coordinate range
• Temporarily comment out text/annotation calls to identify culprit
• Verify saved figure dimensions match expected size

Prevention:

• Prefer pure data coordinates for most annotations
• Only use transforms when specifically needed
• Always verify coordinate ranges match transform type
• Test saved figure size after adding annotations

Float Year Labels on X-Axis

Problem: X-axis shows decimal years (2021.0, 2021.5, 2022.0) instead of clean integers (2021, 2022, 2023).

Cause: Matplotlib's default tick formatter displays float values with decimals when the data type is float.

Solution: Use MaxNLocator with integer=True

import matplotlib.pyplot as plt

# After creating plot
ax.scatter(df['year'], df['value'])

# Fix x-axis to show only integer years
ax.xaxis.set_major_locator(plt.MaxNLocator(integer=True))

Why This Works:

• MaxNLocator(integer=True) constrains tick locations to integers
• Works even when underlying data is float (e.g., release_year column as float64)
• Automatically chooses appropriate spacing (won't show every year if range is large)

Common Use Case: Temporal analyses where year data is stored as float but should display as integer for readability.

Example:

# Data with float years
df['release_year'] = [2021.0, 2022.0, 2023.0, 2024.0, 2025.0]

fig, ax = plt.subplots()
ax.scatter(df['release_year'], df['metric'])

# Without fix: x-axis shows 2021.0, 2021.5, 2022.0, 2022.5, ...
# With fix: x-axis shows 2021, 2022, 2023, 2024, 2025
ax.xaxis.set_major_locator(plt.MaxNLocator(integer=True))

Axis Range Optimization for Compressed Distributions

When data is concentrated at one end of range:

Problem: Cumulative distributions all at 80-100% look compressed with 0-100% Y-axis

Solution: Adjust axis limits to focus on data range

# For chromosome assignment cumulative distribution (mostly 80-100%)
ax.set_ylim(50, 100)  # Start at 50% instead of 0%

# For legend placement with adjusted range
ax.legend(loc='upper left')  # Prevents overlap with curves at top-right

When to adjust axis ranges:

• Data concentrated in narrow range (e.g., 80-100%)
• Improves visibility of differences
• All relevant data still visible
• Makes small differences more apparent

When NOT to adjust:

• Would hide meaningful outliers
• Creates misleading visual impression
• Data actually spans full range
• Standard in field to show full range (0-100%)

Best practice: Show both views if controversial

• Main figure: Zoomed range for clarity
• Supplementary: Full range for context