Scanpy

{
  "description": "\"Single-cell RNA-seq analysis. Load .h5ad/10X data, QC, normalization, PCA/UMAP/t-SNE, Leiden clustering, marker genes, cell type annotation, trajectory, for scRNA-seq analysis.\"",
  "references": {
    "files": [
      "references/api_reference.md",
      "references/plotting_guide.md",
      "references/standard_workflow.md"
    ]
  },
  "content": "### Basic Import and Setup\r\n\r\n```python\r\nimport scanpy as sc\r\nimport pandas as pd\r\nimport numpy as np\r\n\r\nsc.settings.verbosity = 3\r\nsc.settings.set_figure_params(dpi=80, facecolor='white')\r\nsc.settings.figdir = './figures/'\r\n```\r\n\r\n### Loading Data\r\n\r\n```python\r\nadata = sc.read_10x_mtx('path/to/data/')\r\nadata = sc.read_10x_h5('path/to/data.h5')\r\n\r\nadata = sc.read_h5ad('path/to/data.h5ad')\r\n\r\nadata = sc.read_csv('path/to/data.csv')\r\n```\r\n\r\n### Understanding AnnData Structure\r\n\r\nThe AnnData object is the core data structure in scanpy:\r\n\r\n```python\r\nadata.X          # Expression matrix (cells × genes)\r\nadata.obs        # Cell metadata (DataFrame)\r\nadata.var        # Gene metadata (DataFrame)\r\nadata.uns        # Unstructured annotations (dict)\r\nadata.obsm       # Multi-dimensional cell data (PCA, UMAP)\r\nadata.raw        # Raw data backup\r\n\r\n\r\n### 1. Quality Control\r\n\r\nIdentify and filter low-quality cells and genes:\r\n\r\n```python\r\nadata.var['mt'] = adata.var_names.str.startswith('MT-')\r\n\r\nsc.pp.calculate_qc_metrics(adata, qc_vars=['mt'], inplace=True)\r\n\r\nsc.pl.violin(adata, ['n_genes_by_counts', 'total_counts', 'pct_counts_mt'],\r\n             jitter=0.4, multi_panel=True)\r\n\r\nsc.pp.filter_cells(adata, min_genes=200)\r\nsc.pp.filter_genes(adata, min_cells=3)\r\nadata = adata[adata.obs.pct_counts_mt < 5, :]  # Remove high MT% cells\r\n```\r\n\r\n**Use the QC script for automated analysis:**\r\n```bash\r\npython scripts/qc_analysis.py input_file.h5ad --output filtered.h5ad\r\n```\r\n\r\n### 2. Normalization and Preprocessing\r\n\r\n```python\r\nsc.pp.normalize_total(adata, target_sum=1e4)\r\n\r\nsc.pp.log1p(adata)\r\n\r\nadata.raw = adata\r\n\r\nsc.pp.highly_variable_genes(adata, n_top_genes=2000)\r\nsc.pl.highly_variable_genes(adata)\r\n\r\nadata = adata[:, adata.var.highly_variable]\r\n\r\nsc.pp.regress_out(adata, ['total_counts', 'pct_counts_mt'])\r\n\r\nsc.pp.scale(adata, max_value=10)\r\n```\r\n\r\n### 3. Dimensionality Reduction\r\n\r\n```python\r\nsc.tl.pca(adata, svd_solver='arpack')\r\nsc.pl.pca_variance_ratio(adata, log=True)  # Check elbow plot\r\n\r\nsc.pp.neighbors(adata, n_neighbors=10, n_pcs=40)\r\n\r\nsc.tl.umap(adata)\r\nsc.pl.umap(adata, color='leiden')\r\n\r\nsc.tl.tsne(adata)\r\n```\r\n\r\n### 4. Clustering\r\n\r\n```python\r\nsc.tl.leiden(adata, resolution=0.5)\r\nsc.pl.umap(adata, color='leiden', legend_loc='on data')\r\n\r\nfor res in [0.3, 0.5, 0.8, 1.0]:\r\n    sc.tl.leiden(adata, resolution=res, key_added=f'leiden_{res}')\r\n```\r\n\r\n### 5. Marker Gene Identification\r\n\r\n```python\r\nsc.tl.rank_genes_groups(adata, 'leiden', method='wilcoxon')\r\n\r\nsc.pl.rank_genes_groups(adata, n_genes=25, sharey=False)\r\nsc.pl.rank_genes_groups_heatmap(adata, n_genes=10)\r\nsc.pl.rank_genes_groups_dotplot(adata, n_genes=5)\r\n\r\nmarkers = sc.get.rank_genes_groups_df(adata, group='0')\r\n```\r\n\r\n### 6. Cell Type Annotation\r\n\r\n```python\r\nmarker_genes = ['CD3D', 'CD14', 'MS4A1', 'NKG7', 'FCGR3A']\r\n\r\nsc.pl.umap(adata, color=marker_genes, use_raw=True)\r\nsc.pl.dotplot(adata, var_names=marker_genes, groupby='leiden')\r\n\r\ncluster_to_celltype = {\r\n    '0': 'CD4 T cells',\r\n    '1': 'CD14+ Monocytes',\r\n    '2': 'B cells',\r\n    '3': 'CD8 T cells',\r\n}\r\nadata.obs['cell_type'] = adata.obs['leiden'].map(cluster_to_celltype)\r\n\r\nsc.pl.umap(adata, color='cell_type', legend_loc='on data')\r\n```\r\n\r\n### 7. Save Results\r\n\r\n```python\r\nadata.write('results/processed_data.h5ad')\r\n\r\n\r\n### Creating Publication-Quality Plots\r\n\r\n```python\r\nsc.settings.set_figure_params(dpi=300, frameon=False, figsize=(5, 5))\r\nsc.settings.file_format_figs = 'pdf'\r\n\r\nsc.pl.umap(adata, color='cell_type',\r\n           palette='Set2',\r\n           legend_loc='on data',\r\n           legend_fontsize=12,\r\n           legend_fontoutline=2,\r\n           frameon=False,\r\n           save='_publication.pdf')\r\n\r\nsc.pl.heatmap(adata, var_names=genes, groupby='cell_type',\r\n              swap_axes=True, show_gene_labels=True,\r\n              save='_markers.pdf')\r\n\r\nsc.pl.dotplot(adata, var_names=genes, groupby='cell_type',\r\n              save='_dotplot.pdf')\r\n```\r\n\r\nRefer to `references/plotting_guide.md` for comprehensive visualization examples.\r\n\r\n### Trajectory Inference\r\n\r\n```python\r\nsc.tl.paga(adata, groups='leiden')\r\nsc.pl.paga(adata, color='leiden')\r\n\r\nadata.uns['iroot'] = np.flatnonzero(adata.obs['leiden'] == '0')[0]\r\nsc.tl.dpt(adata)\r\nsc.pl.umap(adata, color='dpt_pseudotime')\r\n```\r\n\r\n### Differential Expression Between Conditions\r\n\r\n```python\r\nadata_subset = adata[adata.obs['cell_type'] == 'T cells']\r\nsc.tl.rank_genes_groups(adata_subset, groupby='condition',\r\n                         groups=['treated'], reference='control')\r\nsc.pl.rank_genes_groups(adata_subset, groups=['treated'])\r\n```\r\n\r\n### Gene Set Scoring\r\n\r\n```python\r\ngene_set = ['CD3D', 'CD3E', 'CD3G']\r\nsc.tl.score_genes(adata, gene_set, score_name='T_cell_score')\r\nsc.pl.umap(adata, color='T_cell_score')\r\n```\r\n\r\n### Batch Correction\r\n\r\n```python\r\nsc.pp.combat(adata, key='batch')\r\n\r\n\r\n### scripts/qc_analysis.py\r\nAutomated quality control script that calculates metrics, generates plots, and filters data:\r\n\r\n```bash\r\npython scripts/qc_analysis.py input.h5ad --output filtered.h5ad \\\r\n    --mt-threshold 5 --min-genes 200 --min-cells 3\r\n```\r\n\r\n### references/standard_workflow.md\r\nComplete step-by-step workflow with detailed explanations and code examples for:\r\n- Data loading and setup\r\n- Quality control with visualization\r\n- Normalization and scaling\r\n- Feature selection\r\n- Dimensionality reduction (PCA, UMAP, t-SNE)\r\n- Clustering (Leiden, Louvain)\r\n- Marker gene identification\r\n- Cell type annotation\r\n- Trajectory inference\r\n- Differential expression\r\n\r\nRead this reference when performing a complete analysis from scratch.\r\n\r\n### references/api_reference.md\r\nQuick reference guide for scanpy functions organized by module:\r\n- Reading/writing data (`sc.read_*`, `adata.write_*`)\r\n- Preprocessing (`sc.pp.*`)\r\n- Tools (`sc.tl.*`)\r\n- Plotting (`sc.pl.*`)\r\n- AnnData structure and manipulation\r\n- Settings and utilities\r\n\r\nUse this for quick lookup of function signatures and common parameters.\r\n\r\n### references/plotting_guide.md\r\nComprehensive visualization guide including:\r\n- Quality control plots\r\n- Dimensionality reduction visualizations\r\n- Clustering visualizations\r\n- Marker gene plots (heatmaps, dot plots, violin plots)\r\n- Trajectory and pseudotime plots\r\n- Publication-quality customization\r\n- Multi-panel figures\r\n- Color palettes and styling\r\n\r\nConsult this when creating publication-ready figures.\r\n\r\n### assets/analysis_template.py\r\nComplete analysis template providing a full workflow from data loading through cell type annotation. Copy and customize this template for new analyses:\r\n\r\n```bash\r\ncp assets/analysis_template.py my_analysis.py",
  "name": "scanpy",
  "id": "scientific-pkg-scanpy",
  "sections": {
    "Quick Start": "adata.obs_names  # Cell barcodes\r\nadata.var_names  # Gene names\r\n```",
    "Key Parameters to Adjust": "### Quality Control\r\n- `min_genes`: Minimum genes per cell (typically 200-500)\r\n- `min_cells`: Minimum cells per gene (typically 3-10)\r\n- `pct_counts_mt`: Mitochondrial threshold (typically 5-20%)\r\n\r\n### Normalization\r\n- `target_sum`: Target counts per cell (default 1e4)\r\n\r\n### Feature Selection\r\n- `n_top_genes`: Number of HVGs (typically 2000-3000)\r\n- `min_mean`, `max_mean`, `min_disp`: HVG selection parameters\r\n\r\n### Dimensionality Reduction\r\n- `n_pcs`: Number of principal components (check variance ratio plot)\r\n- `n_neighbors`: Number of neighbors (typically 10-30)\r\n\r\n### Clustering\r\n- `resolution`: Clustering granularity (0.4-1.2, higher = more clusters)",
    "Tips for Effective Analysis": "1. **Start with the template**: Use `assets/analysis_template.py` as a starting point\r\n2. **Run QC script first**: Use `scripts/qc_analysis.py` for initial filtering\r\n3. **Consult references as needed**: Load workflow and API references into context\r\n4. **Iterate on clustering**: Try multiple resolutions and visualization methods\r\n5. **Validate biologically**: Check marker genes match expected cell types\r\n6. **Document parameters**: Record QC thresholds and analysis settings\r\n7. **Save checkpoints**: Write intermediate results at key steps",
    "Additional Resources": "- **Official scanpy documentation**: https://scanpy.readthedocs.io/\r\n- **Scanpy tutorials**: https://scanpy-tutorials.readthedocs.io/\r\n- **scverse ecosystem**: https://scverse.org/ (related tools: squidpy, scvi-tools, cellrank)\r\n- **Best practices**: Luecken & Theis (2019) \"Current best practices in single-cell RNA-seq\"",
    "Common Tasks": "```",
    "Overview": "Scanpy is a scalable Python toolkit for analyzing single-cell RNA-seq data, built on AnnData. Apply this skill for complete single-cell workflows including quality control, normalization, dimensionality reduction, clustering, marker gene identification, visualization, and trajectory analysis.",
    "Standard Analysis Workflow": "adata.obs.to_csv('results/cell_metadata.csv')\r\nadata.var.to_csv('results/gene_metadata.csv')\r\n```",
    "When to Use This Skill": "This skill should be used when:\r\n- Analyzing single-cell RNA-seq data (.h5ad, 10X, CSV formats)\r\n- Performing quality control on scRNA-seq datasets\r\n- Creating UMAP, t-SNE, or PCA visualizations\r\n- Identifying cell clusters and finding marker genes\r\n- Annotating cell types based on gene expression\r\n- Conducting trajectory inference or pseudotime analysis\r\n- Generating publication-quality single-cell plots",
    "Bundled Resources": "python my_analysis.py\r\n```\r\n\r\nThe template includes all standard steps with configurable parameters and helpful comments.",
    "Common Pitfalls and Best Practices": "1. **Always save raw counts**: `adata.raw = adata` before filtering genes\r\n2. **Check QC plots carefully**: Adjust thresholds based on dataset quality\r\n3. **Use Leiden over Louvain**: More efficient and better results\r\n4. **Try multiple clustering resolutions**: Find optimal granularity\r\n5. **Validate cell type annotations**: Use multiple marker genes\r\n6. **Use `use_raw=True` for gene expression plots**: Shows original counts\r\n7. **Check PCA variance ratio**: Determine optimal number of PCs\r\n8. **Save intermediate results**: Long workflows can fail partway through"
  }
}

---
name: scanpy
description: "Single-cell RNA-seq analysis. Load .h5ad/10X data, QC, normalization, PCA/UMAP/t-SNE, Leiden clustering, marker genes, cell type annotation, trajectory, for scRNA-seq analysis."
---

# Scanpy: Single-Cell Analysis

## Overview

Scanpy is a scalable Python toolkit for analyzing single-cell RNA-seq data, built on AnnData. Apply this skill for complete single-cell workflows including quality control, normalization, dimensionality reduction, clustering, marker gene identification, visualization, and trajectory analysis.

## When to Use This Skill

This skill should be used when:
- Analyzing single-cell RNA-seq data (.h5ad, 10X, CSV formats)
- Performing quality control on scRNA-seq datasets
- Creating UMAP, t-SNE, or PCA visualizations
- Identifying cell clusters and finding marker genes
- Annotating cell types based on gene expression
- Conducting trajectory inference or pseudotime analysis
- Generating publication-quality single-cell plots

## Quick Start

### Basic Import and Setup

```python
import scanpy as sc
import pandas as pd
import numpy as np

# Configure settings
sc.settings.verbosity = 3
sc.settings.set_figure_params(dpi=80, facecolor='white')
sc.settings.figdir = './figures/'
```

### Loading Data

```python
# From 10X Genomics
adata = sc.read_10x_mtx('path/to/data/')
adata = sc.read_10x_h5('path/to/data.h5')

# From h5ad (AnnData format)
adata = sc.read_h5ad('path/to/data.h5ad')

# From CSV
adata = sc.read_csv('path/to/data.csv')
```

### Understanding AnnData Structure

The AnnData object is the core data structure in scanpy:

```python
adata.X          # Expression matrix (cells × genes)
adata.obs        # Cell metadata (DataFrame)
adata.var        # Gene metadata (DataFrame)
adata.uns        # Unstructured annotations (dict)
adata.obsm       # Multi-dimensional cell data (PCA, UMAP)
adata.raw        # Raw data backup

# Access cell and gene names
adata.obs_names  # Cell barcodes
adata.var_names  # Gene names
```

## Standard Analysis Workflow

### 1. Quality Control

Identify and filter low-quality cells and genes:

```python
# Identify mitochondrial genes
adata.var['mt'] = adata.var_names.str.startswith('MT-')

# Calculate QC metrics
sc.pp.calculate_qc_metrics(adata, qc_vars=['mt'], inplace=True)

# Visualize QC metrics
sc.pl.violin(adata, ['n_genes_by_counts', 'total_counts', 'pct_counts_mt'],
             jitter=0.4, multi_panel=True)

# Filter cells and genes
sc.pp.filter_cells(adata, min_genes=200)
sc.pp.filter_genes(adata, min_cells=3)
adata = adata[adata.obs.pct_counts_mt < 5, :]  # Remove high MT% cells
```

**Use the QC script for automated analysis:**
```bash
python scripts/qc_analysis.py input_file.h5ad --output filtered.h5ad
```

### 2. Normalization and Preprocessing

```python
# Normalize to 10,000 counts per cell
sc.pp.normalize_total(adata, target_sum=1e4)

# Log-transform
sc.pp.log1p(adata)

# Save raw counts for later
adata.raw = adata

# Identify highly variable genes
sc.pp.highly_variable_genes(adata, n_top_genes=2000)
sc.pl.highly_variable_genes(adata)

# Subset to highly variable genes
adata = adata[:, adata.var.highly_variable]

# Regress out unwanted variation
sc.pp.regress_out(adata, ['total_counts', 'pct_counts_mt'])

# Scale data
sc.pp.scale(adata, max_value=10)
```

### 3. Dimensionality Reduction

```python
# PCA
sc.tl.pca(adata, svd_solver='arpack')
sc.pl.pca_variance_ratio(adata, log=True)  # Check elbow plot

# Compute neighborhood graph
sc.pp.neighbors(adata, n_neighbors=10, n_pcs=40)

# UMAP for visualization
sc.tl.umap(adata)
sc.pl.umap(adata, color='leiden')

# Alternative: t-SNE
sc.tl.tsne(adata)
```

### 4. Clustering

```python
# Leiden clustering (recommended)
sc.tl.leiden(adata, resolution=0.5)
sc.pl.umap(adata, color='leiden', legend_loc='on data')

# Try multiple resolutions to find optimal granularity
for res in [0.3, 0.5, 0.8, 1.0]:
    sc.tl.leiden(adata, resolution=res, key_added=f'leiden_{res}')
```

### 5. Marker Gene Identification

```python
# Find marker genes for each cluster
sc.tl.rank_genes_groups(adata, 'leiden', method='wilcoxon')

# Visualize results
sc.pl.rank_genes_groups(adata, n_genes=25, sharey=False)
sc.pl.rank_genes_groups_heatmap(adata, n_genes=10)
sc.pl.rank_genes_groups_dotplot(adata, n_genes=5)

# Get results as DataFrame
markers = sc.get.rank_genes_groups_df(adata, group='0')
```

### 6. Cell Type Annotation

```python
# Define marker genes for known cell types
marker_genes = ['CD3D', 'CD14', 'MS4A1', 'NKG7', 'FCGR3A']

# Visualize markers
sc.pl.umap(adata, color=marker_genes, use_raw=True)
sc.pl.dotplot(adata, var_names=marker_genes, groupby='leiden')

# Manual annotation
cluster_to_celltype = {
    '0': 'CD4 T cells',
    '1': 'CD14+ Monocytes',
    '2': 'B cells',
    '3': 'CD8 T cells',
}
adata.obs['cell_type'] = adata.obs['leiden'].map(cluster_to_celltype)

# Visualize annotated types
sc.pl.umap(adata, color='cell_type', legend_loc='on data')
```

### 7. Save Results

```python
# Save processed data
adata.write('results/processed_data.h5ad')

# Export metadata
adata.obs.to_csv('results/cell_metadata.csv')
adata.var.to_csv('results/gene_metadata.csv')
```

## Common Tasks

### Creating Publication-Quality Plots

```python
# Set high-quality defaults
sc.settings.set_figure_params(dpi=300, frameon=False, figsize=(5, 5))
sc.settings.file_format_figs = 'pdf'

# UMAP with custom styling
sc.pl.umap(adata, color='cell_type',
           palette='Set2',
           legend_loc='on data',
           legend_fontsize=12,
           legend_fontoutline=2,
           frameon=False,
           save='_publication.pdf')

# Heatmap of marker genes
sc.pl.heatmap(adata, var_names=genes, groupby='cell_type',
              swap_axes=True, show_gene_labels=True,
              save='_markers.pdf')

# Dot plot
sc.pl.dotplot(adata, var_names=genes, groupby='cell_type',
              save='_dotplot.pdf')
```

Refer to `references/plotting_guide.md` for comprehensive visualization examples.

### Trajectory Inference

```python
# PAGA (Partition-based graph abstraction)
sc.tl.paga(adata, groups='leiden')
sc.pl.paga(adata, color='leiden')

# Diffusion pseudotime
adata.uns['iroot'] = np.flatnonzero(adata.obs['leiden'] == '0')[0]
sc.tl.dpt(adata)
sc.pl.umap(adata, color='dpt_pseudotime')
```

### Differential Expression Between Conditions

```python
# Compare treated vs control within cell types
adata_subset = adata[adata.obs['cell_type'] == 'T cells']
sc.tl.rank_genes_groups(adata_subset, groupby='condition',
                         groups=['treated'], reference='control')
sc.pl.rank_genes_groups(adata_subset, groups=['treated'])
```

### Gene Set Scoring

```python
# Score cells for gene set expression
gene_set = ['CD3D', 'CD3E', 'CD3G']
sc.tl.score_genes(adata, gene_set, score_name='T_cell_score')
sc.pl.umap(adata, color='T_cell_score')
```

### Batch Correction

```python
# ComBat batch correction
sc.pp.combat(adata, key='batch')

# Alternative: use Harmony or scVI (separate packages)
```

## Key Parameters to Adjust

### Quality Control
- `min_genes`: Minimum genes per cell (typically 200-500)
- `min_cells`: Minimum cells per gene (typically 3-10)
- `pct_counts_mt`: Mitochondrial threshold (typically 5-20%)

### Normalization
- `target_sum`: Target counts per cell (default 1e4)

### Feature Selection
- `n_top_genes`: Number of HVGs (typically 2000-3000)
- `min_mean`, `max_mean`, `min_disp`: HVG selection parameters

### Dimensionality Reduction
- `n_pcs`: Number of principal components (check variance ratio plot)
- `n_neighbors`: Number of neighbors (typically 10-30)

### Clustering
- `resolution`: Clustering granularity (0.4-1.2, higher = more clusters)

## Common Pitfalls and Best Practices

1. **Always save raw counts**: `adata.raw = adata` before filtering genes
2. **Check QC plots carefully**: Adjust thresholds based on dataset quality
3. **Use Leiden over Louvain**: More efficient and better results
4. **Try multiple clustering resolutions**: Find optimal granularity
5. **Validate cell type annotations**: Use multiple marker genes
6. **Use `use_raw=True` for gene expression plots**: Shows original counts
7. **Check PCA variance ratio**: Determine optimal number of PCs
8. **Save intermediate results**: Long workflows can fail partway through

## Bundled Resources

### scripts/qc_analysis.py
Automated quality control script that calculates metrics, generates plots, and filters data:

```bash
python scripts/qc_analysis.py input.h5ad --output filtered.h5ad \
    --mt-threshold 5 --min-genes 200 --min-cells 3
```

### references/standard_workflow.md
Complete step-by-step workflow with detailed explanations and code examples for:
- Data loading and setup
- Quality control with visualization
- Normalization and scaling
- Feature selection
- Dimensionality reduction (PCA, UMAP, t-SNE)
- Clustering (Leiden, Louvain)
- Marker gene identification
- Cell type annotation
- Trajectory inference
- Differential expression

Read this reference when performing a complete analysis from scratch.

### references/api_reference.md
Quick reference guide for scanpy functions organized by module:
- Reading/writing data (`sc.read_*`, `adata.write_*`)
- Preprocessing (`sc.pp.*`)
- Tools (`sc.tl.*`)
- Plotting (`sc.pl.*`)
- AnnData structure and manipulation
- Settings and utilities

Use this for quick lookup of function signatures and common parameters.

### references/plotting_guide.md
Comprehensive visualization guide including:
- Quality control plots
- Dimensionality reduction visualizations
- Clustering visualizations
- Marker gene plots (heatmaps, dot plots, violin plots)
- Trajectory and pseudotime plots
- Publication-quality customization
- Multi-panel figures
- Color palettes and styling

Consult this when creating publication-ready figures.

### assets/analysis_template.py
Complete analysis template providing a full workflow from data loading through cell type annotation. Copy and customize this template for new analyses:

```bash
cp assets/analysis_template.py my_analysis.py
# Edit parameters and run
python my_analysis.py
```

The template includes all standard steps with configurable parameters and helpful comments.

## Additional Resources

- **Official scanpy documentation**: https://scanpy.readthedocs.io/
- **Scanpy tutorials**: https://scanpy-tutorials.readthedocs.io/
- **scverse ecosystem**: https://scverse.org/ (related tools: squidpy, scvi-tools, cellrank)
- **Best practices**: Luecken & Theis (2019) "Current best practices in single-cell RNA-seq"

## Tips for Effective Analysis

1. **Start with the template**: Use `assets/analysis_template.py` as a starting point
2. **Run QC script first**: Use `scripts/qc_analysis.py` for initial filtering
3. **Consult references as needed**: Load workflow and API references into context
4. **Iterate on clustering**: Try multiple resolutions and visualization methods
5. **Validate biologically**: Check marker genes match expected cell types
6. **Document parameters**: Record QC thresholds and analysis settings
7. **Save checkpoints**: Write intermediate results at key steps