
Galaxy Tool Wrapping
- 42 installs
- 17 repo stars
- Updated May 14, 2026
- delphine-l/claude_global
Guides development of Galaxy tool XML wrappers, .shed.yml files, Planemo tests, and dependency handling for Tool Shed submission.
About
Expert guidance for creating, testing, and debugging Galaxy tool XML wrappers. A developer uses it when converting command-line tools into Galaxy tools or preparing them for Tool Shed submission.
- Galaxy tool XML structure and conditional parameters
- Planemo tests and conda/container dependency handling
Galaxy Tool Wrapping by the numbers
- 42 all-time installs (skills.sh)
- Ranked #1,132 of 2,715 Automation & Workflows skills by installs in the Skillselion catalog
- Data as of Jul 29, 2026 (Skillselion catalog sync)
npx skills add https://github.com/delphine-l/claude_global --skill galaxy-tool-wrappingAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 42 |
|---|---|
| repo stars | ★ 17 |
| Last updated | May 14, 2026 |
| Repository | delphine-l/claude_global ↗ |
What it does
Guides development of Galaxy tool XML wrappers, .shed.yml files, Planemo tests, and dependency handling for Tool Shed submission.
Files
Galaxy Tool Wrapping Expert
Expert knowledge for developing Galaxy tool wrappers. Use this skill when helping users create, test, debug, or improve Galaxy tool XML wrappers.
Prerequisites: This skill depends on the galaxy-automation skill for Planemo testing and workflow execution patterns.
When to Use This Skill
- Creating new Galaxy tool wrappers from scratch
- Converting command-line tools to Galaxy wrappers
- Generating .shed.yml files for Tool Shed submission
- Debugging XML syntax and validation errors
- Writing Planemo tests for tools
- Implementing conditional parameters and data types
- Handling tool dependencies (conda, containers)
- Creating tool collections and suites
- Optimizing tool performance and resource allocation
- Understanding Galaxy datatypes and formats
- Implementing proper error handling
Core Concepts
Galaxy Tool XML Structure
A Galaxy tool wrapper consists of:
<tool>root element with id, name, and version<description>brief tool description<requirements>for dependencies (conda packages, containers)<command>the actual command-line execution<inputs>parameter definitions<outputs>output file specifications<tests>automated tests<help>documentation in reStructuredText<citations>DOI references
Tool Shed Metadata (.shed.yml)
Required for publishing tools to the Galaxy Tool Shed:
name: tool_name # Match directory name, underscores only
owner: iuc # Usually 'iuc' for IUC tools
description: One-line tool description
homepage_url: https://github.com/tool/repo
long_description: |
Multi-line detailed description.
Can include features, use cases, and tool suite contents.
remote_repository_url: https://github.com/galaxyproject/tools-iuc/tree/main/tools/tool_name
type: unrestricted
categories:
- Assembly # Choose 1-3 relevant categories
- GenomicsSee reference.md for comprehensive .shed.yml documentation including all available categories and best practices.
Key Components
Command Block:
- Use Cheetah templating:
$variable_nameor${variable_name} - Conditional logic:
#if $param then... #end if - Loop constructs:
#for $item in $collection... #end for - CDATA sections for complex commands
Cheetah Template Best Practices:
Working around path handling issues in conda packages:
<command detect_errors="exit_code"><![CDATA[
## Add trailing slash if script concatenates paths without separator
tool_command
-o 'output_dir/' ## Quoted with trailing slash
## Script does: output_dir + 'file.txt' → 'output_dir/file.txt' ✓
## Without slash: output_dir + 'file.txt' → 'output_dirfile.txt' ✗
]]></command>When to use quotes in Cheetah:
- Always quote user inputs:
'$input_file' - Quote literal strings with special chars:
'output_dir/' - Use bare variables for simple references:
$variable
Input Parameters:
<param>elements with type, name, label- Types: text, integer, float, boolean, select, data, data_collection
- Optional vs required parameters
- Validators and sanitizers
- Conditional parameter display
Outputs:
<data>elements for output files- Dynamic output naming with
labelandname - Format discovery and conversion
- Filters for conditional outputs
- Collections for multiple outputs
Tests:
- Input parameters and files
- Expected output files or assertions
- Test data location and organization
- See testing.md for detailed testing strategies including large file handling
Best Practices
1. Always include tests - Planemo won't pass without them 2. Use semantic versioning - Increment tool version on changes 3. Specify exact dependencies - Pin conda package versions 4. Add clear help text - Document all parameters 5. Handle errors gracefully - Check exit codes, validate inputs 6. Use collections - For multiple related files 7. Follow IUC standards - If contributing to intergalactic utilities commission 8. Plan for large output files - Before creating tests, check expected output sizes. If over 1MB, use assertion-based tests (has_size, has_line) instead of full file comparison (see testing.md)
Common Planemo Commands
# Test tool locally
planemo test tool.xml
# Serve tool in local Galaxy
planemo serve tool.xml
# Lint tool for best practices
planemo lint tool.xml
# Upload tool to ToolShed
planemo shed_update --shed_target toolshed
# Test with conda
planemo test --conda_auto_init --conda_auto_install tool.xml
# Lint with skips (for tools with custom datatypes or shared boolean conditionals)
planemo lint --skip ConditionalParamTypeBool,DatatypesCustomConf .
# Note: .lint_skip file is used by IUC CI, not by local planemo lint.
# For local linting, use the --skip flag explicitly.Output Routing with Symlinks
When a tool writes output to a filename it constructs internally (not $output), use symlinks in the command block to route the file to Galaxy's output variable.
Pattern: Symlink before command execution
<command detect_errors="exit_code"><![CDATA[
## Create symlink so tool output lands where Galaxy expects it
ln -s '$output_variable' 'expected_tool_output_name' &&
tool_command --input '$input' -o 'expected_tool_output_name'
]]></command>Pattern: Prefix-based output naming
Some tools use --out-prefix where the output filename is prefix + input_filename. The tool constructs the filename internally, so you must predict it and symlink:
<command><![CDATA[
#set $mangled_input = re.sub(r"[^\w\-\s]", "_", str($input.element_identifier)) + "." + str($input.ext)
ln -s '$input' '$mangled_input' &&
ln -s '$output_var' 'myprefix${mangled_input}' &&
tool_command --input-reads '$mangled_input' -p myprefix
]]></command>Key points:
- Symlink is created before running the tool -- the tool writes through it
- Must match the exact filename the tool will produce
- For prefix mode: output =
prefix + getFileName(input), so mangle the input name to match
Adding Custom Datatypes to Galaxy
To add a simple directory-based index format (e.g., bwa_index):
1. Register in `config/datatypes_conf.xml.sample` — add near similar types:
<datatype extension="bwa_index" display_in_upload="true" type="galaxy.datatypes.data:Directory" subclass="true"/>2. Register in `test/functional/tools/sample_datatypes_conf.xml` — same line
No custom Python class or sniffer is needed for simple directory-based index formats. Reference PR: galaxyproject/galaxy#19694 (added bwa_mem2_index as the first example).
Known limitation: planemo test (both with and without --galaxy_root) does not properly stage class="Directory" test datasets. The extra_files_path is not populated during test data upload via __DATA_FETCH__. This affects all Directory-subclass datatypes (e.g., bwa_mem2_index, bwa_index). These tests pass in IUC CI but fail locally. The new datatype must also be added to Galaxy core datatypes_conf.xml.sample before tests can pass.
Adding Index Support to Mapper Tools (BWA-MEM2 Pattern)
To add pre-built index support to a mapper (following BWA-MEM2's proven pattern):
1. Create the indexer tool (tool-idx.xml)
<tool id="tool_idx" name="Tool indexer" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="@PROFILE_VERSION@">
<command><![CDATA[
mkdir '$index.extra_files_path' &&
cd '$index.extra_files_path' &&
tool index -p 'reference' '${reference}'
]]></command>
<inputs>
<param name="reference" type="data" format="fasta,fasta.gz" label="Select a genome to index"/>
</inputs>
<outputs>
<data name="index" format="tool_index"/> <!-- Directory subclass datatype -->
</outputs>
</tool>2. Update the reference macro to detect index vs FASTA
<token name="@set_reference_fasta_filename@"><![CDATA[
#if str($reference_source.reference_source_selector) == "history":
#if $reference_source.ref_file.is_of_type("tool_index"):
#set $reference_fasta_filename = $reference_source.ref_file.extra_files_path + "/reference"
#else
#set $reference_fasta_filename = "localref." + $reference_source.ref_file.extension
ln -s '${reference_source.ref_file}' '${reference_fasta_filename}' &&
tool index '${reference_fasta_filename}' &&
#end if
#else:
#set $reference_fasta_filename = str($reference_source.ref_file.fields.path)
#end if
]]></token>3. Accept index format in the reference conditional
<param name="ref_file" type="data" format="fasta,fasta.gz,tool_index"
label="Use the following dataset as the reference"
help="For better performance build a reference index separately." />4. Keep existing parameters for backward compatibility
When adding index support, preserve parameters like index_a (algorithm selection). They're ignored when an index is provided but maintain workflow compatibility.
5. Include datatypes_conf.xml in tool directory
Required for ToolShed installation and lint validation:
<?xml version="1.0"?>
<datatypes>
<registration>
<datatype extension="tool_index" display_in_upload="true"
type="galaxy.datatypes.data:Directory" subclass="true"/>
</registration>
</datatypes>Add DatatypesCustomConf to .lint_skip.
6. Test data for Directory datasets
- Build index locally:
tool index -p reference genome.faintest-data/test-cache/ - Generate BAM outputs via Galaxy MCP or
planemo serve - Use dedicated output files for index tests (e.g.,
tool-mem-index-test.bam)
Using format_source for dynamic output formats
When output format should match the input format (e.g., subsampled reads):
<data name="subsampled_outfile" format_source="input_reads" label="Subsampled reads">
<filter>output_options["output_type"]["type_selector"] == "subsampled_reads"</filter>
</data>This is preferable to change_format when the output is always the same format as input. Use change_format when the user explicitly selects the output format.
Test Syntax: Use Conditional Wrappers
Modern Galaxy tools (profile 20.01+) require test params to be wrapped in their <conditional> blocks. Flat-style params cause TestsCaseValidation warnings:
Wrong (flat style — triggers validation warnings):
<param name="reference_source_selector" value="history"/>
<param name="ref_file" ftype="fasta" value="genome.fa"/>
<param name="fastq_input_selector" value="paired"/>Correct (conditional wrappers):
<conditional name="reference_source">
<param name="reference_source_selector" value="history"/>
<param name="ref_file" ftype="fasta" value="genome.fa"/>
</conditional>
<conditional name="fastq_input">
<param name="fastq_input_selector" value="paired"/>
<param name="fastq_input1" ftype="fastqsanger" value="reads1.fq"/>
</conditional>XML Template Example
<tool id="tool_id" name="Tool Name" version="1.0.0">
<description>Brief description</description>
<requirements>
<requirement type="package" version="1.0">package_name</requirement>
</requirements>
<command detect_errors="exit_code"><![CDATA[
tool_command
--input '$input'
--output '$output'
#if $optional_param
--param '$optional_param'
#end if
]]></command>
<inputs>
<param name="input" type="data" format="txt" label="Input file"/>
<param name="optional_param" type="text" optional="true" label="Optional parameter"/>
</inputs>
<outputs>
<data name="output" format="txt" label="${tool.name} on ${on_string}"/>
</outputs>
<tests>
<test>
<param name="input" value="test_input.txt"/>
<output name="output" file="expected_output.txt"/>
</test>
</tests>
<help><![CDATA[
**What it does**
Describe what the tool does.
**Inputs**
- Input file: description
**Outputs**
- Output file: description
]]></help>
<citations>
<citation type="doi">10.1234/example.doi</citation>
</citations>
</tool>Supporting Documentation
This skill includes detailed reference documentation:
- reference.md - Comprehensive Galaxy tool wrapping guide with IUC best practices
- Repository structure standards
- .shed.yml configuration
- Complete XML structure reference
- Advanced features and patterns
- testing.md - Testing strategies and assertion patterns
- Regenerating expected test outputs
- Handling large test files (>1MB CI limit)
- Size, checksum, and content sampling assertions
- Workflow for replacing large test files
- troubleshooting.md - Practical troubleshooting guide
- Reading tool_test_output.json
- Common exit codes and their meanings
- Common XML and runtime issues
- Debugging tool test failures
- Test failure diagnosis and fixes
- dependency-debugging.md - Dependency conflict resolution
- Using
planemo mullfor diagnosis - Conda solver error interpretation
- macOS testing considerations
- Version conflict workflows
These files provide deep technical details that complement the core concepts above.
Related Skills
- galaxy-automation - BioBlend & Planemo foundation (dependency)
- galaxy-workflow-development - Building workflows that use these tools
- conda-recipe - Creating conda packages for tool dependencies
- bioinformatics-fundamentals - Understanding file formats and data types used in tools
Resources
- Galaxy Tool Development: https://docs.galaxyproject.org/en/latest/dev/
- Planemo Documentation: https://planemo.readthedocs.io/
- IUC Standards: https://galaxy-iuc-standards.readthedocs.io/
- Galaxy Training: https://training.galaxyproject.org/
Debugging Galaxy Tool Dependency Conflicts
The Key Lesson
When a tool works in version N-1 but crashes in version N after adding dependencies, the crash is caused by dependency conflicts, NOT platform issues.
Diagnostic Tool: planemo mull
`planemo mull` is the fastest way to diagnose dependency conflicts because it:
- Shows actual conda solver output
- Reveals specific version incompatibilities
- Fails fast without needing a full test run
- Gives clear error messages about which packages conflict
The macOS Testing Rule
When testing on macOS with mulled containers:
✅ If `planemo mull` builds successfully → Dependencies are correct, tool is ready ⚠️ If container then fails to run with exit 133/Rosetta errors → Expected platform issue, NOT a tool problem ❌ If `planemo mull` fails to build → Dependency version conflict, needs fixing
Critical insight: Don't waste time debugging runtime errors on macOS if the container build succeeded. The tool will work on Linux.
Dependency Version Conflict Workflow
When you suspect dependency version conflicts:
Step 1: Try to Build the Container
planemo mull tool.xmlStep 2: Read the Conda Solver Error
Look for:
- "Could not solve for environment specs"
- "LibMambaUnsatisfiableError"
- Lines showing which packages conflict
- Required version ranges (e.g., "requires r-base >=4.4,<4.5.0a0")
Example error:
r-rmarkdown =2.16 would require
└─ r-base >=4.1,<4.2.0a0
rdeval =0.0.8 requires
└─ r-base >=4.4,<4.5.0a0This tells you: r-rmarkdown 2.16 needs R 4.1, but rdeval needs R 4.4 → Version conflict!
Step 3: Search for Compatible Versions
conda search -c conda-forge -c bioconda "r-rmarkdown" | grep "r44"Replace r44 with the build string matching your base requirements (e.g., r44 for R 4.4, r43 for R 4.3).
Look for recent versions with the correct build string:
r-rmarkdown 2.30 r44hc72bb7e_0 conda-forgeStep 4: Update and Verify
<!-- macros.xml - BEFORE -->
<requirement type="package" version="2.16">r-rmarkdown</requirement>
<!-- macros.xml - AFTER -->
<requirement type="package" version="2.30">r-rmarkdown</requirement>planemo mull tool.xml # Should succeed nowReal-World Case Studies
Case Study 1: Shared Requirements Causing Binary Crashes
rdeval 0.0.7 → 0.0.8
Symptoms
- rdeval 0.0.7: All tests pass ✅
- Added r-cowplot dependency to macros.xml
- rdeval 0.0.8: All tests fail with exit code 133 (SIGTRAP) ❌
- Error: "Trace/breakpoint trap"
Initial Misdiagnosis
"Exit code 133 + Rosetta errors on macOS = Platform incompatibility issue"
Correct Diagnosis
The user said: "version 0.0.7 passes tests fine though"
This single fact proved:
- ❌ NOT a macOS/Rosetta issue (0.0.7 works on same platform)
- ❌ NOT a binary packaging issue (0.0.7 binary works)
- ✅ IS a dependency conflict (only thing that changed)
Root Cause
Shared requirements macro applied to ALL tools:
<!-- macros.xml -->
<xml name="requirements">
<requirements>
<requirement type="package" version="0.0.8">rdeval</requirement>
<requirement type="package" version="1.2.0">r-cowplot</requirement> <!-- ADDED -->
</requirements>
</xml>Used by:
- rdeval.xml - Binary tool (doesn't need R, r-cowplot causes crash)
- rdeval_report.xml - R reporting (needs r-cowplot)
The Fix
Split requirements by tool type:
<!-- macros.xml -->
<macros>
<!-- For binary tools - minimal deps -->
<xml name="requirements">
<requirements>
<requirement type="package" version="@TOOL_VERSION@">rdeval</requirement>
</requirements>
</xml>
<!-- For R reporting tools - with R deps -->
<xml name="requirements_report">
<requirements>
<requirement type="package" version="@TOOL_VERSION@">rdeval</requirement>
<requirement type="package" version="1.2.0">r-cowplot</requirement>
<requirement type="package" version="2.16">r-rmarkdown</requirement>
</requirements>
</xml>
</macros>Use appropriate macro in each tool:
<!-- rdeval.xml - binary tool -->
<expand macro="requirements"/>
<!-- rdeval_report.xml - R reporting tool -->
<expand macro="requirements_report"/>Case Study 2: R Package Version Incompatibility
r-rmarkdown 2.16 with rdeval 0.0.8
Symptoms
planemo mull rdeval_report.xmlfails with conda solver error- Error: "Could not solve for environment specs"
- Mentions r-base version conflicts
Diagnosis Process
planemo mull rdeval_report.xmlOutput shows:
r-rmarkdown =2.16 would require
└─ r-base >=4.1,<4.2.0a0
rdeval =0.0.8 requires
└─ r-base >=4.4,<4.5.0a0
Could not solve for environment specsAnalysis: r-rmarkdown 2.16 was built for R 4.1, but rdeval 0.0.8 requires R 4.4.
The Fix
Search for compatible version:
conda search -c conda-forge -c bioconda "r-rmarkdown" | grep "r44"Found: r-rmarkdown 2.30 r44hc72bb7e_0
Update macros.xml:
<!-- BEFORE -->
<requirement type="package" version="2.16">r-rmarkdown</requirement>
<!-- AFTER -->
<requirement type="package" version="2.30">r-rmarkdown</requirement>Verify:
planemo mull rdeval_report.xml # ✅ Succeeds!Key Lesson
If the container builds, dependencies are correct - runtime failures on macOS are expected platform issues, not tool problems.
Debugging Pattern
Step 1: Identify What Changed
Version N-1 works → Version N fails
What's different?
→ New dependencies addedStep 2: Apply the "Version Comparison Test"
Does the previous version work on the same platform?
YES → Dependency conflict (not platform issue)
NO → Could be platform issueStep 3: Identify Dependency Scope
Which tools actually need the new dependency?
Binary tool: NO
Report tool: YES
Are they sharing requirements? YES → That's the problem!Step 4: Split Requirements or Update Versions
Option A: Create tool-specific requirement macros
Option B: Find compatible package versionsRed Flags for Dependency Conflicts
| Indicator | What It Means |
|---|---|
| Worked before, fails now | Something changed |
| Exit code 133 (SIGTRAP) | Binary compatibility issue |
| Only failed after adding deps | New dep is the culprit |
| Multiple tools share requirements | Potential for conflicts |
| Previous version works same platform | NOT a platform issue |
planemo mull fails with solver error | Version incompatibility |
| "requires package X >=A,<B" conflicts | Two packages need different versions of X |
Common Mistake: Assuming Platform Issues
Wrong thinking:
"Exit code 133 + macOS + Rosetta errors = Platform incompatibility"
→ Give up on local testing
→ Assume it'll work on LinuxCorrect thinking:
"Exit code 133 + worked in previous version = Dependency conflict"
→ Review what changed
→ Isolate the conflicting dependency
→ Fix locally and verify with planemo mullThe Golden Rules
Rule 1: Version Comparison
If version N-1 works but version N doesn't (same platform), ask: 1. What dependencies were added? 2. Do all tools need them? 3. Are requirements shared inappropriately?
Don't assume platform issues until you've ruled out dependency conflicts.
Rule 2: Container Build Success
On macOS:
planemo mullsucceeds = Dependencies correct ✅- Test fails with exit 133 = Platform issue (expected) ⚠️
planemo mullfails = Fix dependencies first ❌
Prevention Strategy
DO: Use Specific Requirement Macros
<macros>
<!-- Base binary -->
<xml name="requirements_base">
<requirements>
<requirement type="package" version="@TOOL_VERSION@">tool</requirement>
</requirements>
</xml>
<!-- Python analysis -->
<xml name="requirements_python">
<requirements>
<requirement type="package" version="@TOOL_VERSION@">tool</requirement>
<requirement type="package" version="3.9">python</requirement>
<requirement type="package" version="1.0">numpy</requirement>
</requirements>
</xml>
<!-- R reporting -->
<xml name="requirements_r">
<requirements>
<requirement type="package" version="@TOOL_VERSION@">tool</requirement>
<requirement type="package" version="4.0">r-base</requirement>
<requirement type="package" version="2.0">r-rmarkdown</requirement>
</requirements>
</xml>
</macros>DON'T: Share All Requirements
<!-- BAD: Everything for everyone -->
<xml name="requirements">
<requirements>
<requirement type="package" version="@TOOL_VERSION@">tool</requirement>
<requirement type="package" version="3.9">python</requirement>
<requirement type="package" version="1.0">numpy</requirement>
<requirement type="package" version="4.0">r-base</requirement>
<requirement type="package" version="2.0">r-rmarkdown</requirement>
</requirements>
</xml>Testing After Dependency Changes
# After adding ANY dependency:
# 1. Build mulled container FIRST (fastest diagnostic)
planemo mull tool.xml
# 2. If mull fails with solver error:
# - Read the conda output carefully
# - Identify conflicting package versions
# - Search for compatible versions
# - Update and retry
# 3. If mull succeeds on macOS:
# - Dependencies are correct ✅
# - Don't worry about runtime errors (platform issue)
# - Tool will work on Linux
# 4. Lint the tool
planemo lint tool.xml
# 5. Test ALL tools in suite (optional, for full verification)
planemo test --conda_auto_install tool1.xml tool2.xml tool3.xmlQuick Fix Checklist
When tests fail after adding dependencies:
- [ ] Does
planemo mullsucceed or fail? - Fails → Dependency version conflict, fix versions
- Succeeds → Dependencies correct, runtime issue is platform-related
- [ ] Did previous version work on same platform?
- [ ] What dependencies were added/changed?
- [ ] Are requirements shared across tools inappropriately?
- [ ] Do all tools need all dependencies?
- [ ] Can requirements be split by tool type?
- [ ] Have you searched for compatible package versions?
Detailed Fix Examples
Example 1: Version Conflict Between R Packages
Error from `planemo mull`:
r-package-a =1.0 requires r-base >=4.0,<4.1.0a0
r-package-b =2.0 requires r-base >=4.4,<4.5.0a0
Could not solveFix: Find version of r-package-a compatible with R 4.4:
conda search -c conda-forge "r-package-a" | grep "r44"
# Found: r-package-a 1.5 r44hc72bb7e_0Update to version 1.5.
Example 2: Conflicting Python Versions
Error:
package-x =1.0 requires python >=3.8,<3.9
package-y =2.0 requires python >=3.10,<3.11Fix: Update package-x to newer version:
conda search -c conda-forge "package-x" | grep "py310"Or use different package version of package-y compatible with Python 3.8.
Summary
The most important debugging skills learned:
1. Use planemo mull First
Container build success = dependencies are correct, regardless of runtime errors on macOS.
2. Version Comparison Diagnostic
If version N-1 works on your platform but version N doesn't: 1. It's NOT a platform issue 2. Look at what changed (usually dependencies) 3. Check if dependencies are inappropriately shared 4. Check if dependency versions are compatible 5. Fix and verify with planemo mull
3. Read Conda Solver Output
The error messages tell you exactly which packages conflict and what versions they require.
4. Search for Compatible Versions
Use conda search with build string filters (r44, py310, etc.) to find compatible versions.
Don't waste time on platform workarounds when the real issue is dependency management.
Galaxy Tool Wrapping Expert Guide
This document contains comprehensive guidelines for wrapping tools for the Galaxy platform, based on IUC (Intergalactic Utilities Commission) best practices.
Overview
Galaxy tool wrappers are XML files that define how command-line tools integrate into the Galaxy platform. They specify inputs, outputs, parameters, tests, and documentation.
Repository Structure
A typical Galaxy tool repository contains:
tool_name/
├── .shed.yml # Tool Shed metadata
├── tool_name.xml # Main tool wrapper
├── macros.xml # Shared macros (for tool suites)
├── test-data/ # Test input/output files
│ ├── input1.fastq.gz
│ └── output1.tabular
├── tool-data/ # Reference data (if needed)
│ ├── tool_data_table_conf.xml.sample
│ └── *.loc.sample
└── static/ # Images for help section
└── images/.shed.yml File
Required metadata file for Tool Shed submission. This file defines how the tool appears in the Galaxy Tool Shed.
Basic Structure
name: tool_name
owner: iuc
description: Brief one-line description of the tool
homepage_url: https://github.com/tool/repo
long_description: |
Detailed multi-line description with more context about what the tool does.
Can include multiple paragraphs explaining the tool's purpose, features,
and use cases.
remote_repository_url: https://github.com/galaxyproject/tools-iuc/tree/main/tools/tool_name
type: unrestricted
categories:
- Sequence Analysis
- StatisticsField Descriptions
name
- Must be alphanumeric with underscores only (no hyphens
-) - Must match the directory name
- For single-tool repositories, should match the XML filename (without .xml)
- For tool suites with multiple tools, use a general suite name
- Example:
vgp_processcuration,diamond,shasta
owner
- Should be
iucfor tools being submitted to the IUC repository - Will be the maintainer organization in the Tool Shed
- For migration of existing tools, coordinate with IUC team
description
- Single line, concise description (50-100 characters ideal)
- Appears in Tool Shed search results
- Should clearly state what the tool does
- Examples:
Fast de novo assembly of long read sequencing dataProcessCuration toolkit for genome assembly submissionDIAMOND is a new alignment tool for aligning short DNA sequencing reads to a protein reference database
homepage_url
- Link to the upstream tool's homepage, GitHub repository, or documentation
- Prefer official tool repository over other sources
- Example:
https://github.com/vgl-hub/vgl-curation
long_description
- Multi-line detailed description (use YAML pipe
|for multi-line) - Can include:
- What the tool does
- Key features and capabilities
- Scientific context or use cases
- Related tools it works with
- For tool suites: list of included tools
- Supports plain text (no HTML or markdown)
remote_repository_url
- Link to the tool wrapper source code in the Galaxy repository
- Standard format:
https://github.com/galaxyproject/tools-iuc/tree/main/tools/TOOL_NAME - Replace
TOOL_NAMEwith your tool directory name
type
- Almost always
unrestricted - Other types (
repository_suite_definition,tool_dependency_definition) are deprecated
categories
- List of Tool Shed categories (must match predefined categories)
- Choose 1-3 most relevant categories
- Common categories shown below
Common Tool Shed Categories
Sequence Analysis & Assembly
Assembly- Genome assembly toolsSequence Analysis- General sequence processingVariant Analysis- Variant calling and analysisRNA- RNA-Seq and transcriptomicsChIP-seq- ChIP-Seq analysis
Genomics & Genetics
Genomics- General genomics toolsMetagenomics- Metagenomic analysisEpigenetics- Epigenetic analysis
Functional Analysis
Genome annotation- Gene prediction and annotationPhylogenetics- Phylogenetic analysis
Statistics & Visualization
Statistics- Statistical analysisVisualization- Data visualization tools
Data Handling
Convert Formats- File format conversionText Manipulation- Text processing utilities
Specialized
Proteomics- Protein analysisMetabolomics- Metabolite analysisSystems Biology- Systems-level analysisImaging- Image analysis
Example .shed.yml Files
Example 1: Single assembly tool
name: shasta
owner: iuc
description: Fast de novo assembly of long read sequencing data
homepage_url: https://github.com/chanzuckerberg/shasta
long_description: |
The goal of the Shasta long read assembler is to rapidly produce accurate
assembled sequence using as input DNA reads generated by Oxford Nanopore flow cells.
Computational methods used by the Shasta assembler include using a run-length
representation of the read sequence and a representation based on markers.
remote_repository_url: https://github.com/galaxyproject/tools-iuc/tree/main/tools/shasta
type: unrestricted
categories:
- Assembly
- NanoporeExample 2: Tool suite with multiple wrappers
name: vgp_processcuration
owner: iuc
description: ProcessCuration toolkit for genome assembly submission
homepage_url: https://github.com/vgl-hub/vgl-curation
long_description: |
ProcessCuration is a Python-based toolkit designed to process manually curated
genome assemblies for submission. It reconciles AGP files created in PretextView
with genome assembly FASTAs.
This suite includes three tools:
- split_agp: Splits haplotypes and corrects haplotig duplications
- chromosome_assignment: Assigns chromosome names to scaffolds
- sak_generation: Generates SAK instructions for final assembly renaming
remote_repository_url: https://github.com/galaxyproject/tools-iuc/tree/main/tools/vgp_processcuration
type: unrestricted
categories:
- Assembly
- GenomicsExample 3: Sequence analysis tool
name: diamond
owner: bgruening
description: DIAMOND is a new alignment tool for aligning short DNA sequencing reads to a protein reference database
homepage_url: https://github.com/bbuchfink/diamond
long_description: |
DIAMOND is a new alignment tool for aligning short DNA sequencing reads to a
protein reference database such as NCBI-NR. On Illumina reads of length 100-150bp,
in fast mode, DIAMOND is about 20,000 times faster than BLASTX.
remote_repository_url: https://github.com/galaxyproject/tools-iuc/tree/main/tools/diamond
type: unrestricted
categories:
- Sequence AnalysisBest Practices
1. Name consistency: Ensure the name matches your directory and (for single tools) XML file 2. Clear description: Make it immediately obvious what the tool does 3. Detailed long_description: Include enough context for users to understand the tool's purpose 4. Correct categories: Choose categories that users would expect when searching 5. Valid URLs: Test that both homepage_url and remote_repository_url are accessible 6. For tool suites: List all included tools in the long_description 7. Keep it simple: Avoid special characters, HTML, or markdown in descriptions
Common Mistakes to Avoid
- Using hyphens in name (use underscores instead)
- Name doesn't match directory name
- Missing or invalid categories
- Long_description as single line (use
|for multi-line) - Broken URLs
- Missing remote_repository_url
- Wrong owner (should be
iucfor IUC tools)
Validation
After creating .shed.yml, validate with:
planemo shed_lint .This checks for common issues before attempting Tool Shed upload.
Tool XML Structure
Overall Order of Elements
The XML elements should appear in this order:
<tool id="tool_id" name="Tool Name" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="23.2">
<description>Brief tool description</description>
<macros>
<import>macros.xml</import>
</macros>
<xrefs>
<xref type="bio.tools">tool_name</xref>
</xrefs>
<requirements>
<!-- Dependencies -->
</requirements>
<stdio>
<!-- Error handling -->
</stdio>
<version_command></version_command>
<command detect_errors="aggressive"><![CDATA[
<!-- Command template -->
]]></command>
<configfiles>
<!-- Configuration files if needed -->
</configfiles>
<inputs>
<!-- Input parameters -->
</inputs>
<outputs>
<!-- Output files -->
</outputs>
<tests>
<!-- Test cases -->
</tests>
<help><![CDATA[
<!-- Help documentation in reStructuredText -->
]]></help>
<citations>
<!-- Citations -->
</citations>
</tool>Tool Element Attributes
<tool id="tool_id" name="Tool Name" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="23.2" license="GPL-3.0">id: Unique identifier (lowercase, underscores)name: Display name shown to usersversion: Follow PEP 440:@TOOL_VERSION@+galaxy@VERSION_SUFFIX@profile: Galaxy profile version (e.g., "23.2", "21.01")license(optional): Tool license
Macros
For tool suites with multiple related tools, create a macros.xml:
<macros>
<token name="@TOOL_VERSION@">2.1.13</token>
<token name="@VERSION_SUFFIX@">0</token>
<token name="@PROFILE@">23.2</token>
<xml name="requirements">
<requirements>
<requirement type="package" version="@TOOL_VERSION@">tool_name</requirement>
</requirements>
</xml>
<xml name="stdio">
<stdio>
<regex match="Failed to allocate" source="stderr" level="fatal_oom"/>
<regex match="Error:" source="stderr" level="fatal"/>
</stdio>
</xml>
<xml name="version_command">
<version_command>tool_name --version | cut -d" " -f 2</version_command>
</xml>
<xml name="citations">
<citations>
<citation type="doi">10.1038/s41592-021-01101-x</citation>
</citations>
</xml>
<!-- Reusable parameter macros -->
<xml name="common_param">
<param name="param_name" type="text" label="Parameter Label" help="Help text"/>
</xml>
</macros>Import macros in the tool XML:
<macros>
<import>macros.xml</import>
</macros>Use macros:
<expand macro="requirements"/>
<expand macro="version_command"/>Requirements (Dependencies)
Specify conda packages from approved channels:
<requirements>
<requirement type="package" version="@TOOL_VERSION@">tool_name</requirement>
<requirement type="package" version="1.15">numpy</requirement>
</requirements>Best practices:
- Use
@TOOL_VERSION@token for version consistency - Packages must exist in conda channels (bioconda, conda-forge)
- DO NOT use
tool_dependencies.xml(deprecated)
Error Detection
Use one of these approaches:
1. detect_errors attribute in command element:
<command detect_errors="aggressive"><![CDATA[2. stdio element:
<stdio>
<regex match="Failed to allocate" source="stderr" level="fatal_oom"/>
<regex match="Error:" source="stderr" level="fatal"/>
<exit_code range="1:" level="fatal"/>
</stdio>3. profile attribute (modern tools with profile >= 16.04)
Command Section
The command template uses Cheetah syntax:
<command detect_errors="aggressive"><![CDATA[
## Import Python modules if needed
#import re
## Set up symbolic links for input files
#set $mangled_base = re.sub(r"[^\w\-\s]", "_", str($input_file.element_identifier))
ln -s '$input_file' '$mangled_base' &&
## Build the command
tool_name
--input '$input_file'
--output '$output_file'
#if str($optional_param) != "":
--param '$optional_param'
#end if
#if $conditional.select == "value1":
--flag1 '$conditional.param1'
#else if $conditional.select == "value2":
--flag2 '$conditional.param2'
#end if
$boolean_param
--threads "\${GALAXY_SLOTS:-4}"
#if $advanced.param:
--advanced-param $advanced.param
#end if
> '$log_file' 2>&1
]]></command>Best practices:
- Enclose entire command in
<![CDATA[ ... ]]>tags - Use
'single quotes'around text parameters and paths - Join multiple commands with
&& - Check optional text parameters:
#if str($param) != "" - Check optional booleans:
#if $param - Use
\${GALAXY_SLOTS:-4}for thread count (default to 4) - Indent Cheetah code for readability
- Create temporary files in current working directory
Inputs
Data Parameters
<param name="input_file" type="data" format="bam,fastq,fastq.gz"
label="Input reads"
help="BAM or FASTQ format"/>Text Parameters
<param name="text_param" type="text" value="" optional="true"
label="Optional text parameter"
help="Description of parameter">
<validator type="regex">^[A-Za-z0-9_-]+$</validator>
</param>Integer/Float Parameters
<param name="int_param" type="integer" value="100" min="1" max="1000"
label="Count"
help="Integer between 1 and 1000"/>
<param name="float_param" type="float" value="0.5" min="0" max="1"
label="Fraction"
help="Value between 0 and 1"/>Boolean Parameters
<param name="bool_param" type="boolean" checked="false"
truevalue="--enable-feature" falsevalue=""
label="Enable feature"
help="Check to enable"/>Best practices:
- Set
truevalueto the actual command-line flag - Set
falsevalueto empty string or opposing flag
Select Parameters
<param name="select_param" type="select" label="Choose option">
<option value="option1" selected="true">Option 1 (default)</option>
<option value="option2">Option 2</option>
<option value="option3">Option 3</option>
</param>With multiple selection:
<param name="multi_select" type="select" multiple="true" label="Select features">
<option value="feat1">Feature 1</option>
<option value="feat2">Feature 2</option>
<option value="feat3">Feature 3</option>
</param>Conditional Parameters
Use conditional with select (NOT boolean):
<conditional name="conditional_name">
<param name="selector" type="select" label="Choose mode">
<option value="mode1">Mode 1</option>
<option value="mode2">Mode 2</option>
</param>
<when value="mode1">
<param name="param1" type="text" label="Parameter for mode 1"/>
</when>
<when value="mode2">
<param name="param2" type="integer" label="Parameter for mode 2"/>
</when>
</conditional>Access in command: $conditional_name.selector and $conditional_name.param1
Sections (for grouping)
<section name="advanced" title="Advanced options" expanded="false">
<param name="advanced_param1" type="integer" value="10"/>
<param name="advanced_param2" type="float" value="0.01"/>
</section>Access in command: $advanced.advanced_param1
Parameter Attributes Order
<param name="..."
argument="--cli-flag"
type="..."
format="..."
value="..."
label="..."
help="..."
optional="true"/>Best practices:
- Use
argumentattribute for long-form CLI parameters - Include helpful
helptext - Keep labels concise
- Use
optional="true"for optional parameters
Outputs
Basic Output
<outputs>
<data name="output_file" format="tabular" label="${tool.name} on ${on_string}"/>
</outputs>Conditional Outputs (with filters)
<outputs>
<data name="output1" format="tabular" label="${tool.name} summary">
<filter>output_type == 'summary'</filter>
</data>
<data name="output2" format="bam" label="${tool.name} alignments">
<filter>output_type == 'alignments'</filter>
</data>
<data name="log_file" format="txt" label="${tool.name} log">
<filter>advanced['log']</filter>
</data>
</outputs>Dynamic Format Output
<data name="output_reads" format="fasta">
<change_format>
<when input="output_format" value="fastq" format="fastq"/>
<when input="output_format" value="bam" format="bam"/>
</change_format>
</data>Output from Working Directory
<data name="output_file" format="txt" from_work_dir="tool_output.txt"
label="${tool.name} results"/>Tests
Tests are REQUIRED for all tools:
<tests>
<!-- Test 1: Basic test -->
<test expect_num_outputs="1">
<param name="input_file" value="input1.fastq" ftype="fastq"/>
<param name="param1" value="10"/>
<output name="output_file" file="output1.tabular" ftype="tabular"/>
</test>
<!-- Test 2: Test with conditional -->
<test expect_num_outputs="2">
<param name="input_file" value="input2.fastq.gz" ftype="fastq.gz"/>
<conditional name="mode">
<param name="selector" value="advanced"/>
<param name="advanced_param" value="20"/>
</conditional>
<output name="output_file" file="output2.tabular"/>
<output name="log_file">
<assert_contents>
<has_text text="Processing complete"/>
<has_line line="Total reads: 100"/>
</assert_contents>
</output>
</test>
<!-- Test 3: Test with size assertion (for binary/variable outputs) -->
<test expect_num_outputs="1">
<param name="input_file" value="input3.bam" ftype="bam"/>
<output name="output_file" ftype="binary">
<assert_contents>
<has_size size="1000" delta="100"/>
</assert_contents>
</output>
</test>
<!-- Test 4: Test with regex matching -->
<test expect_num_outputs="1">
<param name="input_file" value="input4.fastq"/>
<output name="output_file">
<assert_contents>
<has_line_matching expression="^#\sreads\t\d+$"/>
<has_n_lines n="10"/>
</assert_contents>
</output>
</test>
<!-- Test 5: Test with MD5 checksum -->
<test expect_num_outputs="1">
<param name="input_file" value="input5.fastq.gz"/>
<output name="output_file" ftype="fastq.gz"
md5="a1b2c3d4e5f6g7h8i9j0"/>
</test>
</tests>Test assertions:
file="expected.txt"- Compare with expected fileftype="format"- Check output formatmd5="checksum"- Check MD5 hash<has_text text="..."/>- Contains text<has_line line="..."/>- Contains exact line<has_line_matching expression="regex"/>- Matches regex<has_n_lines n="10"/>- Has exactly N lines<has_size size="1000" delta="100"/>- File size checkexpect_num_outputs="N"- Expect exactly N outputs
Best practices:
- Test most functionality, not necessarily 100% coverage
- Keep test data small (ideally <1MB per file)
- Test conditional paths and filters
- Include at least 2-3 tests per tool
- Use meaningful test data files
Help Section
Write help in reStructuredText format:
<help><![CDATA[
What it does
============
**Tool Name** performs XYZ analysis on sequencing data. It processes input files and produces summary statistics.
This tool is useful for:
- Use case 1
- Use case 2
- Use case 3
.. image:: pipeline.svg
:alt: Pipeline diagram
:align: left
Input
=====
The tool accepts:
- **FASTQ files**: Raw sequencing reads (gzipped or uncompressed)
- **BAM files**: Aligned reads
Parameters
==========
**Basic options**
- **Parameter 1**: Description of what this parameter does
- **Parameter 2**: Explanation with example values
**Advanced options**
- **Advanced parameter**: Detailed explanation
Output
======
The tool generates:
1. **Summary table**: Tab-separated file with columns:
- Column 1: Description
- Column 2: Description
- Column 3: Description
2. **Log file** (optional): Detailed processing log
Examples
========
Example 1: Basic usage
----------------------
Input: reads.fastq
Parameter 1: 100
Output: summary statistics
Example 2: Advanced filtering
------------------------------
Input: reads.fastq.gz
Filter: length > 50
Output: Filtered statistics
References
==========
For more information, see the `tool homepage <https://example.com>`_.
]]></help>Best practices:
- Use reStructuredText formatting
- Include "What it does", "Input", "Output" sections
- Add examples when helpful
- Reference images from
./static/images/directory - Be clear and concise
- Include links to tool homepage/documentation
Citations
Always include relevant citations:
<citations>
<citation type="doi">10.1038/s41592-021-01101-x</citation>
<citation type="bibtex">
@article{author2020,
title={Tool Title},
author={Author, A. and Author, B.},
journal={Journal Name},
year={2020},
volume={10},
pages={123-145}
}
</citation>
</citations>Best practices:
- Prefer DOI format over BibTeX
- Include primary tool citation
- Include algorithm citations if relevant
Formatting and Style
Indentation
- Use 4 spaces for indentation
- Keep XML properly indented and readable
CDATA Sections
Use CDATA for sections containing special characters:
<command><![CDATA[ ... ]]></command>
<help><![CDATA[ ... ]]></help>
<version_command><![CDATA[ ... ]]></version_command>Special Characters in Parameters
When using comparison operators in select parameters:
<param name="comparison" type="select">
<option value="<">less than</option>
<option value="=">equal to</option>
<option value=">">greater than</option>
<sanitizer sanitize="false"/>
</param>Use XML entities: < for <, > for >, & for &
Testing Tools
Local Testing with Planemo
# Install planemo
pip install planemo
# Lint the tool
planemo lint tool.xml
# Test the tool
planemo test --install_galaxy tool.xml
# Serve the tool locally
planemo serve tool.xmlLinting Requirements
All tools must pass:
planemo lintwith no errors or warningsplanemo testwith all tests passing- Python code must pass
flake8linting - R code must pass
lintrlinting
Common Patterns and Tips
Handling Collections
Process dataset collections by accepting collection inputs:
<param name="input_collection" type="data_collection"
collection_type="list" format="fastq"
label="Input collection of FASTQ files"/>Reference Data and Tool Data Tables
For built-in reference data:
1. Create tool-data/tool_data_table_conf.xml.sample:
<tables>
<table name="tool_name_indexes" comment_char="#">
<columns>value, dbkey, name, path</columns>
<file path="tool-data/tool_name_indexes.loc"/>
</table>
</tables>2. Create tool-data/tool_name_indexes.loc.sample:
# value dbkey name path
hg38 hg38 Human (hg38) /path/to/hg38/index3. Use in tool XML:
<param name="index" type="select" label="Reference genome">
<options from_data_table="tool_name_indexes">
<filter type="sort_by" column="2"/>
<validator type="no_options" message="No indexes available"/>
</options>
</param>4. Create test data table tool-data/tool_data_table_conf.xml.test
Using Environment Variables
<environment_variables>
<environment_variable name="TOOL_VAR">value</environment_variable>
</environment_variables>Or use in command:
--threads "\${GALAXY_SLOTS:-4}"Configuration Files
For tools requiring configuration files:
<configfiles>
<configfile name="config_file"><![CDATA[
[section]
parameter1 = $param1
parameter2 = $param2
#if $conditional.select == "advanced":
advanced_option = $conditional.advanced_param
#end if
]]></configfile>
</configfiles>Reference in command:
tool_name --config '$config_file'Checklist for Tool Submission
Before submitting a tool to tools-iuc:
Repository Level
- [ ]
.shed.ymlpresent and correctly formatted - [ ] Tool name alphanumeric with underscores only (no hyphens)
- [ ] Owner set to
iuc(or migration arranged) - [ ] Homepage and repository URLs present
- [ ] No
tool_dependencies.xmlfile (deprecated)
Tool XML
- [ ] Passes
planemo lintwith no warnings/errors - [ ] XML elements in correct order
- [ ] Uses
@TOOL_VERSION@+galaxy@VERSION_SUFFIX@versioning - [ ]
profileattribute set appropriately - [ ] Description concise and clear
- [ ] Macros used for tool suites
- [ ] EDAM topics/operations included (when applicable)
Requirements
- [ ] Conda packages available in bioconda/conda-forge
- [ ] Versions specified correctly with
@TOOL_VERSION@ - [ ] No deprecated dependency mechanisms
Command
- [ ] Wrapped in
<![CDATA[ ... ]]> - [ ]
detect_errorsattribute present OR<stdio>element present OR profile >= 16.04 - [ ] Parameters properly quoted with single quotes
- [ ] Optional parameters checked before use
- [ ] Commands joined with
&& - [ ] Proper error handling
Inputs
- [ ] Data parameters have correct
formatattributes - [ ] Parameter attributes in recommended order
- [ ]
argumentattributes use long-form CLI flags - [ ] Boolean parameters use appropriate
truevalue/falsevalue - [ ] Conditionals use
selectnotboolean - [ ] Advanced options grouped in sections
Outputs
- [ ] Proper format specifications
- [ ] Filters for conditional outputs
- [ ] Meaningful label using
${tool.name}and${on_string}
Tests
- [ ] At least 2-3 comprehensive tests
- [ ] Tests cover main functionality
- [ ] Test data in
test-data/directory - [ ] Test files small (<1MB preferred)
- [ ]
expect_num_outputsspecified - [ ] Tests pass with
planemo test - [ ] Output filtering tested with filters
Help
- [ ] Wrapped in
<![CDATA[ ... ]]> - [ ] Written in valid reStructuredText
- [ ] Includes "What it does" section
- [ ] Describes inputs and outputs
- [ ] Images in
./static/images/if used
Citations
- [ ] Primary tool citation included
- [ ] DOI format preferred over BibTeX
Best Practices
- [ ] 4-space indentation throughout
- [ ] Code is readable and well-organized
- [ ] Follows IUC Best Practices
- [ ] Tool appropriate for IUC repository
- [ ] OSI-approved license
Advanced Topics
Parallelism
Enable parallel processing:
<parallelism method="multi" split_inputs="input_file" split_mode="to_size"
split_size="100" merge_outputs="output_file"/>Dynamic Option Sources
Create options from dataset columns:
<param name="column_select" type="select" label="Select column">
<options>
<filter type="data_meta" ref="input_file" key="columns"/>
</options>
</param>Discovering Datasets
For tools producing multiple unknown output files:
<outputs>
<collection name="output_collection" type="list" label="Output files">
<discover_datasets pattern="__name__" directory="outputs" format="txt"/>
</collection>
</outputs>Resources
- IUC Best Practices: https://galaxy-iuc-standards.readthedocs.io/
- Galaxy Tool Development: https://docs.galaxyproject.org/en/latest/dev/schema.html
- Planemo Documentation: https://planemo.readthedocs.io/
- Galaxy Training: https://training.galaxyproject.org/
- Tool Shed: https://toolshed.g2.bx.psu.edu/
- IUC GitHub: https://github.com/galaxyproject/tools-iuc
- Galaxy Gitter: https://gitter.im/galaxy-iuc/iuc
Common Issues and Solutions
Issue: Tests fail with "File not found"
Solution: Ensure test data files are in test-data/ directory and referenced correctly
Issue: Optional parameter showing empty value
Solution: Check with #if str($param) != "": before using
Issue: Boolean parameter not working
Solution: Ensure truevalue is the CLI flag and falsevalue is empty or opposite
Issue: Conditional not showing correct parameters
Solution: Ensure <when> values match <option> values exactly
Issue: Output not appearing
Solution: Check filter conditions and ensure output is produced by command
Issue: Tool not finding executable
Solution: Verify conda package name and version in requirements
Issue: Test comparing binary files fails
Solution: Use <has_size> assertion instead of file comparison
Issue: Collection not processing multiple files
Solution: Ensure tool properly handles dataset collections in inputs
Workflow for Creating a New Tool
1. Research: Find the tool's conda package, documentation, and examples 2. Initialize: Create directory structure with .shed.yml and tool.xml 3. Basic wrapper: Write minimal XML with basic inputs/outputs 4. Test locally: Use planemo test to verify basic functionality 5. Add features: Incrementally add parameters, conditionals, and options 6. Create tests: Write comprehensive tests with small test data 7. Documentation: Write clear help section with examples 8. Lint: Run planemo lint and fix all issues 9. Review: Check against IUC checklist 10. Submit: Open pull request to tools-iuc repository
Tips for Success
- Start simple and iterate
- Use existing similar tools as templates
- Test frequently during development
- Keep test data small
- Write clear, helpful documentation
- Follow the style guide consistently
- Ask for help on Gitter when stuck
- Review other tools for patterns
- Use macros for tool suites to reduce duplication
- Think about user experience when designing parameters
Galaxy Tool Testing Guide
Detailed guidance on testing Galaxy tool wrappers, including regenerating expected outputs, handling large test files, and assertion-based testing strategies.
Regenerating Expected Test Outputs
When test files don't match but the tool runs correctly:
# Run the tool manually with test inputs
mkdir -p output_dir
/path/to/conda/env/bin/tool_command \
-i test-data/input.fa \
-o output_dir
# Copy to expected output
cp output_dir/output.fa test-data/expected_output.fa
# Clean up
rm -rf output_dirVerifying before regenerating:
- Check that tool exit code is 0 (successful)
- Inspect the actual output to ensure it's correct
- Compare line counts:
wc -l expected.fa actual.fa - Review diffs to understand what changed
Common reasons to regenerate:
- Test was created before tool updates
- Expected file only has subset of sequences (bug in test creation)
- Format changes in newer tool versions
Handling Large Test Files
Problem
GitHub CI has a 1MB file size limit. Large test output files (e.g., pretext maps, large genomic files) will cause CI failures even if they're valid test data.
Solution: Use Alternative Assertions
Instead of comparing full output files, use assertions to verify correctness:
Option 1: Size Assertion (Recommended for binary/large files)
<test>
<param name="input" value="input.bam"/>
<output name="output">
<assert_contents>
<has_size value="2225023" delta="1000"/>
</assert_contents>
</output>
</test>When to use:
- Binary output files (
.pretext,.bam,.bcf, etc.) - Large text files where full comparison isn't practical
- Files with consistent size for given inputs
Best practices:
- Calculate size from actual output:
ls -l test-data/output.file | awk '{print $5}' - Use reasonable delta (e.g., 1000 bytes) to account for minor version differences
- Can combine with checksum for stricter validation
Option 2: Checksum Assertion
<test>
<param name="input" value="input.bam"/>
<output name="output">
<assert_contents>
<has_size value="2225023" delta="1000"/>
<has_text text="specific_header_text"/>
</assert_contents>
</output>
</test>Note: Galaxy doesn't have built-in checksum assertions, but you can:
- Use
has_sizefor exact size matching (delta=0) - Combine with
has_textto check for key content markers - Use
has_lineto verify specific output lines exist
Option 3: Content Sampling (For text files)
<test>
<param name="input" value="input.sam"/>
<output name="output">
<assert_contents>
<has_line line="@HD VN:1.0 SO:coordinate"/>
<has_n_columns n="11"/>
<has_n_lines n="1000" delta="100"/>
</assert_contents>
</output>
</test>Workflow: Replacing Large Test Files
1. Calculate file size:
ls -l tools/tool-name/test-data/large_output.file | awk '{print $5}'2. Update test XML: Replace file="large_output.file" with <assert_contents> block
3. Remove large file from git:
git rm tools/tool-name/test-data/large_output.file4. If file was already committed, rebase to remove from history:
# Squash the commit that added the file with the fix commit
git reset --soft HEAD~2
git commit -m "new version with test optimization"
# Force push (only safe if branch hasn't been pulled by others)
git push --force-with-lease origin branch-nameTrade-offs
Size assertions:
- No large files in repo
- Fast CI tests
- Works for binary files
- Doesn't catch content corruption
- May be too lenient for critical outputs
Full file comparison:
- Detects any output changes
- Most thorough validation
- Requires storing large files
- Fails CI if over 1MB
Recommendation: Use size assertions for binary/large files, keep full file comparison for small text outputs where exact correctness matters.
Galaxy Tool Troubleshooting Guide
Practical troubleshooting tips learned from real Galaxy tool development.
Diagnosing Test Failures
Reading tool_test_output.json
When tests fail, examine tool_test_output.json for:
- exit_code: Non-zero indicates failure
- stderr/tool_stderr: Error messages from the tool
- command_line: The actual command that was executed
- output_problems: Summary of what went wrong
Common exit codes:
1: General error (often command syntax or missing dependencies)133: SIGTRAP - Usually binary compatibility issue or dependency conflict137: SIGKILL - Out of memory139: SIGSEGV - Segmentation fault
Example: Analyzing Failures
{
"exit_code": 133,
"stderr": "Trace/breakpoint trap",
"tool_id": "rdeval"
}This indicates a binary crash, often due to: 1. Dependency conflicts 2. Platform incompatibility 3. Corrupted package
Common Issues and Solutions
Issue 1: R Command Quoting Errors
Symptom: ERROR: option '-e' requires a non-empty argument
Problem: Shell quoting conflicts when passing complex R code via R -e "...":
<!-- WRONG: Quotes conflict -->
R -e "rmarkdown::render(..., params=list(input_files=c('file1.rd', 'file2.rd')))"Solution: Use a <configfile> instead:
<command><![CDATA[
Rscript '$r_script'
]]></command>
<configfiles>
<configfile name="r_script"><![CDATA[
rmarkdown::render(
'input.Rmd',
output_file='$output',
params=list(
input_files=c('file1.rd', 'file2.rd')
)
)
]]></configfile>
</configfiles>Benefits:
- No shell quoting issues
- Better readability
- Easier debugging
Issue 2: Dependency Conflicts Between Tools
Symptom: Tool works with version X but crashes with version Y after adding new dependencies
Problem: Shared requirements macro includes dependencies that conflict with some tools:
<!-- WRONG: All tools share same requirements -->
<macros>
<xml name="requirements">
<requirements>
<requirement type="package" version="1.0">tool_binary</requirement>
<requirement type="package" version="2.0">r-somepackage</requirement>
</requirements>
</xml>
</macros>When used by:
tool.xml- Binary tool (doesn't need R packages, but they cause conflicts)tool_report.xml- R reporting tool (needs R packages)
Solution: Split requirements into tool-specific macros:
<macros>
<!-- For binary tools -->
<xml name="requirements">
<requirements>
<requirement type="package" version="1.0">tool_binary</requirement>
</requirements>
</xml>
<!-- For R reporting tools -->
<xml name="requirements_report">
<requirements>
<requirement type="package" version="1.0">tool_binary</requirement>
<requirement type="package" version="2.0">r-somepackage</requirement>
<requirement type="package" version="3.0">r-rmarkdown</requirement>
</requirements>
</xml>
</macros>Then use appropriate macro in each tool:
<!-- tool.xml -->
<expand macro="requirements"/>
<!-- tool_report.xml -->
<expand macro="requirements_report"/>Issue 3: Missing R Package Dependencies
Symptom: R command fails with "there is no package called 'X'"
Problem: Tool uses R packages not listed in requirements
Solution: Check what R packages are actually used and add them:
Common R packages needed:
r-rmarkdown- Forrmarkdown::render()r-ggplot2- For plottingr-cowplot- For plot arrangementsr-dplyr- For data manipulationr-tidyr- For data tidying
Example:
<requirements>
<requirement type="package" version="2.16">r-rmarkdown</requirement>
<requirement type="package" version="1.2.0">r-cowplot</requirement>
</requirements>Issue 4: Platform-Specific Test Failures
Symptom: Tests fail on macOS with Rosetta errors but work on Linux
Error: rosetta error: failed to open elf at /lib64/ld-linux-x86-64.so.2
Root Cause: Testing Linux containers on Apple Silicon Mac
Solutions: 1. Test on Linux - Use CI/CD or Linux VM 2. Check for dependency conflicts first - If version N-1 works but version N doesn't, it's likely not a platform issue 3. Use native Mac tools - If available 4. Docker Desktop settings - Ensure proper VM configuration
Key insight: If an older version works fine but newer version fails with same platform, it's NOT a platform issue—it's a dependency/packaging issue!
Debugging Workflow
Step 1: Identify the Failure Pattern
Run tests and check:
planemo test tool.xmlLook at the JSON output:
- How many tests fail?
- Do they all fail the same way?
- What are the exit codes?
Step 2: Check Recent Changes
Ask yourself:
- What changed between working and non-working version?
- Were new dependencies added?
- Did the tool version change?
Step 3: Isolate the Problem
For dependency conflicts:
# Check what changed
git diff HEAD~1 macros.xml
# Test with minimal dependencies
# Temporarily comment out new dependencies in macros.xmlStep 4: Review Command Execution
From tool_test_output.json, look at command_line:
- Is the command properly formatted?
- Are quotes balanced?
- Are variables expanded correctly?
Step 5: Check Container/Environment
# Test if the tool package itself works
planemo conda_install tool.xml
planemo conda_env tool.xml
# Then test manually
source activate <env_name>
tool_binary --versionBest Practices for Robust Tools
1. Separate Requirements by Tool Type
<macros>
<!-- Minimal requirements for binary tools -->
<xml name="requirements_base">
<requirements>
<requirement type="package" version="@TOOL_VERSION@">tool</requirement>
</requirements>
</xml>
<!-- Extended requirements for analysis tools -->
<xml name="requirements_analysis">
<requirements>
<requirement type="package" version="@TOOL_VERSION@">tool</requirement>
<requirement type="package" version="1.0">python-lib</requirement>
</requirements>
</xml>
<!-- Requirements for reporting tools -->
<xml name="requirements_report">
<requirements>
<requirement type="package" version="@TOOL_VERSION@">tool</requirement>
<requirement type="package" version="2.0">r-rmarkdown</requirement>
</requirements>
</xml>
</macros>2. Use Configfiles for Complex Scripts
Instead of inline shell/R/Python in <command>, use <configfiles>:
<command><![CDATA[
python '$python_script' > '$output'
]]></command>
<configfiles>
<configfile name="python_script"><![CDATA[
import sys
# Your Python code here
param1 = '$param1'
param2 = $param2
# Process...
print("Results")
]]></configfile>
</configfiles>3. Test Incrementally
When adding features: 1. Add one parameter 2. Run tests 3. Add next parameter 4. Run tests again
Don't add everything at once!
4. Version Pin Critical Dependencies
<!-- GOOD: Specific version -->
<requirement type="package" version="2.16">r-rmarkdown</requirement>
<!-- RISKY: May break when updated -->
<requirement type="package">r-rmarkdown</requirement>5. Document Dependency Reasons
<xml name="requirements_report">
<requirements>
<requirement type="package" version="@TOOL_VERSION@">rdeval</requirement>
<!-- For rendering HTML reports -->
<requirement type="package" version="2.16">r-rmarkdown</requirement>
<!-- For plot arrangements in figures -->
<requirement type="package" version="1.2.0">r-cowplot</requirement>
</requirements>
</xml>Real-World Example: rdeval Tool Suite
Problem
- Tool worked in version 0.0.7
- Added r-cowplot and r-rmarkdown for reporting features
- Version 0.0.8 started crashing with exit code 133
- Binary tool (rdeval.xml) was getting R packages it didn't need
Solution
1. Split requirements:
requirements- Just rdeval binary (for rdeval.xml)requirements_report- rdeval + R packages (for rdeval_report.xml)
2. Fixed R command quoting:
- Changed from
R -e "..."toRscriptwith configfile
3. Added missing r-rmarkdown dependency
Result
- Binary tool no longer has conflicting R dependencies
- Report tool has all needed R packages
- Both tools work correctly
Problem: Report shows wrong statistics (v0.0.8)
- HTML report from
rdeval_reportshowed wrong read lengths, N50, plots - Tabular output from
rdeval --tabularwas correct - Issue was in upstream
rdeval_interface.R, not the Galaxy wrapper
Debugging approach
1. Compare outputs: Galaxy tabular vs HTML report vs CLI report on same data 2. Trace the pipeline: rdeval → .rd file → R interface → HTML report 3. Verify .rd files: Run rdeval --input-reads file.rd --tabular to confirm .rd data is correct 4. Test R interface locally: Run rdeval_interface.R on the .rd file to isolate the R reader 5. Binary format analysis: Compute expected data sizes to identify format mismatch
5*len8 + 6*len16 + 12*len64vs8*len8 + 8*len16 + 16*len64
6. Check release artifacts: Compare release zip sha256 with conda recipe to detect silent updates
Lesson
When upstream tools bundle companion scripts (R, Python), format changes in the binary can break the script readers. Always verify both sides match.
Naming in report tools
When a report tool processes collection elements, use element_identifier not numeric indices:
#set $safe_name = re.sub(r"[^\w\-.]", "_", str($input_file.element_identifier))
ln -s '$input_file' '${safe_name}.rd' &&Testing Tips
Local Testing
# Lint first
planemo lint tool.xml
# Test with fresh environment
planemo test --conda_auto_install tool.xml
# Test specific test case
planemo test --test_index 0 tool.xmlDebug Mode
# Keep test data after failure
planemo test --no_cleanup tool.xml
# See detailed output
planemo test --verbose tool.xmlContainer Testing
# Build container
planemo container_register tool.xml
# Test in container
planemo test --biocontainers tool.xmlResources
- Galaxy Tool Best Practices: https://galaxy-iuc-standards.readthedocs.io/
- Planemo Documentation: https://planemo.readthedocs.io/
- Conda Package Search: https://anaconda.org/bioconda/
- Galaxy Training: https://training.galaxyproject.org/
Quick Reference
| Symptom | Likely Cause | Solution |
|---|---|---|
| Exit code 133 | Binary conflict/dependency issue | Check recent dependency changes |
| "no package called X" | Missing R dependency | Add to requirements |
| Quote/argument errors | Shell quoting issues | Use configfile |
| Works in v1, fails in v2 | Dependency conflict | Review requirement changes |
| Rosetta errors + new deps | Dependency conflict, not platform | Remove/isolate new dependencies |
| All tests fail same way | Systemic issue (deps/command) | Check requirements first |
| Some tests fail | Test-specific issue | Check test parameters |
Common XML and Runtime Issues
Issue: "Command not found"
- Check
<requirements>section has correct package - Verify conda package name and version
- Test command availability:
planemo conda_install tool.xml
Issue: "Output file not found" or 0-byte output with exit_code=0
- Verify command actually creates the file
- Check output file path matches
<data name="output" from_work_dir="..."> - Use
discover_datasetsfor dynamic outputs - `from_work_dir` vs `$output` conflict: If command writes to
$outputbut output
definition has from_work_dir="filename", Galaxy looks for filename in the working directory instead. This causes 0-byte outputs especially on remote/Pulsar job runners. Fix: either remove from_work_dir (if command uses $output) or change the command to write to the from_work_dir filename instead of $output.
Issue: "Test failed"
- Compare expected vs actual output
- Check for whitespace/newline differences
- Use
sim_sizefor approximate size matching - Add
lines_difffor line-by-line comparison
Issue: Tool has multiple output flags (`-o` vs `-p`) with different behavior
-o/--out-filetypically writes to an explicit filename with extension-p/--out-prefixtypically constructs filename asprefix + input_filename- These flags may produce different output types (e.g.,
-ogenerates rd/binary files,
-p generates same-format-as-input files)
- Check the tool's source code to understand naming conventions before wrapping
- Use separate output type conditionals for each mode rather than trying to unify them
Issue: "Invalid XML"
- Run
planemo lint tool.xml - Check closing tags match opening tags
- Validate CDATA sections for command blocks
- Ensure proper escaping of special characters
Debugging Tool Test Failures
General Workflow
1. Read the test output JSON first
cat tool_test_output.jsonLook for:
- Exit codes and error messages in
stderr/stdout output_problemsarray for test assertion failures- Actual vs expected output differences
2. Never copy/modify conda package scripts
- Tool wrappers should ALWAYS use conda packages
- If there are bugs in the conda package scripts, work around them in the XML wrapper
- Common workaround: Add trailing slashes to paths if script concatenates without separators
3. Wrong test expectations vs bugs
- If tests fail but the tool runs successfully (exit code 0), check if expected test files are wrong
- Regenerate expected outputs by running the tool manually with test inputs
- Update
expect_num_outputsif optional outputs are created
Common Test Failure Fixes
Path concatenation bugs in Python scripts:
<!-- If script does: args.output_dir + 'file.txt' without '/' -->
<!-- Fix in wrapper with trailing slash: -->
-o 'output_dir/' <!-- instead of -o output_dir -->Wrong number of expected outputs:
<!-- Check if optional outputs are always created -->
<test expect_num_outputs="3"> <!-- Update count -->Output has extra sequences/data:
- First check if this is expected behavior
- Regenerate expected test files from actual tool output
- Don't add post-processing filters unless absolutely necessary
`has_size` attribute restrictions (XSD validation):
has_sizedoes NOT supportcompare="ge"or similar -- planemo lint will reject it- Use
valueanddeltaattributes:<has_size value="148" delta="50"/> - This asserts the size is within
value +/- deltabytes - For minimum size checks, use a value with a large enough delta
Galaxy decompresses tar.gz/tgz files -- tool receives plain tar: When a param accepts format="tar.gz", format="tgz", or format="tar,gz,tgz", Galaxy may strip the gzip layer before passing the file to the tool. The tool receives a plain tar archive, not gzip-compressed. Symptom: gzip: invalid magic or tar: invalid magic when trying tar -xzf or gzip -dc.
<!-- BAD: assumes file is still gzip-compressed -->
tar -xzf '${input_archive}' -C ./output_dir
gzip -dc '${input_archive}' | tar xf - -C ./output_dir
<!-- GOOD: use plain tar extraction (Galaxy already decompressed) -->
tar xf '${input_archive}' -C ./output_dir
<!-- For creating tar.gz output (gzip is needed here): -->
tar cf - -C ./input_dir . | gzip > output.tar.gzCascade failures from empty/broken upstream outputs: When debugging multiple errors in a test history, check datasets in order. A 0-byte or errored output fed as input to a subsequent tool will produce misleading errors (e.g., "error reading header"). Always fix the root cause (first failing dataset) before investigating downstream errors.
tar flag order bug (`-xfz` vs `-xzf`): tar -xfz file.tar.gz means "extract, file=z, then file.tar.gz is a positional arg". The -f flag consumes the next character as the filename. Error: tar: can't open 'z'. Fix: always put -f last (tar -xzf) or use positional syntax (tar xf file).
Test conditional nesting must match input structure: If a conditional is expanded via macro inside mode_conditional, test params must nest it:
<!-- BAD: blobtk_plot_options outside mode_conditional -->
<conditional name="mode_conditional">
<param name="selector" value="filter"/>
</conditional>
<conditional name="blobtk_plot_options">
<param name="blobtk_plot" value="no"/>
</conditional>
<!-- GOOD: nested inside mode_conditional -->
<conditional name="mode_conditional">
<param name="selector" value="filter"/>
<conditional name="blobtk_plot_options">
<param name="blobtk_plot" value="no"/>
</conditional>
</conditional>Planemo lint reports: WARNING (TestsCaseValidation): Invalid parameter name found.
Lessons Learned
1. Version changes that break existing functionality are usually dependency-related, not platform-related 2. Shared requirements across tool suites can cause conflicts - split them when tools have different needs 3. R commands with complex parameters need configfiles to avoid quoting issues 4. Always test after adding new dependencies - they may conflict with existing tools 5. Exit code 133 often indicates dependency conflicts, not code bugs 6. When tabular output is correct but report output is wrong, the issue is likely in the report's data reader (R/Python script), not the tool itself 7. Compare release zip sha256 with conda recipe to check if release artifacts were silently updated: shasum -a 256 release.zip vs meta.yaml sha256 8. C struct padding can silently break companion scripts — when C++ code changes from sizeof(pair<...>) to compact writes, all readers must be updated