Nextflow Development

Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/

TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION

1. Definitions.

"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.

"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.

"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.

"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.

"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.

"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.

"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).

"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.

"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."

"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.

2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.

3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.

4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:

(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and

(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and

(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and

(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.

You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.

5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.

6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.

7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.

8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.

9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.

END OF TERMS AND CONDITIONS

APPENDIX: How to apply the Apache License to your work.

To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.

Copyright [yyyy] [name of copyright owner]

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

#!/usr/bin/env python3
"""
Pre-flight environment validation for nf-core pipelines.

Checks Docker, Nextflow, Java, system resources, and network connectivity.
Run this BEFORE attempting any pipeline execution.

Usage:
    python check_environment.py
    python check_environment.py --json
"""

import json
import os
import shutil
import subprocess
import sys
from dataclasses import dataclass, field, asdict
from typing import List, Optional


@dataclass
class CheckResult:
    """Result of a single environment check."""
    name: str
    passed: bool
    message: str
    details: Optional[str] = None
    fix: Optional[str] = None


@dataclass
class EnvironmentReport:
    """Complete environment validation report."""
    ready: bool
    checks: List[CheckResult] = field(default_factory=list)
    recommendations: List[str] = field(default_factory=list)

    def to_dict(self):
        return {
            "ready": self.ready,
            "checks": [asdict(c) for c in self.checks],
            "recommendations": self.recommendations
        }


def check_docker() -> CheckResult:
    """Check Docker availability, daemon status, and permissions."""
    if not shutil.which("docker"):
        return CheckResult(
            name="Docker",
            passed=False,
            message="Docker not found in PATH",
            fix="Install Docker: https://docs.docker.com/get-docker/"
        )

    try:
        result = subprocess.run(
            ["docker", "info"],
            capture_output=True,
            text=True,
            timeout=15
        )

        if result.returncode != 0:
            stderr_lower = result.stderr.lower()
            if "permission denied" in stderr_lower:
                return CheckResult(
                    name="Docker",
                    passed=False,
                    message="Docker permission denied",
                    details="Cannot connect to Docker daemon",
                    fix="sudo usermod -aG docker $USER && newgrp docker"
                )
            elif "cannot connect" in stderr_lower or "is the docker daemon running" in stderr_lower:
                return CheckResult(
                    name="Docker",
                    passed=False,
                    message="Docker daemon not running",
                    details=result.stderr[:200] if result.stderr else None,
                    fix="sudo systemctl start docker"
                )
            else:
                return CheckResult(
                    name="Docker",
                    passed=False,
                    message="Docker error",
                    details=result.stderr[:200] if result.stderr else None,
                    fix="Check Docker installation and daemon status"
                )

        return CheckResult(
            name="Docker",
            passed=True,
            message="Docker is available and running"
        )

    except subprocess.TimeoutExpired:
        return CheckResult(
            name="Docker",
            passed=False,
            message="Docker command timed out",
            fix="Check Docker daemon status: sudo systemctl status docker"
        )
    except Exception as e:
        return CheckResult(
            name="Docker",
            passed=False,
            message=f"Docker check failed: {str(e)}"
        )


def check_nextflow() -> CheckResult:
    """Check Nextflow installation and version (requires >= 23.04)."""
    if not shutil.which("nextflow"):
        return CheckResult(
            name="Nextflow",
            passed=False,
            message="Nextflow not found in PATH",
            fix="curl -s https://get.nextflow.io | bash && mv nextflow ~/bin/ && export PATH=$HOME/bin:$PATH"
        )

    try:
        result = subprocess.run(
            ["nextflow", "-version"],
            capture_output=True,
            text=True,
            timeout=30
        )

        output = result.stdout + result.stderr
        version_line = output.strip().split('\n')[0] if output else ""

        import re
        match = re.search(r'(\d+)\.(\d+)\.(\d+)', version_line)

        if match:
            major, minor, patch = int(match.group(1)), int(match.group(2)), int(match.group(3))
            version_str = f"{major}.{minor}.{patch}"

            # Require version >= 23.04
            if major > 23 or (major == 23 and minor >= 4):
                return CheckResult(
                    name="Nextflow",
                    passed=True,
                    message=f"Nextflow {version_str} installed",
                    details=version_line
                )
            else:
                return CheckResult(
                    name="Nextflow",
                    passed=False,
                    message=f"Nextflow {version_str} is outdated (requires >= 23.04)",
                    details=version_line,
                    fix="nextflow self-update"
                )

        return CheckResult(
            name="Nextflow",
            passed=True,
            message="Nextflow installed (version unknown)",
            details=version_line
        )

    except subprocess.TimeoutExpired:
        return CheckResult(
            name="Nextflow",
            passed=False,
            message="Nextflow command timed out",
            fix="Check Nextflow installation"
        )
    except Exception as e:
        return CheckResult(
            name="Nextflow",
            passed=False,
            message=f"Nextflow check failed: {str(e)}"
        )


def check_java() -> CheckResult:
    """Check Java version (requires >= 11)."""
    if not shutil.which("java"):
        return CheckResult(
            name="Java",
            passed=False,
            message="Java not found in PATH",
            fix="Install Java 11+: sudo apt install openjdk-11-jdk"
        )

    try:
        result = subprocess.run(
            ["java", "-version"],
            capture_output=True,
            text=True,
            timeout=10
        )

        # Java version is typically in stderr
        output = result.stderr or result.stdout
        import re
        match = re.search(r'version "(\d+)', output)

        if match:
            version = int(match.group(1))
            version_line = output.strip().split('\n')[0]

            if version >= 11:
                return CheckResult(
                    name="Java",
                    passed=True,
                    message=f"Java {version} installed",
                    details=version_line
                )
            else:
                return CheckResult(
                    name="Java",
                    passed=False,
                    message=f"Java {version} is too old (requires >= 11)",
                    details=version_line,
                    fix="Install Java 11+: sudo apt install openjdk-11-jdk"
                )

        return CheckResult(
            name="Java",
            passed=True,
            message="Java installed",
            details=output.strip().split('\n')[0] if output else None
        )

    except Exception as e:
        return CheckResult(
            name="Java",
            passed=False,
            message=f"Java check failed: {str(e)}"
        )


def check_resources() -> CheckResult:
    """Check system resources (CPU, memory, disk)."""
    try:
        # CPU cores
        cpu_count = os.cpu_count() or 1

        # Memory
        mem_gb = 0
        try:
            # Linux: read from /proc/meminfo
            with open('/proc/meminfo', 'r') as f:
                for line in f:
                    if line.startswith('MemTotal:'):
                        mem_kb = int(line.split()[1])
                        mem_gb = mem_kb / (1024 * 1024)
                        break
        except (FileNotFoundError, PermissionError):
            # macOS: use sysctl
            try:
                result = subprocess.run(
                    ['sysctl', '-n', 'hw.memsize'],
                    capture_output=True, text=True, timeout=5
                )
                if result.returncode == 0:
                    mem_gb = int(result.stdout.strip()) / (1024**3)
            except Exception:
                pass

        # Disk space (current directory)
        disk_gb = 0
        try:
            statvfs = os.statvfs('.')
            disk_gb = (statvfs.f_frsize * statvfs.f_bavail) / (1024**3)
        except Exception:
            pass

        details = f"CPUs: {cpu_count}, Memory: {mem_gb:.1f}GB, Disk: {disk_gb:.1f}GB available"

        # Check minimums
        warnings = []
        if cpu_count < 4:
            warnings.append(f"Low CPU count ({cpu_count}). Consider --max_cpus {cpu_count}")
        if 0 < mem_gb < 8:
            warnings.append(f"Low memory ({mem_gb:.1f}GB). Use --max_memory '{int(mem_gb)}GB'")
        if 0 < disk_gb < 50:
            warnings.append(f"Low disk space ({disk_gb:.1f}GB). Pipelines need ~100GB for human data")

        if warnings:
            return CheckResult(
                name="Resources",
                passed=True,
                message="Resources available (with warnings)",
                details=details,
                fix="; ".join(warnings)
            )

        return CheckResult(
            name="Resources",
            passed=True,
            message="Sufficient resources available",
            details=details
        )

    except Exception as e:
        return CheckResult(
            name="Resources",
            passed=True,  # Don't fail on resource check errors
            message=f"Could not fully check resources: {str(e)}"
        )


def check_network() -> CheckResult:
    """Check network connectivity to Docker Hub and nf-core."""
    try:
        import urllib.request

        # User-Agent header to avoid 403 from sites that block default Python agent
        headers = {'User-Agent': 'nf-core-helper/1.0'}

        # Try Docker Hub
        try:
            req = urllib.request.Request("https://hub.docker.com", headers=headers)
            urllib.request.urlopen(req, timeout=10)
            docker_hub_ok = True
        except Exception:
            docker_hub_ok = False

        # Try nf-core (for pipeline downloads)
        try:
            req = urllib.request.Request("https://nf-co.re", headers=headers)
            urllib.request.urlopen(req, timeout=10)
            nfcore_ok = True
        except Exception:
            nfcore_ok = False

        if docker_hub_ok and nfcore_ok:
            return CheckResult(
                name="Network",
                passed=True,
                message="Network connectivity OK (Docker Hub & nf-core reachable)"
            )
        elif docker_hub_ok:
            return CheckResult(
                name="Network",
                passed=True,
                message="Docker Hub reachable (nf-core.re not reachable)",
                details="Pipeline downloads may still work via GitHub"
            )
        else:
            return CheckResult(
                name="Network",
                passed=False,
                message="Cannot reach Docker Hub",
                fix="Check network connection. Containers require Docker Hub access."
            )

    except Exception as e:
        return CheckResult(
            name="Network",
            passed=False,
            message=f"Network check failed: {str(e)}",
            fix="Check network connection and proxy settings"
        )


def run_all_checks() -> EnvironmentReport:
    """Run all environment checks and return comprehensive report."""
    checks = [
        check_docker(),
        check_nextflow(),
        check_java(),
        check_resources(),
        check_network(),
    ]

    # Critical checks that must pass
    critical_checks = ["Docker", "Nextflow", "Java"]
    ready = all(c.passed for c in checks if c.name in critical_checks)

    # Build recommendations
    recommendations = []
    for check in checks:
        if not check.passed and check.fix:
            recommendations.append(f"{check.name}: {check.fix}")
        elif check.passed and check.fix:  # Warnings
            recommendations.append(f"{check.name} (warning): {check.fix}")

    return EnvironmentReport(
        ready=ready,
        checks=checks,
        recommendations=recommendations
    )


def print_report(report: EnvironmentReport):
    """Print human-readable report to stdout."""
    print("\n" + "=" * 50)
    print("  nf-core Environment Check")
    print("=" * 50 + "\n")

    for check in report.checks:
        status = "\033[92m[PASS]\033[0m" if check.passed else "\033[91m[FAIL]\033[0m"
        print(f"{status} {check.name}: {check.message}")

        if check.details:
            print(f"       {check.details}")

        if not check.passed and check.fix:
            print(f"       \033[93mFix:\033[0m {check.fix}")
        elif check.passed and check.fix:  # Warning
            print(f"       \033[93mWarning:\033[0m {check.fix}")

    print()
    if report.ready:
        print("\033[92m✓ Environment is READY for nf-core pipelines.\033[0m")
    else:
        print("\033[91m✗ Environment is NOT READY. Please address the issues above.\033[0m")

    if report.recommendations:
        print("\n--- Recommendations ---")
        for i, rec in enumerate(report.recommendations, 1):
            print(f"  {i}. {rec}")

    print()


def main():
    import argparse

    parser = argparse.ArgumentParser(
        description="Check environment for nf-core pipeline execution",
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog="""
Examples:
    python check_environment.py           # Human-readable output
    python check_environment.py --json    # JSON output for parsing
        """
    )
    parser.add_argument("--json", action="store_true",
                        help="Output results as JSON")

    args = parser.parse_args()

    report = run_all_checks()

    if args.json:
        print(json.dumps(report.to_dict(), indent=2))
    else:
        print_report(report)

    sys.exit(0 if report.ready else 1)


if __name__ == "__main__":
    main()

# Organism to Genome Mapping for nf-core Pipelines
# Maps organism names (as they appear in GEO/SRA) to iGenomes keys

organisms:
  # Human
  "Homo sapiens":
    genome: "GRCh38"
    taxid: 9606
    aliases: ["human", "hg38", "GRCh38"]
    notes: "Primary human reference genome"

  "Homo sapiens (legacy)":
    genome: "GRCh37"
    taxid: 9606
    aliases: ["hg19", "GRCh37"]
    notes: "Legacy human reference, still used for some clinical data"

  # Mouse
  "Mus musculus":
    genome: "GRCm39"
    taxid: 10090
    aliases: ["mouse", "mm39", "GRCm39"]
    notes: "Current mouse reference genome"

  "Mus musculus (legacy)":
    genome: "GRCm38"
    taxid: 10090
    aliases: ["mm10", "GRCm38"]
    notes: "Legacy mouse reference"

  # Yeast
  "Saccharomyces cerevisiae":
    genome: "R64-1-1"
    taxid: 4932
    aliases: ["yeast", "sacCer3", "S288C", "budding yeast"]
    notes: "S288C reference strain"

  # Fruit fly
  "Drosophila melanogaster":
    genome: "BDGP6"
    taxid: 7227
    aliases: ["fly", "dm6", "fruit fly", "Dmel"]
    notes: "Berkeley Drosophila Genome Project release 6"

  # Worm
  "Caenorhabditis elegans":
    genome: "WBcel235"
    taxid: 6239
    aliases: ["worm", "ce11", "C. elegans", "Cele"]
    notes: "WormBase reference"

  # Zebrafish
  "Danio rerio":
    genome: "GRCz11"
    taxid: 7955
    aliases: ["zebrafish", "danRer11", "Drer"]
    notes: "Genome Reference Consortium Zebrafish Build 11"

  # Arabidopsis
  "Arabidopsis thaliana":
    genome: "TAIR10"
    taxid: 3702
    aliases: ["arabidopsis", "thale cress", "Atha"]
    notes: "The Arabidopsis Information Resource v10"

  # Rat
  "Rattus norvegicus":
    genome: "Rnor_6.0"
    taxid: 10116
    aliases: ["rat", "rn6", "Rnor"]
    notes: "Rnor 6.0 reference"

  # Chicken
  "Gallus gallus":
    genome: "GRCg6a"
    taxid: 9031
    aliases: ["chicken", "galGal6", "Ggal"]
    notes: "Genome Reference Consortium Chicken Build 6a"

  # Pig
  "Sus scrofa":
    genome: "Sscrofa11.1"
    taxid: 9823
    aliases: ["pig", "susScr11", "Sscr"]
    notes: "Swine genome assembly 11.1"

  # Cow
  "Bos taurus":
    genome: "ARS-UCD1.2"
    taxid: 9913
    aliases: ["cow", "bosTau9", "cattle", "Btau"]
    notes: "USDA ARS assembly"

  # Dog
  "Canis lupus familiaris":
    genome: "CanFam3.1"
    taxid: 9615
    aliases: ["dog", "canFam3", "Clup"]
    notes: "Broad Institute CanFam3.1"

  # Frog
  "Xenopus tropicalis":
    genome: "JGI_4.2"
    taxid: 8364
    aliases: ["frog", "xenTro9", "Xtro"]
    notes: "JGI assembly version 4.2"

  # Maize/Corn
  "Zea mays":
    genome: "Zm-B73-REFERENCE-NAM-5.0"
    taxid: 4577
    aliases: ["maize", "corn", "Zmay"]
    notes: "B73 reference genome v5"

  # Rice
  "Oryza sativa":
    genome: "IRGSP-1.0"
    taxid: 39947
    aliases: ["rice", "Osat"]
    notes: "International Rice Genome Sequencing Project"

  # E. coli (common bacterial model)
  "Escherichia coli":
    genome: null
    taxid: 562
    aliases: ["E. coli", "Ecol"]
    notes: "Use specific strain reference; K-12 MG1655 common"

  # Fission yeast
  "Schizosaccharomyces pombe":
    genome: "ASM294v2"
    taxid: 4896
    aliases: ["fission yeast", "S. pombe", "Spom"]
    notes: "PomBase reference"

# Pipeline mapping based on library strategy
pipeline_suggestions:
  "RNA-SEQ": "rnaseq"
  "ATAC-SEQ": "atacseq"
  "CHIP-SEQ": "chipseq"
  "WGS": "sarek"
  "WXS": "sarek"
  "EXOME": "sarek"
  "AMPLICON": "ampliseq"
  "BISULFITE-SEQ": "methylseq"
  "HI-C": "hic"
  "MIRNA-SEQ": "smrnaseq"
  "RRBS": "methylseq"

name: atacseq
version: "2.1.2"
description: "Chromatin accessibility analysis and peak calling"

# Documentation and source - NOTE: Update version in URLs when upgrading pipeline
urls:
  documentation: "https://nf-co.re/atacseq/{version}/"
  parameters: "https://nf-co.re/atacseq/{version}/parameters/"
  output_docs: "https://nf-co.re/atacseq/{version}/docs/output/"
  github: "https://github.com/nf-core/atacseq"
  releases: "https://github.com/nf-core/atacseq/releases"

data_types:
  - ATAC-seq
  - chromatin accessibility
  - open chromatin

detection_hints:
  filename:
    - atac
    - atacseq
    - chromatin
    - accessibility
  directory:
    - atac
    - atacseq
    - chromatin
    - epigenome
    - epigenetics

samplesheet:
  input_types:
    - fastq

  columns:
    - name: sample
      required: true
      type: string
      inference: filename
      description: "Condition/group identifier (replicates share same name)"

    - name: fastq_1
      required: true
      type: path
      inference: auto
      description: "Absolute path to R1 FASTQ"

    - name: fastq_2
      required: true
      type: path
      inference: auto
      description: "Absolute path to R2 FASTQ (paired-end required)"

    - name: replicate
      required: true
      type: integer
      default: 1
      inference: filename
      description: "Replicate number (integer)"

decision_points:
  - parameter: genome
    prompt: "Which reference genome matches your organism?"
    options:
      - value: GRCh38
        label: "Human GRCh38/hg38 (recommended)"
        description: "Latest human reference"
      - value: GRCh37
        label: "Human GRCh37/hg19 (legacy)"
        description: "Older human reference"
      - value: mm10
        label: "Mouse mm10"
        description: "Mouse reference genome"
    default: GRCh38
    recommendation: "Default to GRCh38 for human samples"

  - parameter: read_length
    prompt: "What is the read length of your sequencing data?"
    options:
      - value: 50
        label: "50 bp"
        description: "Short reads"
      - value: 75
        label: "75 bp"
        description: "Standard length"
      - value: 100
        label: "100 bp"
        description: "Common for modern sequencers"
      - value: 150
        label: "150 bp"
        description: "Long reads"
    default: 50
    recommendation: "Check FASTQ files or sequencing report for exact length"

  - parameter: narrow_peak
    prompt: "What type of peaks are you expecting?"
    options:
      - value: "true"
        label: "Narrow peaks (default for ATAC-seq)"
        description: "Standard ATAC-seq open chromatin regions"
      - value: "false"
        label: "Broad peaks"
        description: "For histone marks or broader accessibility regions"
    default: "true"
    recommendation: "Use narrow peaks for standard ATAC-seq"

test_profile:
  command: "nextflow run nf-core/atacseq -r 2.1.2 -profile test,docker --outdir test_atacseq"
  duration: "15 minutes"
  success_indicators:
    - "test_atacseq/multiqc/multiqc_report.html"
  log_pattern: "Pipeline completed successfully"

run_command:
  template: |
    nextflow run nf-core/atacseq \
        -r 2.1.2 \
        -profile docker \
        --input {samplesheet} \
        --outdir {outdir} \
        --genome {genome} \
        --read_length {read_length} \
        -resume

outputs:
  primary:
    - path: "bwa/mergedLibrary/*.mLb.mkD.sorted.bam"
      description: "Filtered, deduplicated alignments"
    - path: "bwa/mergedLibrary/bigwig/*.bigWig"
      description: "Coverage tracks for genome browsers"
    - path: "macs2/narrowPeak/*.narrowPeak"
      description: "Peak calls (BED format)"
    - path: "macs2/narrowPeak/consensus/consensus_peaks.bed"
      description: "Consensus peaks across replicates"

  validation:
    - file: "multiqc/multiqc_report.html"
      check: exists
      description: "QC report must exist"
    - file: "macs2/narrowPeak"
      check: exists
      description: "Peak calls directory"

quality_metrics:
  - name: mapped_reads
    good: ">80%"
    acceptable: "60-80%"
    poor: "<60%"
  - name: mitochondrial
    good: "<20%"
    acceptable: "20-40%"
    poor: ">40%"
  - name: duplicates
    good: "<30%"
    acceptable: "30-50%"
    poor: ">50%"
  - name: frip
    good: ">30%"
    acceptable: "15-30%"
    poor: "<15%"
  - name: tss_enrichment
    good: ">6"
    acceptable: "4-6"
    poor: "<4"

resources:
  min_memory: "8.GB"
  recommended_memory: "32.GB"
  min_cpus: 4
  recommended_cpus: 8
  disk_space: "100.GB"

troubleshooting:
  - error: "Low FRiP score"
    fix: "Check library complexity in plotFingerprint. May indicate over-transposition or low quality"
  - error: "Few peaks called"
    fix: "Lower threshold with --macs_qvalue 0.1 or use --narrow_peak false for broader peaks"
  - error: "High duplicates"
    fix: "Normal for low-input samples. Pipeline removes by default. Consider deeper sequencing"
  - error: "High mitochondrial reads"
    fix: "Sample quality issue. Pipeline filters mito by default (--keep_mito false)"

replicate_patterns:
  - "_rep(\\d+)"
  - "_R(\\d+)_"
  - "_(\\d+)$"
  - "_replicate(\\d+)"

name: rnaseq
version: "3.22.2"
description: "Gene expression quantification and differential expression analysis"

# Documentation and source - NOTE: Update version in URLs when upgrading pipeline
urls:
  documentation: "https://nf-co.re/rnaseq/{version}/"
  parameters: "https://nf-co.re/rnaseq/{version}/parameters/"
  output_docs: "https://nf-co.re/rnaseq/{version}/docs/output/"
  github: "https://github.com/nf-core/rnaseq"
  releases: "https://github.com/nf-core/rnaseq/releases"

data_types:
  - RNA-seq
  - mRNA-seq
  - bulk RNA-seq

detection_hints:
  filename:
    - rna
    - rnaseq
    - mrna
    - expression
  directory:
    - rnaseq
    - rna
    - expression
    - transcriptome

samplesheet:
  input_types:
    - fastq

  columns:
    - name: sample
      required: true
      type: string
      inference: filename
      description: "Sample identifier"

    - name: fastq_1
      required: true
      type: path
      inference: auto
      description: "Absolute path to R1 FASTQ"

    - name: fastq_2
      required: false
      type: path
      inference: auto
      description: "Absolute path to R2 FASTQ (empty for single-end)"

    - name: strandedness
      required: true
      type: enum
      allowed:
        - auto
        - forward
        - reverse
        - unstranded
      default: "auto"
      inference: default
      description: "Library strandedness (auto recommended)"

decision_points:
  - parameter: genome
    prompt: "Which reference genome matches your organism?"
    options:
      - value: GRCh38
        label: "Human GRCh38/hg38 (recommended for human)"
        description: "Latest human reference assembly"
      - value: GRCh37
        label: "Human GRCh37/hg19 (legacy)"
        description: "Older human reference for compatibility"
      - value: mm10
        label: "Mouse mm10/GRCm38"
        description: "Mouse reference genome"
      - value: BDGP6
        label: "Drosophila BDGP6"
        description: "Fruit fly reference"
    default: GRCh38
    recommendation: "Default to GRCh38 for human samples"

  - parameter: aligner
    prompt: "Which alignment strategy would you prefer?"
    options:
      - value: star_salmon
        label: "STAR + Salmon (recommended)"
        description: "Most accurate, standard for differential expression"
      - value: star_rsem
        label: "STAR + RSEM"
        description: "Better for isoform-level quantification"
      - value: hisat2
        label: "HISAT2"
        description: "Lower memory requirements, faster"
    default: star_salmon
    recommendation: "Use star_salmon unless memory-constrained or need isoforms"

test_profile:
  command: "nextflow run nf-core/rnaseq -r 3.22.2 -profile test,docker --outdir test_rnaseq"
  duration: "15 minutes"
  success_indicators:
    - "test_rnaseq/multiqc/multiqc_report.html"
  log_pattern: "Pipeline completed successfully"

run_command:
  template: |
    nextflow run nf-core/rnaseq \
        -r 3.22.2 \
        -profile docker \
        --input {samplesheet} \
        --outdir {outdir} \
        --genome {genome} \
        --aligner {aligner} \
        -resume

outputs:
  primary:
    - path: "star_salmon/salmon.merged.gene_counts.tsv"
      description: "Raw gene counts for DESeq2/edgeR"
    - path: "star_salmon/salmon.merged.gene_tpm.tsv"
      description: "TPM normalized expression values"
    - path: "star_salmon/*.bam"
      description: "Aligned reads"

  validation:
    - file: "multiqc/multiqc_report.html"
      check: exists
      description: "QC report must exist"
    - file: "star_salmon/salmon.merged.gene_counts.tsv"
      check: non_empty
      description: "Count matrix must have data"

resources:
  min_memory: "8.GB"
  recommended_memory: "32.GB"
  min_cpus: 4
  recommended_cpus: 8
  disk_space: "100.GB"

troubleshooting:
  - error: "STAR index fails"
    fix: "Increase memory with --max_memory '64.GB' or provide pre-built --star_index"
  - error: "Low alignment rate"
    fix: "Verify genome matches species; check FastQC for adapter contamination"
  - error: "Strandedness detection fails"
    fix: "Specify explicitly with --strandedness reverse (or forward/unstranded)"

name: sarek
version: "3.7.1"
description: "Variant calling for WGS/WES data (germline and somatic)"

# Documentation and source - NOTE: Update version in URLs when upgrading pipeline
urls:
  documentation: "https://nf-co.re/sarek/{version}/"
  parameters: "https://nf-co.re/sarek/{version}/parameters/"
  output_docs: "https://nf-co.re/sarek/{version}/docs/output/"
  github: "https://github.com/nf-core/sarek"
  releases: "https://github.com/nf-core/sarek/releases"

data_types:
  - WGS
  - WES
  - whole genome sequencing
  - whole exome sequencing
  - tumor-normal
  - germline
  - somatic

detection_hints:
  filename:
    - tumor
    - normal
    - germline
    - wgs
    - wes
    - exome
    - dna
    - variant
  directory:
    - variant
    - wgs
    - wes
    - exome
    - germline
    - somatic

samplesheet:
  input_types:
    - fastq
    - bam
    - cram

  columns:
    - name: patient
      required: true
      type: string
      inference: filename
      description: "Patient/subject identifier for grouping samples"

    - name: sample
      required: true
      type: string
      inference: filename
      description: "Sample identifier (e.g., tumor, normal)"

    - name: lane
      required: false
      type: string
      default: "L001"
      inference: filename
      description: "Sequencing lane"

    - name: fastq_1
      required: true
      type: path
      inference: auto
      condition: "input_type == 'fastq'"
      description: "Absolute path to R1 FASTQ"

    - name: fastq_2
      required: false
      type: path
      inference: auto
      condition: "input_type == 'fastq'"
      description: "Absolute path to R2 FASTQ"

    - name: bam
      required: true
      type: path
      inference: auto
      condition: "input_type in ['bam', 'cram']"
      description: "Absolute path to BAM/CRAM file"

    - name: bai
      required: true
      type: path
      inference: auto
      condition: "input_type in ['bam', 'cram']"
      description: "Absolute path to BAM/CRAM index"

    - name: status
      required: false
      type: integer
      allowed:
        - 0
        - 1
      default: 0
      inference: filename
      description: "0=normal, 1=tumor (critical for somatic calling)"

decision_points:
  - parameter: genome
    prompt: "Which reference genome should be used?"
    options:
      - value: GRCh38
        label: "Human GRCh38/hg38 (recommended)"
        description: "Latest human reference with most annotation support"
      - value: GRCh37
        label: "Human GRCh37/hg19 (legacy)"
        description: "For compatibility with older datasets"
      - value: mm10
        label: "Mouse mm10"
        description: "Mouse reference genome"
    default: GRCh38
    recommendation: "Default to GRCh38 for human data"

  - parameter: tools
    prompt: "What type of variant calling do you need?"
    options:
      - value: "haplotypecaller,snpeff"
        label: "Germline variants (single samples)"
        description: "For finding inherited variants in normal samples"
        condition: "no tumor samples detected"
      - value: "mutect2,strelka,snpeff"
        label: "Somatic variants (tumor-normal pairs)"
        description: "For finding cancer mutations with matched normal"
        condition: "tumor-normal pairs detected"
      - value: "haplotypecaller,deepvariant,snpeff"
        label: "Germline with DeepVariant"
        description: "Higher accuracy germline calling (requires GPU)"
      - value: "mutect2,manta,snpeff"
        label: "Somatic with structural variants"
        description: "Comprehensive tumor analysis including SVs"
    default: "haplotypecaller,snpeff"
    recommendation: "Use somatic tools if tumor/normal pairs detected, otherwise germline"

  - parameter: wes
    prompt: "Is this whole exome sequencing (WES) data?"
    options:
      - value: "false"
        label: "No - Whole Genome Sequencing (WGS)"
        description: "Full genome coverage"
      - value: "true"
        label: "Yes - Whole Exome Sequencing (WES)"
        description: "Requires --intervals BED file"
    default: "false"
    recommendation: "If WES, user must provide intervals BED file"

test_profile:
  command: "nextflow run nf-core/sarek -r 3.7.1 -profile test,docker --outdir test_sarek"
  duration: "20 minutes"
  success_indicators:
    - "test_sarek/multiqc/multiqc_report.html"
  log_pattern: "Pipeline completed successfully"

run_command:
  template: |
    nextflow run nf-core/sarek \
        -r 3.7.1 \
        -profile docker \
        --input {samplesheet} \
        --outdir {outdir} \
        --genome {genome} \
        --tools {tools} \
        -resume

  wes_template: |
    nextflow run nf-core/sarek \
        -r 3.7.1 \
        -profile docker \
        --input {samplesheet} \
        --outdir {outdir} \
        --genome {genome} \
        --tools {tools} \
        --wes \
        --intervals {intervals} \
        -resume

outputs:
  primary:
    - path: "preprocessing/recalibrated/*.recal.bam"
      description: "Analysis-ready BAM files"
    - path: "variant_calling/*/*.vcf.gz"
      description: "Variant call files"
    - path: "annotation/snpeff/*.ann.vcf.gz"
      description: "Annotated variants"

  validation:
    - file: "multiqc/multiqc_report.html"
      check: exists
      description: "QC report must exist"
    - file: "preprocessing/recalibrated"
      check: exists
      description: "Recalibrated BAMs directory"

resources:
  min_memory: "16.GB"
  recommended_memory: "64.GB"
  wgs_memory: "128.GB"
  min_cpus: 4
  recommended_cpus: 16
  disk_space: "500.GB"

troubleshooting:
  - error: "BQSR fails"
    fix: "Check known sites available for genome. Skip with --skip_bqsr for non-standard references"
  - error: "Mutect2 no variants"
    fix: "Verify tumor/normal pairing in samplesheet (check status column: 0=normal, 1=tumor)"
  - error: "Out of memory"
    fix: "--max_memory '128.GB' for WGS data"
  - error: "DeepVariant GPU issues"
    fix: "Ensure NVIDIA Docker runtime configured, or use CPU mode"

tumor_normal_keywords:
  tumor:
    - tumor
    - tumour
    - met
    - metastasis
    - primary
    - cancer
    - malignant
  normal:
    - normal
    - germline
    - blood
    - pbmc
    - control
    - healthy
    - matched

#!/usr/bin/env python3
"""
Auto-detect appropriate nf-core pipeline from data directory.

Analyzes filenames, directory structure, and file content hints to suggest
the most appropriate pipeline for the data.

Usage:
    python detect_data_type.py /path/to/data
    python detect_data_type.py /path/to/data --json
"""

import argparse
import json
import os
import sys
from pathlib import Path
from typing import Dict, List, Tuple

import yaml


def load_all_pipeline_configs() -> Dict[str, Dict]:
    """Load all pipeline configurations."""
    config_dir = Path(__file__).parent / "config" / "pipelines"
    configs = {}

    for config_file in config_dir.glob("*.yaml"):
        if config_file.stem.startswith("_"):
            continue
        with open(config_file) as f:
            configs[config_file.stem] = yaml.safe_load(f)

    return configs


def scan_directory(directory: str) -> Dict:
    """Scan directory and collect file information."""
    info = {
        'fastq_count': 0,
        'bam_count': 0,
        'cram_count': 0,
        'filenames': [],
        'directories': [],
        'total_size_gb': 0,
    }

    directory = os.path.abspath(directory)

    for root, dirs, files in os.walk(directory):
        # Collect directory names
        rel_root = os.path.relpath(root, directory)
        if rel_root != '.':
            info['directories'].append(rel_root.lower())

        for filename in files:
            filename_lower = filename.lower()

            # Count file types
            if any(filename_lower.endswith(ext) for ext in ['.fastq.gz', '.fq.gz', '.fastq', '.fq']):
                info['fastq_count'] += 1
            elif filename_lower.endswith('.bam'):
                info['bam_count'] += 1
            elif filename_lower.endswith('.cram'):
                info['cram_count'] += 1

            # Collect filenames for pattern matching
            info['filenames'].append(filename_lower)

            # Sum file sizes
            try:
                size = os.path.getsize(os.path.join(root, filename))
                info['total_size_gb'] += size / (1024**3)
            except Exception:
                pass

    return info


def calculate_pipeline_scores(scan_info: Dict, configs: Dict) -> Dict[str, Dict]:
    """Calculate confidence scores for each pipeline."""
    scores = {}

    for pipeline_name, config in configs.items():
        score = 0
        matches = []

        # Check detection hints
        hints = config.get('detection_hints', {})

        # Filename hints
        filename_hints = hints.get('filename', [])
        for hint in filename_hints:
            hint_lower = hint.lower()
            for filename in scan_info['filenames']:
                if hint_lower in filename:
                    score += 10
                    matches.append(f"Filename contains '{hint}'")
                    break

        # Directory hints
        directory_hints = hints.get('directory', [])
        for hint in directory_hints:
            hint_lower = hint.lower()
            for dirname in scan_info['directories']:
                if hint_lower in dirname:
                    score += 15
                    matches.append(f"Directory contains '{hint}'")
                    break

        # Check data type compatibility
        data_types = config.get('data_types', [])
        input_types = config.get('samplesheet', {}).get('input_types', ['fastq'])

        # Prefer pipelines that support the available file types
        if 'fastq' in input_types and scan_info['fastq_count'] > 0:
            score += 5
        if 'bam' in input_types and scan_info['bam_count'] > 0:
            score += 5
        if 'cram' in input_types and scan_info['cram_count'] > 0:
            score += 5

        # Pipeline-specific boosts
        if pipeline_name == 'sarek':
            # Check for tumor/normal indicators
            tumor_indicators = ['tumor', 'tumour', 'cancer', 'met', 'primary']
            normal_indicators = ['normal', 'germline', 'blood', 'control']

            has_tumor = any(ind in ' '.join(scan_info['filenames']) for ind in tumor_indicators)
            has_normal = any(ind in ' '.join(scan_info['filenames']) for ind in normal_indicators)

            if has_tumor or has_normal:
                score += 20
                if has_tumor:
                    matches.append("Found tumor sample indicators")
                if has_normal:
                    matches.append("Found normal sample indicators")

            # DNA-related hints
            dna_hints = ['wgs', 'wes', 'exome', 'dna', 'variant', 'snp', 'indel']
            for hint in dna_hints:
                if hint in ' '.join(scan_info['filenames'] + scan_info['directories']):
                    score += 10
                    matches.append(f"Found DNA/variant indicator: '{hint}'")
                    break

        elif pipeline_name == 'rnaseq':
            # RNA-related hints
            rna_hints = ['rna', 'rnaseq', 'mrna', 'expression', 'transcript', 'counts']
            for hint in rna_hints:
                if hint in ' '.join(scan_info['filenames'] + scan_info['directories']):
                    score += 15
                    matches.append(f"Found RNA indicator: '{hint}'")
                    break

        elif pipeline_name == 'atacseq':
            # ATAC-related hints
            atac_hints = ['atac', 'atacseq', 'chromatin', 'accessibility', 'peak', 'macs']
            for hint in atac_hints:
                if hint in ' '.join(scan_info['filenames'] + scan_info['directories']):
                    score += 20
                    matches.append(f"Found ATAC-seq indicator: '{hint}'")
                    break

        scores[pipeline_name] = {
            'score': score,
            'matches': matches,
            'description': config.get('description', ''),
            'version': config.get('version', 'unknown'),
        }

    return scores


def detect_pipeline(directory: str) -> Tuple[str, Dict]:
    """
    Detect the most appropriate pipeline for the data.

    Args:
        directory: Path to data directory

    Returns:
        Tuple of (recommended_pipeline, all_scores)
    """
    if not os.path.isdir(directory):
        raise ValueError(f"Not a directory: {directory}")

    configs = load_all_pipeline_configs()
    scan_info = scan_directory(directory)

    # Check if any sequencing files found
    total_files = scan_info['fastq_count'] + scan_info['bam_count'] + scan_info['cram_count']
    if total_files == 0:
        raise ValueError(f"No sequencing files (FASTQ/BAM/CRAM) found in {directory}")

    scores = calculate_pipeline_scores(scan_info, configs)

    # Find highest scoring pipeline
    best_pipeline = max(scores.keys(), key=lambda k: scores[k]['score'])

    return best_pipeline, scores


def print_results(
    directory: str,
    recommended: str,
    scores: Dict,
    scan_info: Dict,
    output_json: bool = False
):
    """Print detection results."""
    if output_json:
        result = {
            'recommended': recommended,
            'scores': scores,
            'scan_info': {
                'fastq_count': scan_info['fastq_count'],
                'bam_count': scan_info['bam_count'],
                'cram_count': scan_info['cram_count'],
                'total_size_gb': round(scan_info['total_size_gb'], 2),
            }
        }
        print(json.dumps(result, indent=2))
        return

    print("\n" + "=" * 50)
    print("  nf-core Pipeline Detection")
    print("=" * 50)
    print(f"\nDirectory: {directory}")
    print(f"Files found: {scan_info['fastq_count']} FASTQ, "
          f"{scan_info['bam_count']} BAM, {scan_info['cram_count']} CRAM")
    print(f"Total size: {scan_info['total_size_gb']:.1f} GB")

    print("\n--- Pipeline Scores ---")
    sorted_pipelines = sorted(scores.keys(), key=lambda k: scores[k]['score'], reverse=True)

    for pipeline in sorted_pipelines:
        info = scores[pipeline]
        indicator = "→" if pipeline == recommended else " "
        print(f"\n{indicator} {pipeline} (score: {info['score']})")
        print(f"  {info['description']}")
        if info['matches']:
            print(f"  Matches: {', '.join(info['matches'][:3])}")

    print(f"\n{'=' * 50}")
    print(f"\n\033[92mRecommended: {recommended}\033[0m")
    print(f"Version: {scores[recommended]['version']}")

    # Print suggested next steps
    print(f"\n--- Next Steps ---")
    print(f"1. Run environment check:")
    print(f"   python scripts/check_environment.py")
    print(f"\n2. Run test profile:")
    config = load_all_pipeline_configs().get(recommended, {})
    test_cmd = config.get('test_profile', {}).get('command', '')
    if test_cmd:
        print(f"   {test_cmd}")
    print(f"\n3. Generate samplesheet:")
    print(f"   python scripts/generate_samplesheet.py {directory} {recommended}")


def main():
    parser = argparse.ArgumentParser(
        description='Detect appropriate nf-core pipeline for data',
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog="""
Examples:
    %(prog)s ./data
    %(prog)s ./fastqs --json
        """
    )

    parser.add_argument('directory', help='Directory containing sequencing data')
    parser.add_argument('--json', action='store_true', help='Output as JSON')

    args = parser.parse_args()

    try:
        scan_info = scan_directory(args.directory)
        recommended, scores = detect_pipeline(args.directory)
        print_results(args.directory, recommended, scores, scan_info, args.json)
        sys.exit(0)

    except ValueError as e:
        if args.json:
            print(json.dumps({'error': str(e)}))
        else:
            print(f"Error: {e}")
        sys.exit(1)

    except Exception as e:
        if args.json:
            print(json.dumps({'error': str(e)}))
        else:
            print(f"Error: {e}")
        sys.exit(1)


if __name__ == '__main__':
    main()

#!/usr/bin/env python3
"""
Enhanced nf-core samplesheet generator.

Features:
- FASTQ, BAM, and CRAM support
- Tumor/normal status inference for sarek
- Robust R1/R2 matching with scoring
- Pre-write validation with clear error messages
- Pipeline config-driven column generation

Usage:
    python generate_samplesheet.py /path/to/data rnaseq -o samplesheet.csv
    python generate_samplesheet.py /path/to/bams sarek --input-type bam
    python generate_samplesheet.py --validate samplesheet.csv rnaseq
"""

import argparse
import os
import sys
from pathlib import Path
from typing import Dict, List, Optional, Tuple

import yaml

# Add parent directory to path for utils import
sys.path.insert(0, str(Path(__file__).parent))

from utils.file_discovery import discover_files, detect_input_type, find_index_file
from utils.sample_inference import (
    extract_sample_info,
    infer_tumor_normal_status,
    match_read_pairs,
    extract_replicate_number
)
from utils.validators import validate_samplesheet, ValidationResult


def load_pipeline_config(pipeline: str) -> Dict:
    """Load pipeline configuration from YAML."""
    config_dir = Path(__file__).parent / "config" / "pipelines"
    config_file = config_dir / f"{pipeline}.yaml"

    if not config_file.exists():
        available = [f.stem for f in config_dir.glob("*.yaml") if not f.stem.startswith("_")]
        raise ValueError(f"Unknown pipeline '{pipeline}'. Available: {', '.join(available)}")

    with open(config_file) as f:
        return yaml.safe_load(f)


def generate_samplesheet(
    input_dir: str,
    pipeline: str,
    output_file: Optional[str] = None,
    input_type: str = "auto",
    single_end: bool = False,
    interactive: bool = True
) -> Tuple[Optional[str], ValidationResult]:
    """
    Generate samplesheet for specified pipeline.

    Args:
        input_dir: Directory containing sequencing files
        pipeline: Pipeline name (rnaseq, sarek, atacseq)
        output_file: Output CSV path (default: samplesheet_{pipeline}.csv)
        input_type: File type (auto, fastq, bam, cram)
        single_end: Suppress pairing warnings for single-end data
        interactive: Prompt for missing info

    Returns:
        Tuple of (output_path, validation_result)
    """
    config = load_pipeline_config(pipeline)
    samplesheet_config = config.get("samplesheet", {})
    supported_types = samplesheet_config.get("input_types", ["fastq"])

    # Determine input type
    if input_type == "auto":
        input_type = detect_input_type(input_dir)
        print(f"Auto-detected input type: {input_type.upper()}")

    if input_type not in supported_types:
        return None, ValidationResult(
            valid=False,
            errors=[f"Pipeline '{pipeline}' does not support {input_type.upper()} input. "
                    f"Supported: {supported_types}"]
        )

    # Discover files
    try:
        files = discover_files(input_dir, input_type)
    except ValueError as e:
        return None, ValidationResult(valid=False, errors=[str(e)])

    if not files:
        return None, ValidationResult(
            valid=False,
            errors=[f"No {input_type.upper()} files found in {input_dir}"],
            suggestions=[
                "Check directory path is correct",
                "Verify file extensions (.fastq.gz, .fq.gz, .bam, .cram)",
                f"Run: ls {input_dir}"
            ]
        )

    print(f"Found {len(files)} {input_type.upper()} files")

    # Process based on input type
    if input_type == "fastq":
        rows = _process_fastq_files(files, config, single_end)
    else:
        rows = _process_alignment_files(files, config, input_type)

    if not rows:
        return None, ValidationResult(
            valid=False,
            errors=["Could not generate any samplesheet rows from files"]
        )

    print(f"Generated {len(rows)} samplesheet rows")

    # Pipeline-specific processing
    if pipeline == "sarek":
        rows = _process_sarek_samples(rows, interactive)
    elif pipeline == "atacseq":
        rows = _process_atacseq_samples(rows)

    # Validate before writing
    validation = validate_samplesheet(rows, pipeline, config)

    if not validation.valid:
        print("\nValidation errors:")
        for error in validation.errors:
            print(f"  - {error}")

        if interactive:
            response = input("\nProceed anyway? [y/N]: ").strip().lower()
            if response != 'y':
                return None, validation
    elif validation.warnings:
        print("\nWarnings:")
        for warning in validation.warnings:
            print(f"  - {warning}")

    # Determine output path
    output_path = output_file or f"samplesheet_{pipeline}.csv"

    # Write samplesheet
    _write_samplesheet(rows, config, output_path)

    print(f"\nGenerated: {output_path}")
    print(f"  Pipeline: {pipeline} v{config.get('version', 'unknown')}")
    print(f"  Samples: {len(set(r.get('sample', r.get('patient', '')) for r in rows))}")
    print(f"  Rows: {len(rows)}")

    # Preview
    _print_preview(rows, config)

    return output_path, validation


def _process_fastq_files(files, config: Dict, single_end: bool) -> List[Dict]:
    """Process FASTQ files into samplesheet rows."""
    pairs = match_read_pairs(files)

    if not pairs:
        return []

    # Check for unpaired files
    unpaired = [k for k, v in pairs.items() if v.get('r1') and not v.get('r2')]
    if unpaired and not single_end:
        print(f"\nNote: {len(unpaired)} samples appear to be single-end (no R2)")

    rows = []
    columns = config.get("samplesheet", {}).get("columns", [])

    for sample_key, pair_info in sorted(pairs.items()):
        if not pair_info.get('r1'):
            continue  # Skip entries with only R2

        info = pair_info.get('info', {})

        row = {
            'sample': info.get('sample', sample_key),
            'fastq_1': str(Path(pair_info['r1']).absolute()),
            'fastq_2': str(Path(pair_info['r2']).absolute()) if pair_info.get('r2') else '',
        }

        # Add additional info from filename
        if 'patient' in [c['name'] for c in columns]:
            row['patient'] = info.get('patient', info.get('sample', sample_key))

        if 'lane' in [c['name'] for c in columns]:
            row['lane'] = info.get('lane', 'L001')

        # Apply defaults from config
        for col in columns:
            if col['name'] not in row and 'default' in col:
                row[col['name']] = col['default']

        rows.append(row)

    return rows


def _process_alignment_files(files, config: Dict, input_type: str) -> List[Dict]:
    """Process BAM/CRAM files into samplesheet rows."""
    rows = []
    columns = config.get("samplesheet", {}).get("columns", [])

    for file_info in files:
        # Find index file
        index_path = find_index_file(file_info.path)

        info = extract_sample_info(file_info.path)

        row = {
            'sample': info.get('sample', file_info.stem),
            'bam': str(Path(file_info.path).absolute()),
            'bai': str(Path(index_path).absolute()) if index_path else '',
        }

        # Add patient for sarek
        if 'patient' in [c['name'] for c in columns]:
            row['patient'] = info.get('patient', info.get('sample', file_info.stem))

        # Apply defaults
        for col in columns:
            if col['name'] not in row and 'default' in col:
                row[col['name']] = col['default']

        # Warn if no index found
        if not index_path:
            print(f"  Warning: No index found for {file_info.name}")

        rows.append(row)

    return rows


def _process_sarek_samples(rows: List[Dict], interactive: bool) -> List[Dict]:
    """Process sarek samples: infer and confirm tumor/normal status."""
    # Auto-infer status from sample names
    for row in rows:
        sample_name = row.get('sample', '')
        inferred = infer_tumor_normal_status(sample_name)
        if inferred is not None:
            row['status'] = inferred

    # Report inference results
    inferred_tumor = [r for r in rows if r.get('status') == 1]
    inferred_normal = [r for r in rows if r.get('status') == 0]
    unknown = [r for r in rows if 'status' not in r]

    if inferred_tumor or inferred_normal:
        print(f"\nTumor/normal inference:")
        print(f"  Tumor samples: {len(inferred_tumor)}")
        print(f"  Normal samples: {len(inferred_normal)}")

    # Handle unknown samples
    if unknown and interactive:
        print(f"\n{len(unknown)} sample(s) with unknown status:")
        for r in unknown:
            print(f"  - {r.get('sample')}")

        print("\nSpecify status for each (0=normal, 1=tumor, Enter=skip):")
        for r in unknown:
            response = input(f"  {r.get('sample')} [0/1/Enter]: ").strip()
            if response in ['0', '1']:
                r['status'] = int(response)
            else:
                r['status'] = 0  # Default to normal
                print(f"    Defaulting to normal (0)")
    elif unknown:
        # Non-interactive: default to normal
        for r in unknown:
            r['status'] = 0

    return rows


def _process_atacseq_samples(rows: List[Dict]) -> List[Dict]:
    """Process ATAC-seq samples: ensure replicate numbers."""
    # Group by sample name
    sample_counts = {}
    for row in rows:
        sample = row.get('sample', '')
        if sample not in sample_counts:
            sample_counts[sample] = 0
        sample_counts[sample] += 1

    # Assign replicate numbers if not present
    sample_rep = {}
    for row in rows:
        sample = row.get('sample', '')

        if 'replicate' not in row or not row['replicate']:
            # Try to extract from filename
            extracted = extract_replicate_number(row.get('fastq_1', ''))
            if extracted:
                row['replicate'] = extracted
            else:
                # Auto-assign sequential
                if sample not in sample_rep:
                    sample_rep[sample] = 0
                sample_rep[sample] += 1
                row['replicate'] = sample_rep[sample]

    return rows


def _write_samplesheet(rows: List[Dict], config: Dict, output_path: str):
    """Write samplesheet to CSV file."""
    columns = config.get("samplesheet", {}).get("columns", [])
    column_names = [c['name'] for c in columns]

    # Filter to columns that have data
    active_columns = [c for c in column_names if any(c in row and row[c] for row in rows)]

    # Ensure fastq_1/fastq_2 or bam/bai are included
    for required in ['fastq_1', 'bam']:
        if required in column_names and required not in active_columns:
            if any(required in row for row in rows):
                active_columns.append(required)

    # Maintain original column order
    active_columns = [c for c in column_names if c in active_columns]

    with open(output_path, 'w') as f:
        f.write(','.join(active_columns) + '\n')
        for row in rows:
            values = [str(row.get(col, '')) for col in active_columns]
            f.write(','.join(values) + '\n')


def _print_preview(rows: List[Dict], config: Dict):
    """Print preview of generated samplesheet."""
    columns = config.get("samplesheet", {}).get("columns", [])
    column_names = [c['name'] for c in columns]
    active_columns = [c for c in column_names if any(c in row for row in rows)]

    print(f"\nPreview (first 3 rows):")
    print(','.join(active_columns))
    for row in rows[:3]:
        values = [str(row.get(col, ''))[:40] for col in active_columns]  # Truncate long paths
        print(','.join(values))
    if len(rows) > 3:
        print(f"... ({len(rows) - 3} more rows)")


def validate_existing_samplesheet(csv_path: str, pipeline: str) -> ValidationResult:
    """Validate an existing samplesheet file."""
    import csv

    if not os.path.exists(csv_path):
        return ValidationResult(valid=False, errors=[f"File not found: {csv_path}"])

    try:
        with open(csv_path, 'r') as f:
            reader = csv.DictReader(f)
            rows = list(reader)
    except Exception as e:
        return ValidationResult(valid=False, errors=[f"Failed to read CSV: {e}"])

    if not rows:
        return ValidationResult(valid=False, errors=["Samplesheet is empty"])

    config = load_pipeline_config(pipeline)
    return validate_samplesheet(rows, pipeline, config)


def main():
    parser = argparse.ArgumentParser(
        description='Generate nf-core samplesheet from data directory',
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog="""
Examples:
    # Generate samplesheet for RNA-seq
    %(prog)s ./fastqs rnaseq -o samples.csv

    # Generate samplesheet for sarek from BAM files
    %(prog)s ./bams sarek --input-type bam

    # Validate existing samplesheet
    %(prog)s --validate samplesheet.csv rnaseq

Supported pipelines: rnaseq, sarek, atacseq
        """
    )

    parser.add_argument('input', help='Directory with data files, or CSV path for --validate')
    parser.add_argument('pipeline', help='Pipeline name (rnaseq, sarek, atacseq)')
    parser.add_argument('-o', '--output', help='Output CSV filename')
    parser.add_argument('--input-type', choices=['auto', 'fastq', 'bam', 'cram'],
                        default='auto', help='Input file type (default: auto-detect)')
    parser.add_argument('--single-end', action='store_true',
                        help='Treat as single-end data (suppress pairing warnings)')
    parser.add_argument('--validate', action='store_true',
                        help='Validate existing samplesheet instead of generating')
    parser.add_argument('--no-interactive', action='store_true',
                        help='Non-interactive mode (use defaults)')

    args = parser.parse_args()

    try:
        if args.validate:
            # Validate existing samplesheet
            result = validate_existing_samplesheet(args.input, args.pipeline)
            if result.valid:
                print(f"✓ Samplesheet is valid for {args.pipeline}")
                if result.warnings:
                    print("\nWarnings:")
                    for w in result.warnings:
                        print(f"  - {w}")
                sys.exit(0)
            else:
                print(f"✗ Samplesheet validation failed")
                print(result.summary())
                sys.exit(1)
        else:
            # Generate new samplesheet
            if not os.path.isdir(args.input):
                print(f"Error: Not a directory: {args.input}")
                sys.exit(1)

            output_path, result = generate_samplesheet(
                args.input,
                args.pipeline,
                args.output,
                args.input_type,
                args.single_end,
                interactive=not args.no_interactive
            )

            if output_path is None:
                print("\nFailed to generate samplesheet.")
                if result.suggestions:
                    print("\nSuggestions:")
                    for s in result.suggestions:
                        print(f"  - {s}")
                sys.exit(1)

            sys.exit(0)

    except ValueError as e:
        print(f"Error: {e}")
        sys.exit(1)
    except KeyboardInterrupt:
        print("\nAborted.")
        sys.exit(1)


if __name__ == '__main__':
    main()