
Publication Converter
- 139 installs
- 38 repo stars
- Updated January 5, 2026
- qodex-ai/ai-agent-skills
Convert publications across formats—PDF, LaTeX, Markdown, HTML—for ingestion, republishing, or agent-readable corpora with structure preserved.
About
Enables Claude to build publication converter tooling for agent pipelines: parse PDFs and academic formats, emit clean Markdown or HTML, preserve headings and citations, and batch-convert corpora for search, summarization, or republication workflows.
- Multi-format publication parsing
- Structure-preserving conversion
- Agent-callable conversion tools
- Batch and pipeline workflows
- Metadata and citation extraction
Publication Converter by the numbers
- 139 all-time installs (skills.sh)
- Ranked #678 of 2,715 Automation & Workflows skills by installs in the Skillselion catalog
- Data as of Aug 4, 2026 (Skillselion catalog sync)
npx skills add https://github.com/qodex-ai/ai-agent-skills --skill publication-converterAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 139 |
|---|---|
| repo stars | ★ 38 |
| Last updated | January 5, 2026 |
| Repository | qodex-ai/ai-agent-skills ↗ |
What it does
Convert publications across formats—PDF, LaTeX, Markdown, HTML—for ingestion, republishing, or agent-readable corpora with structure preserved.
Files
Markdown to EPUB Converter Skill
This skill transforms markdown documents into professional EPUB ebook files. Perfect for converting research documents, blog posts, articles, or chat conversation summaries into portable, device-agnostic ebook formats.
Overview
The skill accepts markdown content in multiple formats and generates a properly formatted EPUB3 file that works across all major ebook readers including:
- Apple Books
- Amazon Kindle (via Kindle for Mac/Windows/iOS/Android)
- Google Play Books
- Kobo and other EPUB readers
- Any standard EPUB reader
Input Formats
Option 1: Raw Markdown Text
Provide markdown content directly in your message:
Convert this markdown to EPUB:
# My Book Title
## Chapter 1
This is chapter one content...Option 2: File Path
Provide a path to a markdown file to be converted.
How It Works
1. Markdown Parsing: Analyzes your markdown and automatically:
- Treats H1 headers (
#) as chapter boundaries - Treats H2 headers (
##) as section headings within chapters - Preserves formatting (bold, italic, links, lists, code blocks)
2. Structure Generation: Creates proper EPUB structure:
- Automatic table of contents from chapters
- Navigation document (EPUB3 standard)
- Metadata (title, language, etc.)
3. File Creation: Generates a valid EPUB3 file ready for download and use
Usage Examples
Example 1: Convert a Blog Post
"Convert this markdown blog post to EPUB:
How to Build a Simple Web Server
Introduction
...content..."
Example 2: Convert a Research Summary
"I have research notes in markdown format. Convert them to an EPUB ebook. The content is:
Research Project: Machine Learning Basics
Chapter 1: Fundamentals
..."
Example 3: Convert a Chat Summary
"Summarize our conversation so far as markdown and convert it to an EPUB for reference"
Output
The skill generates a downloadable EPUB file that includes:
- Professional formatting
- Automatic table of contents
- Proper chapter structure
- Support for markdown formatting elements:
- Headers (all levels)
- Bold and italic text
- Hyperlinks
- Lists (ordered and unordered)
- Code blocks and inline code
- Blockquotes
- Horizontal rules
Markdown Elements Supported
| Element | Markdown | Support | Notes |
|---|---|---|---|
| Headers | # H1 through ###### H6 | Full | Auto TOC generation |
| Bold | **text** or __text__ | Full | |
| Italic | *text* or _text_ | Full | |
| Links | [text](url) | Full | Clickable in ebooks |
| Lists | - item or 1. item | Full | Nested lists supported |
| Code blocks | `language | Enhanced | Syntax highlighting ready, monospace fonts |
| Inline code | code | Enhanced | Styled background, borders |
| Tables | Markdown tables | Enhanced | Styled headers, alternating rows |
| Blockquotes | > quote | Full | Styled with left border |
| Horizontal rule | --- or *** | Full |
Advanced Features
Enhanced Code Block Support
Code blocks are beautifully formatted with:
- Premium monospace fonts: SF Mono, Monaco, Fira Code, Consolas, and more
- Styled backgrounds: Subtle gray background with blue accent border
- Language detection: Specify language after
`for future syntax highlighting - Proper escaping: HTML characters are safely escaped
- Overflow handling: Horizontal scrolling for long lines
Example:
def fibonacci(n):
if n <= 1:
return n
return fibonacci(n-1) + fibonacci(n-2)Enhanced Table Support
Tables are rendered with professional styling:
- Styled headers: Blue background with white text
- Alternating rows: Zebra striping for readability
- Cell padding: Comfortable spacing for easy reading
- Inline formatting: Code, bold, italic, and links work in cells
- Responsive: Tables adapt to different screen sizes
Example:
| Feature | Status | Notes |
|---|---|---|
| Headers | ✓ | Full support |
| Code | ✓ | Enhanced styling |
| Tables | ✓ | Professional layout |
Custom Title and Metadata
You can specify EPUB metadata:
- Book title (defaults to first H1 header)
- Author name
- Language
- Publication date
Chapter Organization
Chapters are automatically detected from:
- H1 headers (
#) as primary chapter breaks - Logical content sections between H1s
- Automatic page breaks between chapters
Styling
The generated EPUB uses clean, readable default styling that:
- Respects the reader's font preferences
- Works on all screen sizes
- Maintains proper spacing and hierarchy
- Includes appropriate margins and padding
Technical Details
- Format: EPUB3 (compatible with all modern readers)
- Encoding: UTF-8
- HTML Version: XHTML 1.1
- CSS Support: Responsive styling
Downloading Your EPUB
After generation, the file will be available for download. You can then: 1. Download the EPUB to your computer 2. Open it with your preferred ebook reader 3. Transfer to your Kindle, iPad, or other device 4. Upload directly to Kindle via email or cloud
Tips for Best Results
1. Use Proper Markdown Structure: The skill works best when markdown follows standard conventions (H1 for titles, H2 for sections)
2. Clear Chapter Breaks: Use H1 headers to clearly mark chapter divisions
3. Descriptive Headers: Headers become the table of contents, so make them clear and descriptive
4. Content Organization: Place content logically between headers
5. Supported Formatting: Stick to basic markdown formatting for best compatibility across all readers
Troubleshooting
EPUB doesn't open: Ensure your markdown is properly formatted. Check for matching brackets in links and proper syntax.
Table of contents is empty: Make sure your markdown includes H1 headers to define chapters.
Formatting looks different: EPUB readers apply their own fonts and styling. This is normal and expected behavior.
Scripts
epub_generator.py- Core EPUB file creation and formattingmarkdown_processor.py- Markdown parsing and structure extraction
Future Enhancements
- Auto-generated cover pages with custom images
- Kindle-specific optimizations (.mobi format)
- Custom CSS styling per user preferences
- Multi-document merging
- Image embedding and optimization
- Advanced metadata support
Markdown to EPUB Skill - Technical Reference
Advanced technical documentation for extending and customizing the Markdown to EPUB skill.
Module Overview
markdown_processor.py
Core module for parsing markdown and extracting document structure.
Main Classes
MarkdownProcessor
processor = MarkdownProcessor()
result = processor.process(markdown_content)Methods:
process(markdown_content: str) -> Dict- Parse markdown and extract structure_extract_frontmatter(content: str) -> str- Extract YAML frontmatter_extract_metadata(content: str)- Extract metadata from document headers_parse_chapters(content: str) -> List[Chapter]- Parse into chapters_parse_sections(content: str, min_level: int) -> List[Section]- Parse sections_build_toc() -> List[Dict]- Build table of contentsmarkdown_to_html(markdown_text: str) -> str- Convert markdown to HTML_render_inline(text: str) -> str- Process inline elements (bold, italic, links)
Data Classes
EbookMetadata
metadata = EbookMetadata(
title="My Book",
author="John Doe",
language="en",
date="2025-01-15",
identifier="unique-id"
)Chapter
chapter = Chapter(
title="Chapter 1",
content="Introduction text...",
sections=[Section(...), ...],
anchor="chapter-1"
)Section
section = Section(
title="Section 1.1",
level=2,
content="Section content...",
anchor="section-1-1"
)epub_generator.py
EPUB file creation and management using ebooklib.
Main Classes
EPUBGenerator
generator = EPUBGenerator(metadata)
success = generator.generate(chapters, output_path)Methods:
generate(chapters: List[Chapter], output_path: str) -> bool- Main generation method_create_book()- Initialize EPUB book object_add_chapters()- Add chapters to book_render_chapter(chapter: Chapter) -> str- Render chapter to XHTML_render_content(content: str) -> str- Render markdown content to HTML_add_style()- Add CSS styling_add_toc()- Generate table of contents_write_epub(output_path: str)- Write EPUB file to disk
Default CSS
- Embedded in
EPUBGenerator.DEFAULT_CSS - Customizable by subclassing
- Responsive design for all screen sizes
Convenience Functions
# Create EPUB from markdown string
create_epub_from_markdown(
markdown_content: str,
output_path: str,
title: Optional[str] = None,
author: Optional[str] = None
) -> boolHTML/XHTML Generation
Markdown to HTML Conversion
The markdown_to_html() method converts markdown to semantic HTML:
markdown = "# Title\n\nSome **bold** text"
html = MarkdownProcessor.markdown_to_html(markdown)
# Returns: <h1>Title</h1>\n<p>Some <strong>bold</strong> text</p>Supported Elements
- Headers (H1-H6):
# to ###### - Emphasis:
**bold**,*italic*,__bold__,_italic_ - Links:
[text](url) - Lists:
- item,* item,1. item - Code: `
inline,``(blocks) - Blockquotes:
> quote - Horizontal Rules:
---,***,___
Special Handling
- HTML escaping:
&,<,>,",'are automatically escaped - Code blocks: Content is escaped and preserved as-is
- Paragraphs: Double newlines create new
<p>tags - Links: Properly encoded href attributes
EPUB Structure
Generated File Layout
metadata.opf # Package metadata
nav.xhtml # EPUB3 navigation
toc.ncx # NCBI NCX (compatibility)
OEBPS/
├── chap_001.xhtml # Chapter files
├── chap_002.xhtml
├── ...
├── style/
│ └── main.css # Embedded stylesheet
└── [other resources]Metadata Fields
From EbookMetadata:
- identifier: Unique ID (auto-generated UUID if not provided)
- title: Book title (required)
- language: Language code (default: "en")
- author: Author name
- date: Publication date (optional)
Navigation
- NCX (Navigation Control File): For backward compatibility with older readers
- NAV (EPUB3 Navigation Document): Standard for EPUB3 readers
- Automatic generation from chapter/section hierarchy
Customization
Extending the Classes
Custom Styling
class CustomEPUBGenerator(EPUBGenerator):
CUSTOM_CSS = """/* Your CSS here */"""
def _add_style(self):
# Custom styling logic
super()._add_style()Custom Metadata
metadata = EbookMetadata()
metadata.title = "My Custom Title"
metadata.author = "Custom Author"
generator = EPUBGenerator(metadata)Custom HTML Rendering
class CustomProcessor(MarkdownProcessor):
@staticmethod
def markdown_to_html(markdown_text):
# Custom conversion logic
return htmlAdding New Markdown Features
1. Extend markdown_to_html() in MarkdownProcessor 2. Add parsing logic for new markdown syntax 3. Return proper HTML equivalent 4. Test with test_epub_skill.py
Example - Add strikethrough support:
# In markdown_to_html()
text = re.sub(r'~~(.+?)~~', r'<del>\1</del>', text)Performance Considerations
Large Documents
- Processing is O(n) where n = document length
- Memory usage: ~3-5x the markdown source size
- EPUB generation typically < 100ms for 100+ page documents
Optimization Tips
1. Batch processing: Process multiple documents in one run 2. Chapter splitting: Break very large documents into smaller files 3. Content optimization: Remove unnecessary whitespace/formatting 4. Lazy loading: Generate EPUBs on-demand rather than precomputing
Debugging
Enable Debug Output
import logging
logging.basicConfig(level=logging.DEBUG)
processor = MarkdownProcessor()
result = processor.process(markdown_content)Common Issues
Empty chapters
- Check that markdown has proper structure
- Verify no sections have completely empty content
- Use
_render_content()to debug HTML output
Invalid XHTML
- Verify all tags are properly closed
- Check for unescaped special characters
- Use validator tools on generated EPUB
Missing TOC
- Ensure chapters/sections have proper headers
- Verify anchor generation works correctly
- Check that sections list is populated
API Integration
Using with Claude Skills
# In your skill implementation
def generate_ebook(markdown_content, title=None, author=None):
from epub_generator import create_epub_from_markdown
success = create_epub_from_markdown(
markdown_content,
"output.epub",
title=title,
author=author
)
if success:
# Return file_id or stream
return read_epub_file("output.epub")
else:
return NoneFile Handling
The skill can work with:
- Direct file paths
- File contents via Files API
- Markdown strings
- Chat message content
Testing
Unit Tests
# Test markdown parsing
processor = MarkdownProcessor()
result = processor.process(test_markdown)
assert len(result['chapters']) == expected_count
# Test HTML generation
html = MarkdownProcessor.markdown_to_html(test_content)
assert '<h1>' in html
assert '<strong>' in htmlIntegration Tests
# Test end-to-end EPUB generation
success = create_epub_from_markdown(
markdown_content,
test_output_path,
title="Test",
author="Tester"
)
assert success
assert Path(test_output_path).exists()Test Coverage
Current test coverage:
- ✓ Markdown parsing with multiple header levels
- ✓ YAML frontmatter extraction
- ✓ HTML generation and escaping
- ✓ EPUB file creation
- ✓ Edge cases (empty content, special characters)
- ✓ Table of contents generation
- ✓ Large documents (100+ chapters)
Version History
v1.0.0 (Current)
- Initial release
- Full markdown to EPUB conversion
- YAML frontmatter support
- Automatic TOC generation
- EPUB3 compliance
- Complete test suite
Future Roadmap
v1.1.0 (Planned)
- Cover page generation
- Custom CSS templates
- Image embedding
v2.0.0 (Future)
- Kindle format support (.mobi, .azw3)
- Advanced table support
- Footnotes and cross-references
- Experimental MCP integration for cover images
Contributing
Code Guidelines
- Follow PEP 8 style guide
- Add docstrings to all functions
- Include type hints
- Write tests for new features
- Update this reference documentation
Adding Features
1. Create a feature branch 2. Implement with tests 3. Update SKILL.md and REFERENCE.md 4. Submit with test results
---
Last Updated: 2025-01-16 Maintained By: Skills Development Team
ebooklib==0.18.0
markdown2==2.4.12
Pygments==2.17.2
"""
EPUB file generation module.
This module handles creating proper EPUB3 files from parsed markdown structure
using the ebooklib library.
"""
import uuid
from pathlib import Path
from typing import List, Optional, Dict
from datetime import datetime
from ebooklib import epub
from markdown_processor import (
Chapter, Section, EbookMetadata, MarkdownProcessor
)
class EPUBGenerator:
"""Generate EPUB3 files from markdown chapters and sections."""
# Default CSS for EPUB styling
DEFAULT_CSS = """
body {
font-family: Georgia, serif;
font-size: 75%;
line-height: 1.5;
margin: 0;
padding: 1em;
}
h1 {
font-size: 1.3em;
font-weight: bold;
margin: 1.5em 0 0.75em 0;
color: #1a1a1a;
page-break-before: always;
}
h2 {
font-size: 1.15em;
font-weight: bold;
margin: 1.25em 0 0.5em 0;
color: #2c3e50;
}
h3 {
font-size: 1.05em;
font-weight: bold;
margin: 1em 0 0.5em 0;
color: #34495e;
}
h4, h5, h6 {
font-size: 0.95em;
font-weight: bold;
margin: 0.75em 0 0.5em 0;
}
p {
margin: 0.75em 0;
text-align: justify;
font-size: 1em;
}
a {
color: #0066cc;
text-decoration: none;
}
a:visited {
color: #663399;
}
strong {
font-weight: bold;
}
em {
font-style: italic;
}
code, pre {
font-family: 'SF Mono', 'Monaco', 'Inconsolata', 'Fira Code', 'Fira Mono', 'Roboto Mono', 'Consolas', 'Courier New', monospace;
font-size: 0.72em;
}
code {
background-color: #f5f5f5;
padding: 0.15em 0.4em;
border-radius: 3px;
border: 1px solid #e0e0e0;
}
pre {
background-color: #f8f8f8;
border: 1px solid #e0e0e0;
border-left: 3px solid #0066cc;
padding: 0.8em;
overflow-x: auto;
margin: 1em 0;
border-radius: 4px;
line-height: 1.3;
tab-size: 2;
-moz-tab-size: 2;
}
pre code {
padding: 0;
background-color: transparent;
border: none;
font-size: 1em;
line-height: 1.3;
}
/* Line numbers in code blocks */
.line-number {
display: inline-block;
width: 2.2em;
text-align: right;
padding-right: 0.3em;
color: #999;
user-select: none;
-webkit-user-select: none;
-moz-user-select: none;
border-right: 1px solid #ddd;
margin-right: 0.4em;
}
.line-content {
display: inline;
}
/* Syntax highlighting (Pygments-based) */
.syn-keyword {
color: #0066cc;
font-weight: 600;
}
.syn-string {
color: #22863a;
}
.syn-comment {
color: #6a737d;
font-style: italic;
}
.syn-number {
color: #005cc5;
}
.syn-function {
color: #6f42c1;
}
.syn-class {
color: #d73a49;
font-weight: 600;
}
blockquote {
border-left: 4px solid #0066cc;
margin: 1em 0;
padding-left: 1em;
color: #555;
font-style: italic;
}
ul, ol {
margin: 0.75em 0;
padding-left: 2em;
}
li {
margin: 0.5em 0;
}
table {
border-collapse: collapse;
width: 100%;
margin: 1.5em 0;
font-size: 0.88em;
border: 1px solid #ddd;
}
thead {
background-color: #0066cc;
color: white;
}
th {
padding: 0.75em 1em;
text-align: left;
font-weight: bold;
border: 1px solid #0052a3;
}
td {
padding: 0.65em 1em;
border: 1px solid #ddd;
text-align: left;
}
tbody tr:nth-child(even) {
background-color: #f8f9fa;
}
tbody tr:hover {
background-color: #f0f0f0;
}
/* Code in table cells */
td code, th code {
font-size: 0.85em;
}
hr {
border: none;
border-top: 2px solid #ddd;
margin: 2em 0;
}
.toc-entry {
margin: 0.5em 0;
}
.toc-entry-h2 {
margin-left: 1.5em;
}
.toc-entry-h3 {
margin-left: 3em;
}
"""
def __init__(self, metadata: EbookMetadata = None):
"""
Initialize EPUB generator.
Args:
metadata: EbookMetadata object with title, author, etc.
"""
self.metadata = metadata or EbookMetadata()
self.book = None
self.chapters = []
self.toc_items = []
def generate(self, chapters: List[Chapter], output_path: str) -> bool:
"""
Generate EPUB file from chapters.
Args:
chapters: List of Chapter objects from markdown parser
output_path: Path where EPUB file should be saved
Returns:
True if successful, False otherwise
"""
try:
self.chapters = chapters
self._create_book()
self._add_chapters()
self._add_style()
self._add_toc()
self._write_epub(output_path)
return True
except Exception as e:
print(f"Error generating EPUB: {e}")
raise
def _create_book(self) -> None:
"""Create and configure EPUB book object."""
self.book = epub.EpubBook()
# Set metadata
self.book.set_identifier(self.metadata.identifier or str(uuid.uuid4()))
self.book.set_title(self.metadata.title or "Untitled Book")
self.book.set_language(self.metadata.language)
if self.metadata.author:
self.book.add_author(self.metadata.author)
def _add_chapters(self) -> None:
"""Add chapters to EPUB."""
all_items = []
for chapter_idx, chapter in enumerate(self.chapters):
# Create chapter HTML file
chapter_html = self._render_chapter(chapter)
chapter_file = epub.EpubHtml(
title=chapter.title,
file_name=f'chap_{chapter_idx + 1:03d}.xhtml',
lang=self.metadata.language
)
# ebooklib's set_content expects the body content, not full XHTML
# Extract just the body content
body_match = chapter_html.find('<body>')
if body_match != -1:
body_start = body_match + 6
body_end = chapter_html.find('</body>')
body_content = chapter_html[body_start:body_end]
else:
body_content = chapter_html
chapter_file.set_content(body_content)
self.book.add_item(chapter_file)
all_items.append(chapter_file)
# Add chapter to TOC
section_items = []
for section in chapter.sections:
section_items.append(section)
self.toc_items.append((chapter, section_items))
def _render_chapter(self, chapter: Chapter) -> str:
"""
Render chapter to XHTML.
Args:
chapter: Chapter object
Returns:
XHTML string
"""
html_parts = []
# Chapter title
if chapter.title:
html_parts.append(f'<h1 id="{chapter.anchor}">{self._escape_html(chapter.title)}</h1>')
# Chapter content
if chapter.content:
html_parts.append(self._render_content(chapter.content))
# Sections
for section in chapter.sections:
html_parts.append(f'<h{section.level} id="{section.anchor}">{self._escape_html(section.title)}</h{section.level}>')
if section.content:
html_parts.append(self._render_content(section.content))
# Ensure we have some content
content = '\n'.join(html_parts)
if not content.strip():
# Add empty paragraph if no content
content = '<p></p>'
# Wrap in proper XHTML document
xhtml = f"""<?xml version='1.0' encoding='utf-8'?>
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="{self.metadata.language}">
<head>
<title>{self._escape_html(chapter.title)}</title>
</head>
<body>
{content}
</body>
</html>"""
return xhtml
def _render_content(self, content: str) -> str:
"""
Render markdown content to HTML.
Args:
content: Markdown content
Returns:
HTML string
"""
if not content:
return ""
# Use markdown processor to convert to HTML
html = MarkdownProcessor.markdown_to_html(content)
return html
def _add_style(self) -> None:
"""Add CSS styling to EPUB."""
if not self.book:
return
style = epub.EpubItem()
style.file_name = 'style/main.css'
style.media_type = 'text/css'
style.set_content(self.DEFAULT_CSS)
self.book.add_item(style)
# Add CSS to all chapters
for item in self.book.items:
if isinstance(item, epub.EpubHtml):
item.add_link(
rel='stylesheet',
href='style/main.css',
type='text/css'
)
def _add_toc(self) -> None:
"""Add table of contents to EPUB."""
if not self.book or not self.toc_items:
return
toc_items = []
for chapter, sections in self.toc_items:
# Find chapter item in book
chapter_link = None
for item in self.book.items:
if isinstance(item, epub.EpubHtml) and item.title == chapter.title:
chapter_link = item
break
if chapter_link:
if sections:
# Chapter with sections - create nested structure
section_links = []
for section in sections:
# Add anchor to section headers in rendered HTML
section_links.append(
epub.Link(
f"{chapter_link.file_name}#{section.anchor}",
section.title,
f"sec_{section.anchor}"
)
)
# Add chapter with nested sections as tuple
toc_items.append((chapter_link, section_links))
else:
# Chapter without sections
toc_items.append(chapter_link)
if toc_items:
self.book.toc = tuple(toc_items)
def _write_epub(self, output_path: str) -> None:
"""
Write EPUB file to disk.
Args:
output_path: Path where EPUB should be saved
"""
if not self.book:
raise ValueError("Book not initialized")
# Ensure output directory exists
Path(output_path).parent.mkdir(parents=True, exist_ok=True)
# Add navigation items (must be added before setting spine)
ncx = epub.EpubNcx()
nav = epub.EpubNav()
self.book.add_item(ncx)
self.book.add_item(nav)
# Define spine - only include HTML items, nav is implicit
html_items = [item for item in self.book.items if isinstance(item, epub.EpubHtml)]
self.book.spine = html_items
# Write EPUB
epub.write_epub(output_path, self.book, {})
@staticmethod
def _escape_html(text: str) -> str:
"""Escape HTML special characters."""
if not text:
return ""
return (text
.replace('&', '&')
.replace('<', '<')
.replace('>', '>')
.replace('"', '"')
.replace("'", '''))
def create_epub_from_markdown(markdown_content: str, output_path: str,
title: Optional[str] = None,
author: Optional[str] = None) -> bool:
"""
Convenience function to create EPUB from markdown content.
Args:
markdown_content: Raw markdown text
output_path: Path where EPUB should be saved
title: Book title (optional, will use first H1 if not provided)
author: Author name (optional)
Returns:
True if successful, False otherwise
"""
# Parse markdown
processor = MarkdownProcessor()
result = processor.process(markdown_content)
# Create metadata
metadata = result['metadata']
if title:
metadata.title = title
if author:
metadata.author = author
# Generate EPUB
generator = EPUBGenerator(metadata)
return generator.generate(result['chapters'], output_path)
"""
Markdown processing module for converting markdown to EPUB-compatible structure.
This module handles:
- Parsing markdown into chapters and sections
- Converting markdown to HTML
- Extracting metadata and structure
- Building table of contents
"""
import re
from typing import List, Dict, Tuple, Optional
from dataclasses import dataclass
from enum import Enum
try:
from pygments import highlight
from pygments.lexers import get_lexer_by_name, guess_lexer, TextLexer
from pygments.formatters import HtmlFormatter
from pygments.util import ClassNotFound
PYGMENTS_AVAILABLE = True
except ImportError:
PYGMENTS_AVAILABLE = False
class HeaderLevel(Enum):
"""Header hierarchy levels."""
H1 = 1
H2 = 2
H3 = 3
H4 = 4
H5 = 5
H6 = 6
@dataclass
class Header:
"""Represents a markdown header."""
level: HeaderLevel
text: str
anchor: Optional[str] = None
def __post_init__(self):
if not self.anchor:
# Generate anchor from text
self.anchor = re.sub(r'[^\w\s-]', '', self.text.lower())
self.anchor = re.sub(r'[-\s]+', '-', self.anchor).strip('-')
@dataclass
class Chapter:
"""Represents a chapter (H1 section)."""
title: str
content: str
sections: List['Section']
anchor: Optional[str] = None
def __post_init__(self):
if not self.anchor:
self.anchor = re.sub(r'[^\w\s-]', '', self.title.lower())
self.anchor = re.sub(r'[-\s]+', '-', self.anchor).strip('-')
@dataclass
class Section:
"""Represents a section (H2-H6)."""
title: str
level: int
content: str
anchor: Optional[str] = None
def __post_init__(self):
if not self.anchor:
self.anchor = re.sub(r'[^\w\s-]', '', self.title.lower())
self.anchor = re.sub(r'[-\s]+', '-', self.anchor).strip('-')
@dataclass
class EbookMetadata:
"""Metadata for the ebook."""
title: Optional[str] = None
author: Optional[str] = "Unknown Author"
language: str = "en"
date: Optional[str] = None
identifier: Optional[str] = None
class MarkdownProcessor:
"""Process markdown content into EPUB-compatible structure."""
def __init__(self):
"""Initialize the markdown processor."""
self.chapters: List[Chapter] = []
self.metadata = EbookMetadata()
self.front_matter: Dict[str, str] = {}
def process(self, markdown_content: str) -> Dict:
"""
Process markdown content into structured format.
Args:
markdown_content: Raw markdown text
Returns:
Dictionary containing chapters, metadata, and TOC
"""
# Extract front matter if present
content = self._extract_frontmatter(markdown_content)
# Extract metadata from headers
self._extract_metadata(content)
# Parse content into chapters
self.chapters = self._parse_chapters(content)
# Build table of contents
toc = self._build_toc()
return {
'chapters': self.chapters,
'metadata': self.metadata,
'toc': toc,
'front_matter': self.front_matter
}
def _extract_frontmatter(self, content: str) -> str:
"""
Extract YAML front matter if present.
Args:
content: Raw markdown content
Returns:
Markdown content without front matter
"""
if not content.startswith('---'):
return content
# Find closing ---
try:
end_idx = content.index('---', 3)
fm_text = content[3:end_idx].strip()
# Parse simple key: value pairs
for line in fm_text.split('\n'):
if ':' in line:
key, value = line.split(':', 1)
self.front_matter[key.strip().lower()] = value.strip()
# Update metadata from front matter
if 'title' in self.front_matter:
self.metadata.title = self.front_matter['title']
if 'author' in self.front_matter:
self.metadata.author = self.front_matter['author']
if 'date' in self.front_matter:
self.metadata.date = self.front_matter['date']
if 'language' in self.front_matter:
self.metadata.language = self.front_matter['language']
return content[end_idx + 3:].lstrip()
except ValueError:
return content
def _extract_metadata(self, content: str) -> None:
"""
Extract metadata from document headers.
Args:
content: Markdown content
"""
# If no title from front matter, use first H1
if not self.metadata.title:
h1_match = re.search(r'^# (.+)$', content, re.MULTILINE)
if h1_match:
self.metadata.title = h1_match.group(1).strip()
def _parse_chapters(self, content: str) -> List[Chapter]:
"""
Parse markdown into chapters.
Chapters are delimited by H1 headers. Content between H1s
becomes a chapter with potential subsections (H2-H6).
Args:
content: Markdown content
Returns:
List of Chapter objects
"""
# First, remove code blocks to avoid matching headers inside them
# We'll use a placeholder to mark where code blocks were
code_blocks = []
code_block_pattern = r'```[^\n]*\n.*?```'
def save_code_block(match):
code_blocks.append(match.group(0))
return f'\n__CODE_BLOCK_{len(code_blocks)-1}__\n'
content_no_code = re.sub(code_block_pattern, save_code_block, content, flags=re.DOTALL)
# Split by H1 headers (now safe from code blocks)
h1_pattern = r'^# (.+)$'
chapters = []
# Find all H1 headers and their positions in content WITHOUT code blocks
h1_matches = list(re.finditer(h1_pattern, content_no_code, re.MULTILINE))
if not h1_matches:
# No chapters, treat entire content as single chapter
if content.strip():
sections = self._parse_sections(content, 2) # Start from H2
chapters.append(Chapter(
title="Untitled",
content="",
sections=sections
))
return chapters
# Process each chapter
# We need to map positions from content_no_code back to original content
for i, match in enumerate(h1_matches):
title = match.group(1).strip()
start_no_code = match.end()
# Find next H1 or end of content
if i + 1 < len(h1_matches):
end_no_code = h1_matches[i + 1].start()
else:
end_no_code = len(content_no_code)
# Get the chapter content from content_no_code
chapter_content_no_code = content_no_code[start_no_code:end_no_code]
# Restore code blocks
def restore_code_block(match):
idx = int(match.group(1))
return code_blocks[idx]
chapter_content = re.sub(r'__CODE_BLOCK_(\d+)__', restore_code_block, chapter_content_no_code).rstrip()
# Parse sections within this chapter
sections = self._parse_sections(chapter_content, 2)
# Extract direct content (before first H2)
direct_content = ""
if sections:
first_h2_pos = chapter_content.find('\n##')
if first_h2_pos == -1:
direct_content = chapter_content
else:
direct_content = chapter_content[:first_h2_pos].strip()
else:
direct_content = chapter_content
chapters.append(Chapter(
title=title,
content=direct_content,
sections=sections
))
return chapters
def _parse_sections(self, content: str, min_level: int = 2) -> List[Section]:
"""
Parse sections from content (H2 and below).
Args:
content: Markdown content
min_level: Minimum header level to parse (2-6)
Returns:
List of Section objects
"""
sections = []
# Build pattern for headers from min_level to 6
header_pattern = r'^(#{' + str(min_level) + r',6}) (.+)$'
matches = list(re.finditer(header_pattern, content, re.MULTILINE))
if not matches:
return sections
for i, match in enumerate(matches):
hashes = match.group(1)
level = len(hashes)
title = match.group(2).strip()
start = match.end()
# Find next header or end
if i + 1 < len(matches):
end = matches[i + 1].start()
else:
end = len(content)
section_content = content[start:end].rstrip()
sections.append(Section(
title=title,
level=level,
content=section_content
))
return sections
def _build_toc(self) -> List[Dict]:
"""
Build table of contents from chapters and sections.
Returns:
List of TOC entries with links and hierarchy
"""
toc = []
for chapter in self.chapters:
chapter_entry = {
'title': chapter.title,
'anchor': chapter.anchor,
'level': 1,
'subsections': []
}
for section in chapter.sections:
section_entry = {
'title': section.title,
'anchor': section.anchor,
'level': section.level
}
chapter_entry['subsections'].append(section_entry)
toc.append(chapter_entry)
return toc
@staticmethod
def markdown_to_html(markdown_text: str) -> str:
"""
Convert markdown to HTML.
This is a simplified converter for common markdown elements.
Args:
markdown_text: Markdown text
Returns:
HTML string
"""
if not markdown_text or not markdown_text.strip():
return "<p></p>"
lines = markdown_text.split('\n')
html_parts = []
in_code_block = False
code_block_content = []
code_block_language = None
in_table = False
table_lines = []
current_paragraph = []
for line in lines:
# Handle code blocks
if line.strip().startswith('```'):
if in_code_block:
# End code block
code_html = '\n'.join(code_block_content)
# Add line numbers and syntax highlighting
code_html = MarkdownProcessor._add_line_numbers_and_highlighting(
code_html, code_block_language
)
html_parts.append(code_html)
code_block_content = []
code_block_language = None
in_code_block = False
else:
# Start code block
if current_paragraph:
paragraph_text = ' '.join(current_paragraph).strip()
if paragraph_text:
html_parts.append(f'<p>{MarkdownProcessor._render_inline(paragraph_text)}</p>')
current_paragraph = []
# Extract language if specified
lang = line.strip()[3:].strip()
code_block_language = lang if lang else None
in_code_block = True
continue
if in_code_block:
code_block_content.append(line)
continue
# Handle tables
if '|' in line and line.strip().startswith('|') or (line.count('|') >= 2):
if not in_table:
# Start table
if current_paragraph:
paragraph_text = ' '.join(current_paragraph).strip()
if paragraph_text:
html_parts.append(f'<p>{MarkdownProcessor._render_inline(paragraph_text)}</p>')
current_paragraph = []
in_table = True
table_lines.append(line)
continue
elif in_table:
# End of table
table_html = MarkdownProcessor._parse_table(table_lines)
if table_html:
html_parts.append(table_html)
table_lines = []
in_table = False
# Fall through to process current line
# Handle blockquotes
if line.strip().startswith('>'):
if current_paragraph:
paragraph_text = ' '.join(current_paragraph).strip()
if paragraph_text:
html_parts.append(f'<p>{MarkdownProcessor._render_inline(paragraph_text)}</p>')
current_paragraph = []
quote_text = line.lstrip('> ').strip()
html_parts.append(f'<blockquote>{MarkdownProcessor._render_inline(quote_text)}</blockquote>')
continue
# Handle headers
if line.startswith('#'):
if current_paragraph:
paragraph_text = ' '.join(current_paragraph).strip()
if paragraph_text:
html_parts.append(f'<p>{MarkdownProcessor._render_inline(paragraph_text)}</p>')
current_paragraph = []
level = len(line) - len(line.lstrip('#'))
header_text = line.lstrip('# ').strip()
if level <= 6:
html_parts.append(f'<h{level}>{MarkdownProcessor._render_inline(header_text)}</h{level}>')
continue
# Handle lists
if line.strip().startswith(('- ', '* ')) or re.match(r'^\d+\.\s', line.strip()):
if current_paragraph:
paragraph_text = ' '.join(current_paragraph).strip()
if paragraph_text:
html_parts.append(f'<p>{MarkdownProcessor._render_inline(paragraph_text)}</p>')
current_paragraph = []
list_item = re.sub(r'^[-*\d.]\s+', '', line.strip())
html_parts.append(f'<li>{MarkdownProcessor._render_inline(list_item)}</li>')
continue
# Handle horizontal rules
if line.strip() in ('---', '***', '___'):
if current_paragraph:
paragraph_text = ' '.join(current_paragraph).strip()
if paragraph_text:
html_parts.append(f'<p>{MarkdownProcessor._render_inline(paragraph_text)}</p>')
current_paragraph = []
html_parts.append('<hr/>')
continue
# Handle empty lines (paragraph break)
if not line.strip():
if current_paragraph:
paragraph_text = ' '.join(current_paragraph).strip()
if paragraph_text:
html_parts.append(f'<p>{MarkdownProcessor._render_inline(paragraph_text)}</p>')
current_paragraph = []
continue
# Add to current paragraph
current_paragraph.append(line.strip())
# Finish any remaining paragraph
if current_paragraph:
paragraph_text = ' '.join(current_paragraph).strip()
if paragraph_text:
html_parts.append(f'<p>{MarkdownProcessor._render_inline(paragraph_text)}</p>')
# Finish any remaining table
if in_table and table_lines:
table_html = MarkdownProcessor._parse_table(table_lines)
if table_html:
html_parts.append(table_html)
html = '\n'.join(html_parts) if html_parts else '<p></p>'
return html
@staticmethod
def _render_inline(text: str) -> str:
"""
Render inline markdown elements (bold, italic, links, code).
Args:
text: Text with inline markdown
Returns:
HTML string
"""
# Escape HTML special characters first, but be careful with our markers
text = (text
.replace('&', '&')
.replace('<', '<')
.replace('>', '>'))
# Bold
text = re.sub(r'\*\*(.+?)\*\*', r'<strong>\1</strong>', text)
text = re.sub(r'__(.+?)__', r'<strong>\1</strong>', text)
# Italic
text = re.sub(r'\*(.+?)\*', r'<em>\1</em>', text)
text = re.sub(r'_(.+?)_', r'<em>\1</em>', text)
# Inline code
text = re.sub(r'`(.+?)`', r'<code>\1</code>', text)
# Links
text = re.sub(r'\[(.+?)\]\((.+?)\)', r'<a href="\2">\1</a>', text)
return text
@staticmethod
def _parse_table(table_lines: List[str]) -> str:
"""
Parse markdown table into HTML.
Args:
table_lines: List of table lines
Returns:
HTML table string
"""
if not table_lines or len(table_lines) < 2:
return ""
# Remove empty lines
table_lines = [line.strip() for line in table_lines if line.strip()]
if len(table_lines) < 2:
return ""
# Parse header
header_line = table_lines[0]
headers = [cell.strip() for cell in header_line.split('|')]
headers = [h for h in headers if h] # Remove empty cells
# Skip separator line (second line with dashes)
if len(table_lines) < 2:
return ""
# Check if second line is separator
separator = table_lines[1]
if not re.match(r'^[\s|:-]+$', separator):
# Not a valid table
return ""
# Parse data rows
rows = []
for line in table_lines[2:]:
cells = [cell.strip() for cell in line.split('|')]
cells = [c for c in cells if c] # Remove empty cells from leading/trailing pipes
if cells:
rows.append(cells)
# Build HTML table
html_parts = ['<table>']
# Header
html_parts.append('<thead><tr>')
for header in headers:
html_parts.append(f'<th>{MarkdownProcessor._render_inline(header)}</th>')
html_parts.append('</tr></thead>')
# Body
if rows:
html_parts.append('<tbody>')
for row in rows:
html_parts.append('<tr>')
for i, cell in enumerate(row):
if i < len(headers): # Don't exceed header count
html_parts.append(f'<td>{MarkdownProcessor._render_inline(cell)}</td>')
# Fill missing cells
for i in range(len(row), len(headers)):
html_parts.append('<td></td>')
html_parts.append('</tr>')
html_parts.append('</tbody>')
html_parts.append('</table>')
return '\n'.join(html_parts)
@staticmethod
def escape_html(text: str) -> str:
"""Escape HTML special characters."""
return (text
.replace('&', '&')
.replace('<', '<')
.replace('>', '>')
.replace('"', '"')
.replace("'", '''))
@staticmethod
def _add_line_numbers_and_highlighting(code: str, language: Optional[str] = None) -> str:
"""
Add line numbers and syntax highlighting to code block using Pygments.
Args:
code: Raw code string
language: Programming language identifier
Returns:
HTML string with line numbers and syntax highlighting
"""
if PYGMENTS_AVAILABLE and language:
try:
# Get the appropriate lexer for the language
lexer = get_lexer_by_name(language, stripall=False)
# Use Pygments to tokenize and highlight
from pygments import lex
from pygments.token import Token
lines = code.split('\n')
html_lines = []
for line_num, line in enumerate(lines, 1):
# Tokenize this line
tokens = list(lex(line + '\n', lexer))
highlighted_parts = []
for token_type, value in tokens:
# Skip the newline at the end
if value == '\n':
continue
# Escape HTML in the value
value = (value
.replace('&', '&')
.replace('<', '<')
.replace('>', '>'))
# Map Pygments token types to our CSS classes
css_class = MarkdownProcessor._get_token_css_class(token_type)
if css_class:
highlighted_parts.append(f'<span class="{css_class}">{value}</span>')
else:
highlighted_parts.append(value)
# Add line number and content
html_lines.append(
f'<span class="line-number">{line_num:3d}</span>'
f'<span class="line-content">{"".join(highlighted_parts)}\n</span>'
)
# Build final HTML
safe_lang = re.sub(r'[^a-zA-Z0-9_-]', '', language)
lang_class = f' language-{safe_lang}' if safe_lang else ''
return f'<pre class="code-block{lang_class}"><code>{"".join(html_lines)}</code></pre>'
except ClassNotFound:
# Language not found, fall back to plain text
pass
except Exception:
# Any other error, fall back to plain text
pass
# Fallback: no syntax highlighting, just line numbers and escaping
code = (code
.replace('&', '&')
.replace('<', '<')
.replace('>', '>'))
lines = code.split('\n')
html_lines = []
for i, line in enumerate(lines, 1):
html_lines.append(
f'<span class="line-number">{i:3d}</span>'
f'<span class="line-content">{line}\n</span>'
)
# Build final HTML
if language:
safe_lang = re.sub(r'[^a-zA-Z0-9_-]', '', language)
lang_class = f' language-{safe_lang}' if safe_lang else ''
else:
lang_class = ''
return f'<pre class="code-block{lang_class}"><code>{"".join(html_lines)}</code></pre>'
@staticmethod
def _get_token_css_class(token_type) -> Optional[str]:
"""
Map Pygments token types to our CSS classes.
Args:
token_type: Pygments token type
Returns:
CSS class name or None
"""
from pygments.token import Token
# Map token types to our simpler CSS classes
if token_type in Token.Keyword:
return 'syn-keyword'
elif token_type in Token.String:
return 'syn-string'
elif token_type in Token.Comment:
return 'syn-comment'
elif token_type in Token.Number:
return 'syn-number'
elif token_type in Token.Name.Function:
return 'syn-function'
elif token_type in Token.Name.Class:
return 'syn-class'
else:
return None