
Pdf Processing
- 13 installs
- Updated November 18, 2025
- wesley1600/claudecodeframework
Helps with ai & agent building tasks.
About
pdf-processing is a Claude Code skill for ai & agent building. It helps solo builders move faster with AI-assisted coding.
- pdf-processing
- AI & Agent Building
- AI-coding skill
Pdf Processing by the numbers
- 13 all-time installs (skills.sh)
- Ranked #11,409 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
- Data as of Jul 27, 2026 (Skillselion catalog sync)
npx skills add https://github.com/wesley1600/claudecodeframework --skill pdf-processingAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 13 |
|---|---|
| Last updated | November 18, 2025 |
| Repository | wesley1600/claudecodeframework ↗ |
What it does
Helps with ai & agent building tasks.
Files
PDF Processing Skill
Overview
This skill provides comprehensive PDF processing capabilities leveraging Claude's native PDF support, which can:
- Extract and parse text content
- Identify and extract tables with structure preservation
- Analyze charts, graphs, and visual elements
- Understand document layout and formatting
- Generate summaries and insights
- Convert PDFs to various formats (Markdown, JSON, CSV, plain text)
When to Use
Claude should automatically activate this skill when:
- User provides a PDF file path or wants to process a PDF
- User asks to extract text, tables, or data from PDFs
- User requests PDF analysis, summarization, or conversion
- User needs to understand charts, diagrams, or visual content in PDFs
- User wants to transform PDF content to another format
Key Capabilities
1. Text Extraction
- Full document text extraction with formatting preservation
- Page-by-page text extraction
- Section and paragraph identification
- Header and footer detection
2. Table Extraction
- Automatic table detection and extraction
- Structure-preserving conversion to CSV, JSON, or Markdown
- Multi-page table handling
- Cell merging and complex table support
3. Visual Content Analysis
- Chart and graph interpretation
- Diagram and flowchart understanding
- Image and figure description
- Infographic analysis
4. Document Understanding
- Layout analysis and structure detection
- Multi-column text handling
- Form field identification
- Metadata extraction (title, author, creation date, page count)
5. Format Conversion
- PDF to Markdown (preserving headings, lists, tables)
- PDF to JSON (structured data extraction)
- PDF to CSV (table extraction)
- PDF to plain text
Instructions
Step 1: Validate PDF Input
First, determine how the PDF is provided:
Option A: File Path
# Verify the file exists and is a PDF
ls -lh /path/to/document.pdf
file /path/to/document.pdfOption B: Base64-Encoded PDF If the user provides base64-encoded content, save it first:
# Decode and save base64 PDF
python .claude/skills/pdf-processing/scripts/decode_pdf.py --input base64_string.txt --output document.pdfOption C: URL If the user provides a URL, download it:
# Download PDF from URL
python .claude/skills/pdf-processing/scripts/download_pdf.py --url "https://example.com/doc.pdf" --output document.pdfStep 2: Read and Analyze PDF
Use the Read tool to access PDF files. Claude's native PDF support will:
- Display the PDF content visually
- Extract text and structure automatically
- Identify tables, charts, and images
# Example: Read PDF using the Read tool
# The Read tool handles PDFs natively and extracts text + visual content
Read(file_path="/absolute/path/to/document.pdf")Step 3: Process Based on User Request
A. Text Extraction
For simple text extraction:
python .claude/skills/pdf-processing/scripts/extract_text.py \
--input document.pdf \
--output document.txt \
--preserve-formatting trueFor page-specific extraction:
python .claude/skills/pdf-processing/scripts/extract_text.py \
--input document.pdf \
--pages 1,3,5-10 \
--output selected_pages.txtB. Table Extraction
Extract all tables to CSV:
python .claude/skills/pdf-processing/scripts/extract_tables.py \
--input document.pdf \
--format csv \
--output-dir ./extracted_tables/Extract tables to JSON with structure:
python .claude/skills/pdf-processing/scripts/extract_tables.py \
--input document.pdf \
--format json \
--output tables.jsonC. Document Summarization
Generate a summary of the PDF:
python .claude/skills/pdf-processing/scripts/summarize_pdf.py \
--input document.pdf \
--output summary.md \
--style concise # Options: concise, detailed, executiveD. Format Conversion
Convert PDF to Markdown:
python .claude/skills/pdf-processing/scripts/convert_pdf.py \
--input document.pdf \
--output document.md \
--format markdown \
--preserve-images trueConvert PDF to structured JSON:
python .claude/skills/pdf-processing/scripts/convert_pdf.py \
--input document.pdf \
--output document.json \
--format json \
--extract-metadata trueE. Visual Content Analysis
Analyze charts and graphs:
python .claude/skills/pdf-processing/scripts/analyze_visuals.py \
--input document.pdf \
--output analysis.json \
--elements charts,graphs,diagramsStep 4: Post-Processing and Output
After extraction/conversion:
1. Validate Output: Check that the output file was created successfully
ls -lh output_file.{txt,md,json,csv}2. Preview Results: Show the user a preview of the extracted content
head -n 20 output_file.txt # For text files
cat output_file.json | python -m json.tool | head -n 50 # For JSON3. Provide Summary: Summarize what was extracted and offer next steps
Step 5: Handle Edge Cases
Password-Protected PDFs
python .claude/skills/pdf-processing/scripts/extract_text.py \
--input document.pdf \
--password "user_provided_password" \
--output document.txtScanned PDFs (OCR Required)
# Use OCR for scanned PDFs
python .claude/skills/pdf-processing/scripts/ocr_pdf.py \
--input scanned_document.pdf \
--output document.txt \
--language eng # Language code: eng, fra, deu, etc.Large PDFs (Memory Optimization)
# Process large PDFs in chunks
python .claude/skills/pdf-processing/scripts/extract_text.py \
--input large_document.pdf \
--output document.txt \
--chunk-size 10 # Process 10 pages at a timeError Handling
Common Issues
1. File Not Found
- Verify the path with
lsorGlob - Check for typos in the filename
- Ensure absolute paths are used
2. Corrupted PDF
- Try reading with the Read tool first
- Use repair mode:
python scripts/repair_pdf.py --input corrupted.pdf --output repaired.pdf
3. Unsupported PDF Features
- Some PDFs with complex DRM or encryption may fail
- Inform the user and suggest alternatives
4. OCR Failures
- Check if tesseract is installed:
which tesseract - Verify image quality is sufficient
- Try different language settings
Best Practices
1. Always use the Read tool first - This leverages Claude's native PDF support for best results 2. Preserve structure - When extracting tables or converting formats, maintain the original structure 3. Validate outputs - Always check that output files were created successfully 4. Provide context - Tell the user what was extracted and what they can do next 5. Handle errors gracefully - If processing fails, explain why and suggest alternatives 6. Respect privacy - Remind users not to upload sensitive documents without proper authorization
Output Formats
Text Output
Plain text with optional formatting preservation
Line breaks and paragraphs maintained
Special characters preservedMarkdown Output
# Document Title
## Section Heading
Paragraph text with **bold** and *italic* formatting.
| Column 1 | Column 2 | Column 3 |
|----------|----------|----------|
| Data 1 | Data 2 | Data 3 |
JSON Output
{
"metadata": {
"title": "Document Title",
"author": "Author Name",
"pages": 42,
"creation_date": "2024-01-15"
},
"content": [
{
"page": 1,
"type": "text",
"content": "Page 1 text content..."
},
{
"page": 2,
"type": "table",
"headers": ["Col1", "Col2", "Col3"],
"rows": [["A", "B", "C"], ["D", "E", "F"]]
}
]
}CSV Output (for tables)
Column1,Column2,Column3
Value1,Value2,Value3
Value4,Value5,Value6Advanced Features
Batch Processing
# Process multiple PDFs
python .claude/skills/pdf-processing/scripts/batch_process.py \
--input-dir ./pdfs/ \
--output-dir ./extracted/ \
--format markdownCustom Templates
# Use custom conversion templates
python .claude/skills/pdf-processing/scripts/convert_pdf.py \
--input document.pdf \
--output document.md \
--template .claude/skills/pdf-processing/assets/custom_template.mdSelective Extraction
# Extract only specific sections
python .claude/skills/pdf-processing/scripts/extract_sections.py \
--input document.pdf \
--sections "Introduction,Methods,Results" \
--output extracted_sections.mdIntegration with Other Tools
This skill works well with:
- Data analysis tools - Extract tables and feed to pandas/numpy
- Documentation generators - Convert PDFs to Markdown for wikis
- Search systems - Extract text for indexing
- Automation workflows - Batch process invoices, reports, forms
Examples
Example 1: Extract and Summarize
# User: "Please extract the key points from this research paper"
1. Read(file_path="/path/to/paper.pdf")
2. python scripts/summarize_pdf.py --input paper.pdf --output summary.md --style executive
3. Show the user the summary with key findings highlightedExample 2: Extract Tables to CSV
# User: "Get all tables from this financial report"
1. Read(file_path="/path/to/report.pdf")
2. python scripts/extract_tables.py --input report.pdf --format csv --output-dir ./tables/
3. List the extracted CSV files and preview the first tableExample 3: Convert to Markdown
# User: "Convert this PDF to markdown"
1. Read(file_path="/path/to/document.pdf")
2. python scripts/convert_pdf.py --input document.pdf --output document.md --format markdown
3. Show preview of the markdown and confirm successful conversionDependencies
The scripts in this skill require:
- Python 3.8+
- PyPDF2 or pypdf (PDF parsing)
- pdfplumber (table extraction)
- pdf2image (image extraction)
- pytesseract (OCR for scanned PDFs)
- Pillow (image processing)
- requests (URL downloads)
These are installed via the requirements file in assets/requirements.txt.
References
See the references/ directory for:
pdf_capabilities.md- Detailed breakdown of Claude's PDF supportapi_reference.md- Complete API documentation for all scriptsexamples.md- More usage examples and use casestroubleshooting.md- Common issues and solutions
Notes for Claude
- Always read PDFs with the Read tool first - This is the most reliable method
- After reading, analyze what the user needs - Text, tables, summary, conversion?
- Use the appropriate script - Don't try to do everything manually
- Validate outputs - Always check that files were created successfully
- Provide helpful context - Explain what was extracted and suggest next steps
- Handle errors gracefully - If something fails, explain why and offer alternatives
- Be efficient - Use batch processing for multiple PDFs
- Preserve structure - Maintain document formatting when converting
Version History
- 1.0.0 (2025-11-18) - Initial release with core PDF processing capabilities
# PDF Processing Skill - Python Dependencies
# Core PDF libraries
pypdf>=3.17.0 # Modern PDF parsing (PyPDF2 successor)
pdfplumber>=0.10.0 # Advanced table extraction and layout analysis
pdf2image>=1.16.0 # Convert PDF pages to images
# OCR support (optional but recommended)
pytesseract>=0.3.10 # Python wrapper for Tesseract OCR
Pillow>=10.0.0 # Image processing
# Utilities
requests>=2.31.0 # HTTP library for downloading PDFs
# Optional: Enhanced text processing
# python-magic>=0.4.27 # File type detection
# tabula-py>=2.8.0 # Alternative table extraction (requires Java)
PDF Processing Skill
A comprehensive Claude Code skill for processing, analyzing, and converting PDF documents.
Overview
This skill provides Claude with powerful PDF processing capabilities including:
- Text Extraction - Extract text from PDFs with formatting preservation
- Table Extraction - Extract tables and convert to CSV, JSON, or Markdown
- Document Conversion - Convert PDFs to Markdown, JSON, or plain text
- Summarization - Generate summaries with different styles
- Visual Analysis - Analyze charts, graphs, and visual content via Claude's native PDF support
- Utility Functions - Download PDFs from URLs, decode base64-encoded PDFs
Quick Start
Installation
1. Install required Python packages:
cd .claude/skills/pdf-processing
pip install -r assets/requirements.txt2. (Optional) Install Tesseract for OCR support:
# Ubuntu/Debian
sudo apt-get install tesseract-ocr
# macOS
brew install tesseract
# Windows
# Download from: https://github.com/UB-Mannheim/tesseract/wikiUsage
The skill activates automatically when working with PDFs. Claude will use the appropriate scripts based on your request.
Example requests:
- "Extract text from this PDF"
- "Get all tables from this financial report"
- "Convert this PDF to Markdown"
- "Summarize this research paper"
- "Download and process this PDF: https://example.com/doc.pdf"
Directory Structure
pdf-processing/
├── SKILL.md # Main skill definition (read by Claude)
├── README.md # This file
├── scripts/ # Python scripts for PDF processing
│ ├── extract_text.py # Text extraction
│ ├── extract_tables.py # Table extraction
│ ├── convert_pdf.py # Format conversion
│ ├── summarize_pdf.py # Document summarization
│ ├── download_pdf.py # Download PDFs from URLs
│ └── decode_pdf.py # Decode base64-encoded PDFs
├── references/ # Documentation
│ ├── pdf_capabilities.md # Claude's PDF capabilities
│ ├── api_reference.md # Complete API documentation
│ └── examples.md # Real-world examples
└── assets/ # Supporting files
└── requirements.txt # Python dependenciesKey Features
1. Native PDF Reading
Claude can read PDFs directly using the Read tool, which provides:
- Visual content understanding
- Text extraction
- Table recognition
- Chart and graph analysis
- Document structure analysis
2. Advanced Text Extraction
# Extract all text
python scripts/extract_text.py --input document.pdf --output document.txt
# Extract specific pages
python scripts/extract_text.py --input document.pdf --pages "1,5-10" --output text.txt
# Handle password-protected PDFs
python scripts/extract_text.py --input secure.pdf --password "secret" --output text.txt3. Table Extraction
# Extract to CSV
python scripts/extract_tables.py --input report.pdf --format csv --output-dir ./tables/
# Extract to JSON
python scripts/extract_tables.py --input report.pdf --format json --output tables.json
# Extract to Markdown
python scripts/extract_tables.py --input report.pdf --format markdown --output-dir ./tables/4. Format Conversion
# Convert to Markdown
python scripts/convert_pdf.py --input doc.pdf --output doc.md --format markdown
# Convert to JSON with metadata
python scripts/convert_pdf.py --input doc.pdf --output doc.json --format json --extract-metadata
# Convert to text
python scripts/convert_pdf.py --input doc.pdf --output doc.txt --format text5. Document Summarization
# Concise summary
python scripts/summarize_pdf.py --input paper.pdf --output summary.md --style concise
# Detailed summary
python scripts/summarize_pdf.py --input paper.pdf --output summary.md --style detailed
# Executive summary
python scripts/summarize_pdf.py --input paper.pdf --output summary.md --style executiveCommon Use Cases
Academic Research
- Extract key findings from research papers
- Extract data tables for analysis
- Generate literature review summaries
Financial Analysis
- Extract financial tables from reports
- Analyze quarterly statements
- Process invoices and receipts
Legal Documents
- Extract specific clauses from contracts
- Search across multiple documents
- Convert to searchable formats
Technical Documentation
- Convert PDF manuals to Markdown
- Extract API references
- Build searchable documentation
Dependencies
Required Python packages (see assets/requirements.txt):
pypdf- PDF parsingpdfplumber- Advanced table extractionpdf2image- Image extractionpytesseract- OCR supportPillow- Image processingrequests- URL downloads
Documentation
- [SKILL.md](SKILL.md) - Complete skill instructions for Claude
- [PDF Capabilities](references/pdf_capabilities.md) - Detailed breakdown of features
- [API Reference](references/api_reference.md) - Complete script documentation
- [Examples](references/examples.md) - Real-world use cases and workflows
Troubleshooting
Common Issues
Problem: "Module not found" errors
# Solution: Install dependencies
pip install -r assets/requirements.txtProblem: Text extraction from scanned PDFs fails
# Solution: Use OCR
# 1. Install tesseract (see Installation above)
# 2. The skill will automatically detect scanned PDFsProblem: Tables not detected
# Solution: Try different extraction settings or manual verification
# Some complex tables may need manual adjustmentProblem: Password-protected PDFs
# Solution: Provide password
python scripts/extract_text.py --input secure.pdf --password "yourpassword" --output text.txtPerformance
| PDF Type | Pages | Processing Time | Memory Usage |
|---|---|---|---|
| Simple text | 1-50 | <5 seconds | Low |
| Simple text | 51-200 | 5-30 seconds | Medium |
| Complex layout | 1-50 | 10-60 seconds | Medium |
| Scanned (OCR) | 1-50 | 1-10 minutes | High |
Best Practices
1. Always read PDFs with Claude's Read tool first - This provides the best understanding of content and structure 2. Use appropriate extraction method - Different PDFs require different approaches 3. Validate outputs - Always verify extracted data is accurate 4. Handle errors gracefully - Check for password protection, corruption, etc. 5. Respect privacy - Don't process sensitive documents without authorization
Version History
- 1.0.0 (2025-11-18) - Initial release
- Text extraction
- Table extraction
- Format conversion (Markdown, JSON, text)
- Document summarization
- PDF download from URLs
- Base64 PDF decoding
- Comprehensive documentation
Contributing
This skill is part of the ClaudeCodeFrameWork project. For issues or improvements, please refer to the main project repository.
License
This skill follows the license of the parent ClaudeCodeFrameWork project.
Support
For help with this skill: 1. Check the API Reference for detailed usage 2. Review Examples for common use cases 3. Refer to PDF Capabilities for technical details 4. Ask Claude for help - the skill is designed to work seamlessly with Claude's assistance
---
Note: This skill leverages Claude's native PDF processing capabilities combined with powerful Python libraries for comprehensive PDF handling.
PDF Processing Scripts - API Reference
Complete reference for all PDF processing scripts in this skill.
Table of Contents
1. extract_text.py 2. extract_tables.py 3. convert_pdf.py 4. summarize_pdf.py 5. download_pdf.py 6. decode_pdf.py
---
extract_text.py
Extract text content from PDF files with optional formatting preservation.
Usage
python extract_text.py --input INPUT_PDF [OPTIONS]Arguments
| Argument | Short | Type | Required | Default | Description |
|---|---|---|---|---|---|
--input | -i | Path | Yes | - | Input PDF file path |
--output | -o | Path | No | stdout | Output text file path |
--pages | -p | String | No | all | Pages to extract (e.g., "1,3,5-10") |
--password | - | String | No | - | Password for encrypted PDFs |
--preserve-formatting | - | Boolean | No | true | Preserve formatting with page markers |
--chunk-size | - | Integer | No | - | Process in chunks of N pages |
Examples
# Extract all text
python extract_text.py --input document.pdf --output document.txt
# Extract specific pages
python extract_text.py --input document.pdf --pages "1,5-10,15" --output selected.txt
# Extract from encrypted PDF
python extract_text.py --input secure.pdf --password "secret123" --output document.txt
# Process large PDF in chunks
python extract_text.py --input large.pdf --chunk-size 10 --output document.txt
# Print to stdout
python extract_text.py --input document.pdfReturn Codes
0- Success1- Error (file not found, processing failed, etc.)
Output Format
================================================================================
PAGE 1
================================================================================
[Page 1 text content...]
================================================================================
PAGE 2
================================================================================
[Page 2 text content...]---
extract_tables.py
Extract tables from PDF files and convert to CSV, JSON, or Markdown.
Usage
python extract_tables.py --input INPUT_PDF [OPTIONS]Arguments
| Argument | Short | Type | Required | Default | Description |
|---|---|---|---|---|---|
--input | -i | Path | Yes | - | Input PDF file path |
--format | -f | String | No | csv | Output format (csv, json, markdown) |
--output-dir | -d | Path | No | - | Directory for individual table files |
--output | -o | Path | No | - | Single output file (JSON only) |
Examples
# Extract all tables to CSV
python extract_tables.py --input document.pdf --format csv --output-dir ./tables/
# Extract to JSON
python extract_tables.py --input document.pdf --format json --output tables.json
# Extract to Markdown
python extract_tables.py --input document.pdf --format markdown --output-dir ./tables/Output Files
CSV Format:
table_1_page_2.csvtable_2_page_5.csv- etc.
JSON Format (single file):
[
{
"page": 2,
"table_number": 1,
"headers": ["Column 1", "Column 2", "Column 3"],
"rows": [
["Data 1", "Data 2", "Data 3"],
["Data 4", "Data 5", "Data 6"]
]
}
]Markdown Format:
# Table 1 (Page 2)
| Column 1 | Column 2 | Column 3 |
|---|---|---|
| Data 1 | Data 2 | Data 3 |
| Data 4 | Data 5 | Data 6 |Return Codes
0- Success1- Error
---
convert_pdf.py
Convert PDF files to various formats (Markdown, JSON, plain text).
Usage
python convert_pdf.py --input INPUT_PDF --output OUTPUT_FILE [OPTIONS]Arguments
| Argument | Short | Type | Required | Default | Description |
|---|---|---|---|---|---|
--input | -i | Path | Yes | - | Input PDF file path |
--output | -o | Path | Yes | - | Output file path |
--format | -f | String | No | markdown | Output format (markdown, json, text) |
--preserve-images | - | Flag | No | false | Preserve and extract images (Markdown only) |
--extract-metadata | - | Flag | No | false | Include PDF metadata in output |
Examples
# Convert to Markdown
python convert_pdf.py --input document.pdf --output document.md --format markdown
# Convert to JSON with metadata
python convert_pdf.py --input document.pdf --output document.json --format json --extract-metadata
# Convert to text
python convert_pdf.py --input document.pdf --output document.txt --format text
# Markdown with images and metadata
python convert_pdf.py --input document.pdf --output document.md --format markdown --preserve-images --extract-metadataOutput Formats
Markdown:
---
title: Document Title
author: Author Name
pages: 42
---
## Page 1
Document content...
### Table 1
| Col1 | Col2 |
|------|------|
| A | B |JSON:
{
"source": "/path/to/document.pdf",
"metadata": {
"title": "Document Title",
"pages": 42
},
"content": [
{
"page": 1,
"type": "text",
"content": "..."
},
{
"page": 2,
"type": "table",
"headers": ["Col1", "Col2"],
"rows": [["A", "B"]]
}
]
}Return Codes
0- Success1- Error
---
summarize_pdf.py
Summarize PDF content by extracting key information.
Usage
python summarize_pdf.py --input INPUT_PDF --output OUTPUT_FILE [OPTIONS]Arguments
| Argument | Short | Type | Required | Default | Description |
|---|---|---|---|---|---|
--input | -i | Path | Yes | - | Input PDF file path |
--output | -o | Path | Yes | - | Output summary file (Markdown) |
--style | -s | String | No | concise | Summary style (concise, detailed, executive) |
Summary Styles
| Style | Description | Length | Use Case |
|---|---|---|---|
concise | Brief overview with key points | Short | Quick review |
detailed | Comprehensive summary with excerpts | Medium | Thorough understanding |
executive | Executive-level summary with highlights | Medium | Decision-making |
Examples
# Concise summary
python summarize_pdf.py --input document.pdf --output summary.md --style concise
# Detailed summary
python summarize_pdf.py --input document.pdf --output summary.md --style detailed
# Executive summary
python summarize_pdf.py --input document.pdf --output summary.md --style executiveOutput Format
# PDF Document Summary
**Source:** document.pdf
**Generated:** summarize_pdf.py
## Document Information
- **Title:** Document Title
- **Author:** Author Name
- **Pages:** 42
- **Words:** ~10,500
- **Tables:** 5
## Key Content
1. First key point extracted from document.
2. Second key point with important information.
...
---
*This summary was automatically generated.*Return Codes
0- Success1- Error
---
download_pdf.py
Download PDF files from URLs.
Usage
python download_pdf.py --url URL --output OUTPUT_FILE [OPTIONS]Arguments
| Argument | Short | Type | Required | Default | Description |
|---|---|---|---|---|---|
--url | -u | String | Yes | - | URL of the PDF file |
--output | -o | Path | Yes | - | Output file path |
--no-verify-ssl | - | Flag | No | false | Disable SSL verification |
Examples
# Download PDF
python download_pdf.py --url "https://example.com/document.pdf" --output document.pdf
# Download without SSL verification (use with caution)
python download_pdf.py --url "https://example.com/doc.pdf" --output doc.pdf --no-verify-sslOutput
Downloading PDF from: https://example.com/document.pdf
Progress: 100.0%
PDF downloaded successfully: document.pdf
File size: 2543.2 KBReturn Codes
0- Success1- Error (network error, invalid URL, etc.)
---
decode_pdf.py
Decode base64-encoded PDF content.
Usage
python decode_pdf.py --output OUTPUT_FILE [INPUT_OPTIONS]Arguments
| Argument | Short | Type | Required | Default | Description |
|---|---|---|---|---|---|
--input | -i | Path | No* | - | Input file with base64 content |
--string | -s | String | No* | - | Base64 string directly |
--output | -o | Path | Yes | - | Output PDF file path |
\* Either --input or --string must be provided
Examples
# Decode from file
python decode_pdf.py --input base64_content.txt --output document.pdf
# Decode from string
python decode_pdf.py --string "JVBERi0xLjQKJeLjz9MKM..." --output document.pdf
# Decode data URI
python decode_pdf.py --string "data:application/pdf;base64,JVBERi..." --output document.pdfInput Format
The script accepts:
- Plain base64 string
- Data URI format:
data:application/pdf;base64,<base64-data> - File containing base64 content
Output
Decoding base64 content...
PDF decoded successfully: document.pdf
File size: 1234.5 KBReturn Codes
0- Success1- Error (invalid base64, not a PDF, etc.)
---
Common Error Handling
All scripts follow consistent error handling:
File Not Found
Error: PDF file not found: /path/to/document.pdf
Exit code: 1Password Protected
Error: PDF is password-protected. Please provide password with --password
Exit code: 1Invalid PDF
Warning: Decoded data doesn't appear to be a valid PDFProcessing Error
Error: Failed to extract text: [detailed error message]
Exit code: 1Integration with Claude
All scripts are designed to work seamlessly with Claude's workflow:
1. Read PDF first - Use Claude's Read tool to analyze the PDF 2. Determine needs - Claude identifies what extraction is needed 3. Execute script - Run appropriate script with correct parameters 4. Validate output - Check that files were created successfully 5. Present results - Show user what was extracted
Example Integration
# Step 1: Claude reads PDF
Read(file_path="/path/to/document.pdf")
# Step 2: Claude determines user wants tables
# User said: "Extract all tables from this PDF"
# Step 3: Execute extraction
Bash: python .claude/skills/pdf-processing/scripts/extract_tables.py \
--input /path/to/document.pdf \
--format csv \
--output-dir ./extracted_tables/
# Step 4: Validate
Bash: ls -la ./extracted_tables/
# Step 5: Present results
Read: ./extracted_tables/table_1_page_2.csvVersion History
- 1.0.0 (2025-11-18) - Initial API documentation
PDF Processing Examples
Real-world examples and use cases for the PDF processing skill.
Table of Contents
1. Academic Research 2. Financial Reports 3. Legal Documents 4. Technical Documentation 5. Form Processing 6. Invoice Extraction 7. Batch Processing 8. Advanced Workflows
---
Academic Research
Use Case: Extract Key Findings from Research Paper
Scenario: Researcher wants to quickly extract key findings from a 50-page academic paper.
Workflow:
# Step 1: Read PDF to understand structure
Read(file_path="/path/to/research_paper.pdf")
# Step 2: Generate executive summary
python scripts/summarize_pdf.py \
--input /path/to/research_paper.pdf \
--output paper_summary.md \
--style executive
# Step 3: Extract tables with statistical data
python scripts/extract_tables.py \
--input /path/to/research_paper.pdf \
--format csv \
--output-dir ./paper_tables/
# Step 4: Convert references section to text
python scripts/extract_text.py \
--input /path/to/research_paper.pdf \
--pages "48-50" \
--output references.txtOutput:
paper_summary.md- Executive summary with key findings./paper_tables/- CSV files with statistical datareferences.txt- Bibliography for citation
---
Financial Reports
Use Case: Analyze Quarterly Financial Statement
Scenario: Analyst needs to extract financial tables and key metrics from quarterly report.
Workflow:
# Step 1: Read PDF
Read(file_path="/path/to/Q4_2024_Financial_Report.pdf")
# Step 2: Extract all financial tables
python scripts/extract_tables.py \
--input /path/to/Q4_2024_Financial_Report.pdf \
--format json \
--output financial_data.json
# Step 3: Extract key sections
python scripts/extract_text.py \
--input /path/to/Q4_2024_Financial_Report.pdf \
--pages "3-5,10-12" \
--output key_sections.txt
# Step 4: Generate summary
python scripts/summarize_pdf.py \
--input /path/to/Q4_2024_Financial_Report.pdf \
--output report_summary.md \
--style executiveFollow-up Analysis:
import json
import pandas as pd
# Load extracted tables
with open('financial_data.json') as f:
tables = json.load(f)
# Convert to DataFrame for analysis
for table in tables:
if table['type'] == 'table':
df = pd.DataFrame(table['rows'], columns=table['headers'])
# Perform financial analysis
print(df.describe())---
Legal Documents
Use Case: Extract Clauses from Contract
Scenario: Lawyer needs to extract specific clauses from a 100-page contract.
Workflow:
# Step 1: Read PDF
Read(file_path="/path/to/contract.pdf")
# Step 2: Convert to searchable format
python scripts/convert_pdf.py \
--input /path/to/contract.pdf \
--output contract.md \
--format markdown \
--extract-metadata
# Step 3: Extract specific pages (identified after reading)
python scripts/extract_text.py \
--input /path/to/contract.pdf \
--pages "15-20,45-50,80-85" \
--output key_clauses.txtThen search in Markdown:
# Search for specific terms
grep -i "liability" contract.md
grep -i "termination" contract.md
grep -i "indemnification" contract.md---
Technical Documentation
Use Case: Convert PDF Manual to Markdown Documentation
Scenario: Developer wants to convert PDF user manual to Markdown for a wiki.
Workflow:
# Step 1: Read PDF
Read(file_path="/path/to/user_manual.pdf")
# Step 2: Convert to Markdown with metadata
python scripts/convert_pdf.py \
--input /path/to/user_manual.pdf \
--output user_manual.md \
--format markdown \
--preserve-images \
--extract-metadata
# Step 3: Extract tables separately for review
python scripts/extract_tables.py \
--input /path/to/user_manual.pdf \
--format markdown \
--output-dir ./manual_tables/
# Step 4: Generate TOC summary
python scripts/summarize_pdf.py \
--input /path/to/user_manual.pdf \
--output manual_overview.md \
--style detailedPost-Processing:
# Clean up Markdown formatting
# Add to version control
git add user_manual.md manual_tables/ manual_overview.md
git commit -m "Add converted user manual"---
Form Processing
Use Case: Extract Data from Filled Forms
Scenario: Process hundreds of filled PDF forms and extract data to database.
Workflow:
# Step 1: Process single form to understand structure
Read(file_path="/path/to/forms/form_001.pdf")
# Step 2: Extract to structured JSON
python scripts/convert_pdf.py \
--input /path/to/forms/form_001.pdf \
--output form_001.json \
--format json \
--extract-metadata
# Step 3: Extract any tables (for multi-entry forms)
python scripts/extract_tables.py \
--input /path/to/forms/form_001.pdf \
--format json \
--output form_001_tables.jsonBatch Processing:
# Process all forms
for form in /path/to/forms/*.pdf; do
base_name=$(basename "$form" .pdf)
python scripts/convert_pdf.py \
--input "$form" \
--output "processed/${base_name}.json" \
--format json
done---
Invoice Extraction
Use Case: Extract Invoice Data for Accounting
Scenario: Accountant needs to extract invoice details from PDF invoices.
Workflow:
# Step 1: Read sample invoice
Read(file_path="/path/to/invoices/invoice_2024_001.pdf")
# Step 2: Extract text to identify key fields
python scripts/extract_text.py \
--input /path/to/invoices/invoice_2024_001.pdf \
--output invoice_text.txt
# Step 3: Extract invoice table (line items)
python scripts/extract_tables.py \
--input /path/to/invoices/invoice_2024_001.pdf \
--format csv \
--output-dir ./invoice_data/Parse Invoice Fields:
import re
with open('invoice_text.txt') as f:
text = f.read()
# Extract invoice number
invoice_num = re.search(r'Invoice #(\d+)', text)
# Extract date
date = re.search(r'Date: (\d{2}/\d{2}/\d{4})', text)
# Extract total
total = re.search(r'Total: \$([0-9,]+\.\d{2})', text)
invoice_data = {
'invoice_number': invoice_num.group(1) if invoice_num else None,
'date': date.group(1) if date else None,
'total': total.group(1) if total else None
}---
Batch Processing
Use Case: Process Multiple PDFs in Bulk
Scenario: Process 100 PDF documents and extract all content.
Batch Script:
#!/bin/bash
# batch_process.sh
INPUT_DIR="./pdfs"
OUTPUT_DIR="./processed"
TABLES_DIR="./tables"
SUMMARIES_DIR="./summaries"
mkdir -p "$OUTPUT_DIR" "$TABLES_DIR" "$SUMMARIES_DIR"
# Process each PDF
for pdf in "$INPUT_DIR"/*.pdf; do
base_name=$(basename "$pdf" .pdf)
echo "Processing: $base_name"
# Extract text
python scripts/extract_text.py \
--input "$pdf" \
--output "$OUTPUT_DIR/${base_name}.txt"
# Extract tables
python scripts/extract_tables.py \
--input "$pdf" \
--format csv \
--output-dir "$TABLES_DIR/${base_name}/"
# Generate summary
python scripts/summarize_pdf.py \
--input "$pdf" \
--output "$SUMMARIES_DIR/${base_name}_summary.md" \
--style concise
echo "Completed: $base_name"
done
echo "Batch processing complete!"Run Batch:
chmod +x batch_process.sh
./batch_process.sh---
Advanced Workflows
Use Case 1: PDF to Blog Post
Scenario: Convert a whitepaper PDF to a blog post with proper formatting.
# 1. Read PDF
Read(file_path="/path/to/whitepaper.pdf")
# 2. Convert to Markdown
python scripts/convert_pdf.py \
--input /path/to/whitepaper.pdf \
--output blog_draft.md \
--format markdown \
--preserve-images
# 3. Extract key quotes for callouts
python scripts/summarize_pdf.py \
--input /path/to/whitepaper.pdf \
--output key_points.md \
--style executive
# 4. Manual editing in blog_draft.md
# - Add SEO meta tags
# - Format code blocks
# - Add call-to-actionUse Case 2: PDF Comparison
Scenario: Compare two versions of a document.
# 1. Extract text from both versions
python scripts/extract_text.py \
--input document_v1.pdf \
--output v1.txt
python scripts/extract_text.py \
--input document_v2.pdf \
--output v2.txt
# 2. Use diff to compare
diff -u v1.txt v2.txt > changes.diff
# 3. Or use a more sophisticated tool
git diff --no-index v1.txt v2.txtUse Case 3: PDF Data Pipeline
Scenario: Automated pipeline for processing uploaded PDFs.
#!/usr/bin/env python3
# pdf_pipeline.py
import subprocess
import json
from pathlib import Path
def process_pdf_pipeline(pdf_path):
"""Complete PDF processing pipeline."""
base_name = Path(pdf_path).stem
output_dir = Path(f"./processed/{base_name}")
output_dir.mkdir(parents=True, exist_ok=True)
# Step 1: Extract metadata and convert to JSON
json_path = output_dir / f"{base_name}.json"
subprocess.run([
'python', 'scripts/convert_pdf.py',
'--input', pdf_path,
'--output', str(json_path),
'--format', 'json',
'--extract-metadata'
])
# Step 2: Extract tables
tables_dir = output_dir / 'tables'
subprocess.run([
'python', 'scripts/extract_tables.py',
'--input', pdf_path,
'--format', 'csv',
'--output-dir', str(tables_dir)
])
# Step 3: Generate summary
summary_path = output_dir / f"{base_name}_summary.md"
subprocess.run([
'python', 'scripts/summarize_pdf.py',
'--input', pdf_path,
'--output', str(summary_path),
'--style', 'concise'
])
# Step 4: Load and return results
with open(json_path) as f:
data = json.load(f)
return {
'pdf': pdf_path,
'output_dir': str(output_dir),
'metadata': data.get('metadata'),
'tables_count': len(list(tables_dir.glob('*.csv'))) if tables_dir.exists() else 0,
'summary': str(summary_path)
}
# Use in automation
if __name__ == '__main__':
result = process_pdf_pipeline('./uploads/new_document.pdf')
print(json.dumps(result, indent=2))Use Case 4: OCR for Scanned PDFs
Scenario: Extract text from scanned documents.
# 1. Convert PDF to images
python -c "
from pdf2image import convert_from_path
images = convert_from_path('scanned_document.pdf')
for i, img in enumerate(images):
img.save(f'page_{i+1}.png', 'PNG')
"
# 2. Apply OCR to each page
for img in page_*.png; do
tesseract "$img" "${img%.png}" -l eng
done
# 3. Combine all text files
cat page_*.txt > scanned_document_ocr.txt
# 4. Clean up temporary files
rm page_*.png page_*.txtUse Case 5: PDF Search Index
Scenario: Build searchable index of PDF library.
#!/usr/bin/env python3
# build_search_index.py
import json
import subprocess
from pathlib import Path
from whoosh.index import create_in
from whoosh.fields import Schema, TEXT, ID
from whoosh.qparser import QueryParser
# Define search schema
schema = Schema(
path=ID(stored=True),
filename=TEXT(stored=True),
content=TEXT,
title=TEXT(stored=True),
author=TEXT(stored=True)
)
# Create index
index_dir = Path("./search_index")
index_dir.mkdir(exist_ok=True)
ix = create_in(str(index_dir), schema)
# Process all PDFs
writer = ix.writer()
for pdf_path in Path("./pdf_library").glob("**/*.pdf"):
print(f"Indexing: {pdf_path}")
# Extract text
result = subprocess.run([
'python', 'scripts/extract_text.py',
'--input', str(pdf_path)
], capture_output=True, text=True)
text_content = result.stdout
# Extract metadata
metadata_result = subprocess.run([
'python', 'scripts/convert_pdf.py',
'--input', str(pdf_path),
'--output', '/tmp/temp.json',
'--format', 'json',
'--extract-metadata'
], capture_output=True)
with open('/tmp/temp.json') as f:
data = json.load(f)
metadata = data.get('metadata', {})
# Add to index
writer.add_document(
path=str(pdf_path),
filename=pdf_path.name,
content=text_content,
title=metadata.get('title', pdf_path.stem),
author=metadata.get('author', 'Unknown')
)
writer.commit()
print("Search index built successfully!")
# Example search
from whoosh.qparser import MultifieldParser
with ix.searcher() as searcher:
query = MultifieldParser(["content", "title"], ix.schema).parse("machine learning")
results = searcher.search(query, limit=10)
print(f"\nFound {len(results)} results for 'machine learning':")
for hit in results:
print(f" - {hit['filename']}: {hit['title']}")---
Performance Tips
For Large PDFs (>100 pages)
# Use chunked processing
python scripts/extract_text.py \
--input large_document.pdf \
--chunk-size 20 \
--output document.txtFor High-Volume Processing
# Parallel processing with GNU parallel
find ./pdfs -name "*.pdf" | \
parallel python scripts/extract_text.py --input {} --output {.}.txtFor Scanned PDFs
# Optimize OCR accuracy with preprocessing
convert input.pdf -density 300 -depth 8 -quality 85 output.pdf
python scripts/extract_text.py --input output.pdf --output text.txt---
Troubleshooting Examples
Problem: Jumbled Text from Multi-Column PDF
Solution:
# Use pdfplumber for better layout analysis
python -c "
import pdfplumber
with pdfplumber.open('multi_column.pdf') as pdf:
for page in pdf.pages:
# Extract with layout preservation
text = page.extract_text(layout=True)
print(text)
"Problem: Missing Tables
Solution:
# Try different table extraction settings
python -c "
import pdfplumber
with pdfplumber.open('document.pdf') as pdf:
for page in pdf.pages:
tables = page.extract_tables(table_settings={
'vertical_strategy': 'text',
'horizontal_strategy': 'text'
})
for table in tables:
print(table)
"---
Version History
- 1.0.0 (2025-11-18) - Initial examples documentation
Claude's PDF Processing Capabilities
Overview
Claude has native support for processing PDF documents, enabling comprehensive analysis and extraction without requiring external services. This document outlines Claude's PDF capabilities and how to leverage them effectively.
Native PDF Support
What Claude Can Do
Claude can directly read and process PDF files through the Read tool, which provides:
1. Visual Content Understanding
- View PDF pages as images
- Understand layout and formatting
- Analyze charts, graphs, and diagrams
- Interpret infographics and visual elements
- Describe images and figures
2. Text Extraction
- Extract text from standard PDFs
- Preserve paragraph structure
- Recognize headers and footers
- Handle multi-column layouts
- Maintain reading order
3. Table Recognition
- Detect tables automatically
- Preserve table structure
- Extract headers and data rows
- Handle merged cells
- Support multi-page tables
4. Document Structure Analysis
- Identify document sections
- Recognize headings and subheadings
- Detect lists and bullet points
- Understand document hierarchy
- Parse form fields
5. Metadata Extraction
- Title, author, subject
- Creation and modification dates
- Page count
- PDF version
- Creator application
Processing Workflow
Recommended Approach
When working with PDFs, follow this workflow:
1. Read PDF with Claude's Read tool
↓
2. Analyze content and structure
↓
3. Determine user's specific needs
↓
4. Use appropriate script for extraction/conversion
↓
5. Validate and present resultsExample Workflow
# Step 1: Read PDF directly
Read(file_path="/path/to/document.pdf")
# Step 2: Claude analyzes the content automatically
# - Sees the visual layout
# - Extracts text content
# - Identifies tables and charts
# - Understands document structure
# Step 3: Based on analysis, use scripts for specific tasks
# e.g., Extract tables to CSV
Bash: python scripts/extract_tables.py --input document.pdf --format csv --output-dir ./tables/
# Step 4: Validate output
Bash: ls -la ./tables/
Read: ./tables/table_1_page_2.csvContent Types and Handling
1. Text-Based PDFs
Characteristics:
- Generated from word processors or design tools
- Text is selectable
- High accuracy for text extraction
Best Practices:
- Use Read tool for initial analysis
- Extract with
extract_text.pyfor bulk processing - Convert to Markdown for documentation purposes
2. Scanned PDFs (Images)
Characteristics:
- Created from scanned documents
- Text is embedded in images
- Requires OCR for text extraction
Best Practices:
- Claude can still view and analyze visually
- Use OCR tools (pytesseract) for text extraction
- May require preprocessing for best results
3. Mixed Content PDFs
Characteristics:
- Combination of text, images, and graphics
- Complex layouts with multiple columns
- Embedded charts and diagrams
Best Practices:
- Read with Claude first to understand layout
- Use pdfplumber for table extraction
- Manually verify complex structures
4. Form PDFs
Characteristics:
- Interactive fields
- Structured data entry
- May include checkboxes and signatures
Best Practices:
- Extract field names and values
- Convert to structured JSON
- Maintain field relationships
Limitations and Considerations
Current Limitations
1. Encrypted/Password-Protected PDFs
- Requires password for processing
- Some DRM-protected PDFs may be unreadable
- Use
--passwordflag in scripts
2. Scanned Documents
- Native extraction won't work
- Requires OCR (Optical Character Recognition)
- Quality depends on scan resolution
3. Complex Layouts
- Multi-column layouts may have reading order issues
- Rotated text may not extract correctly
- Mixed left-to-right and right-to-left text
4. Large Files
- Very large PDFs (>100MB) may be slow to process
- Memory constraints on large page counts
- Use chunked processing when possible
5. Non-Standard Fonts
- Custom or embedded fonts may cause issues
- Some characters may not extract correctly
- Unicode characters may need special handling
Performance Considerations
| PDF Type | Pages | Processing Time | Memory Usage |
|---|---|---|---|
| Simple text | 1-50 | <5 seconds | Low |
| Simple text | 51-200 | 5-30 seconds | Medium |
| Complex layout | 1-50 | 10-60 seconds | Medium |
| Complex layout | 51-200 | 1-5 minutes | High |
| Scanned (OCR) | 1-50 | 1-10 minutes | High |
Advanced Features
1. Table Extraction
Claude and pdfplumber can extract tables with high accuracy:
# Automatic table detection
tables = page.extract_tables()
# Customized table extraction
tables = page.extract_tables(table_settings={
"vertical_strategy": "lines",
"horizontal_strategy": "lines",
"intersection_tolerance": 3
})2. Image Extraction
Extract embedded images from PDFs:
from pdf2image import convert_from_path
# Convert PDF pages to images
images = convert_from_path('document.pdf')
for i, image in enumerate(images):
image.save(f'page_{i+1}.png', 'PNG')3. Layout Analysis
Understand document structure:
# Get layout objects
layout = page.layout_objects
words = page.extract_words()
chars = page.chars4. Coordinate-Based Extraction
Extract specific regions:
# Extract specific area
bbox = (x0, y0, x1, y1) # Bounding box coordinates
region = page.within_bbox(bbox)
text = region.extract_text()Format Conversion Best Practices
PDF to Markdown
Use Cases:
- Documentation
- Blog posts
- Wiki pages
- README files
Considerations:
- Preserve heading hierarchy
- Convert tables to Markdown tables
- Include image references
- Maintain list formatting
PDF to JSON
Use Cases:
- Data extraction
- API integration
- Structured data processing
- Database import
Considerations:
- Define clear schema
- Handle nested structures
- Preserve metadata
- Include data types
PDF to CSV
Use Cases:
- Spreadsheet import
- Data analysis
- Database loading
- Table-specific extraction
Considerations:
- One table per file
- Consistent column headers
- Handle merged cells
- Preserve data types
PDF to Plain Text
Use Cases:
- Full-text search indexing
- Text analysis
- Content migration
- Backup/archival
Considerations:
- Preserve reading order
- Maintain paragraph breaks
- Include page markers
- Handle special characters
Quality Assurance
Validation Checklist
After processing PDFs, verify:
- [ ] All pages processed successfully
- [ ] Text extraction is accurate
- [ ] Tables are properly structured
- [ ] Special characters are preserved
- [ ] Images/charts are identified
- [ ] Output format is correct
- [ ] File size is reasonable
- [ ] No data loss occurred
Common Issues and Solutions
| Issue | Cause | Solution |
|---|---|---|
| Missing text | Scanned PDF | Use OCR with pytesseract |
| Jumbled text | Multi-column layout | Adjust extraction settings |
| Missing tables | Complex table borders | Use pdfplumber with custom settings |
| Slow processing | Large file size | Use chunked processing |
| Password error | Encrypted PDF | Provide password with --password |
| Unicode errors | Special characters | Use UTF-8 encoding |
Security and Privacy
Best Practices
1. Sensitive Documents
- Verify user authorization
- Don't log sensitive content
- Delete temporary files
- Use secure storage
2. Password Protection
- Never log passwords
- Clear from memory after use
- Validate before processing
- Handle errors securely
3. Data Handling
- Minimize data retention
- Use secure file permissions
- Encrypt output when needed
- Follow data protection regulations
Integration Examples
With Data Analysis
# Extract tables and analyze with pandas
import pandas as pd
# Extract table
tables = extract_tables_from_pdf('report.pdf')
df = pd.DataFrame(tables[0]['rows'], columns=tables[0]['headers'])
# Analyze
summary = df.describe()With Search Systems
# Extract text for indexing
text = extract_text_from_pdf('document.pdf')
# Index with search engine
search_index.add_document({
'id': doc_id,
'content': text,
'metadata': extract_metadata('document.pdf')
})With Automation
# Batch process invoices
for pdf in invoices/*.pdf; do
python scripts/extract_tables.py --input "$pdf" --format csv --output-dir ./data/
doneResources
Python Libraries
- pypdf - Modern PDF parsing (successor to PyPDF2)
- pdfplumber - Advanced table extraction and layout analysis
- pdf2image - Convert PDF pages to images
- pytesseract - OCR for scanned PDFs
- PyMuPDF (fitz) - Fast PDF processing with advanced features
- tabula-py - Alternative table extraction (requires Java)
External Tools
- Tesseract OCR - Open-source OCR engine
- Ghostscript - PDF manipulation and conversion
- pdftk - PDF toolkit for merging, splitting, etc.
- wkhtmltopdf - HTML to PDF conversion
Documentation
- PyPDF Documentation: https://pypdf.readthedocs.io/
- pdfplumber: https://github.com/jsvine/pdfplumber
- PDF Reference (Adobe): https://www.adobe.com/devnet/pdf/pdf_reference.html
Version History
- 1.0.0 (2025-11-18) - Initial documentation
#!/usr/bin/env python3
"""
Convert PDF files to various formats (Markdown, JSON, plain text).
This script provides comprehensive PDF conversion with structure preservation.
"""
import argparse
import json
import sys
from pathlib import Path
from typing import Dict, Any, List, Optional
try:
import PyPDF2
except ImportError:
try:
import pypdf as PyPDF2
except ImportError:
print("Error: PyPDF2 or pypdf library required. Install with: pip install pypdf", file=sys.stderr)
sys.exit(1)
try:
import pdfplumber
except ImportError:
pdfplumber = None
print("Warning: pdfplumber not available. Table extraction will be limited.", file=sys.stderr)
def extract_metadata(pdf_path: Path) -> Dict[str, Any]:
"""Extract PDF metadata."""
metadata = {}
try:
with open(pdf_path, 'rb') as f:
pdf_reader = PyPDF2.PdfReader(f)
# Basic metadata
info = pdf_reader.metadata
if info:
metadata['title'] = info.get('/Title', 'Unknown')
metadata['author'] = info.get('/Author', 'Unknown')
metadata['subject'] = info.get('/Subject', 'Unknown')
metadata['creator'] = info.get('/Creator', 'Unknown')
metadata['producer'] = info.get('/Producer', 'Unknown')
metadata['creation_date'] = str(info.get('/CreationDate', 'Unknown'))
metadata['modification_date'] = str(info.get('/ModDate', 'Unknown'))
metadata['pages'] = len(pdf_reader.pages)
metadata['encrypted'] = pdf_reader.is_encrypted
except Exception as e:
print(f"Warning: Could not extract metadata: {e}", file=sys.stderr)
return metadata
def convert_to_markdown(
input_path: Path,
output_path: Path,
preserve_images: bool = False,
extract_metadata: bool = True
) -> str:
"""
Convert PDF to Markdown format.
Args:
input_path: Path to input PDF
output_path: Path to output Markdown file
preserve_images: Whether to extract and reference images
extract_metadata: Whether to include metadata header
Returns:
Markdown content
"""
markdown_lines = []
# Add metadata header
if extract_metadata:
metadata = extract_metadata(input_path)
markdown_lines.append("---\n")
for key, value in metadata.items():
markdown_lines.append(f"{key}: {value}\n")
markdown_lines.append("---\n\n")
# Extract text content
with open(input_path, 'rb') as f:
pdf_reader = PyPDF2.PdfReader(f)
for page_num, page in enumerate(pdf_reader.pages, start=1):
text = page.extract_text()
# Add page marker
markdown_lines.append(f"\n## Page {page_num}\n\n")
# Process text to improve formatting
# (basic heuristics - could be enhanced)
lines = text.split('\n')
for line in lines:
line = line.strip()
if not line:
markdown_lines.append('\n')
continue
# Detect potential headings (all caps, short lines)
if len(line) < 80 and line.isupper() and len(line.split()) <= 10:
markdown_lines.append(f"### {line}\n\n")
else:
markdown_lines.append(f"{line}\n")
markdown_lines.append('\n')
# Extract tables if pdfplumber is available
if pdfplumber:
try:
with pdfplumber.open(input_path) as pdf:
table_count = 0
for page_num, page in enumerate(pdf.pages, start=1):
tables = page.extract_tables()
for table in tables:
table_count += 1
markdown_lines.append(f"\n### Table {table_count}\n\n")
if table and len(table) > 0:
# Headers
headers = table[0]
markdown_lines.append('| ' + ' | '.join(str(h) if h else '' for h in headers) + ' |\n')
markdown_lines.append('|' + '|'.join(['---' for _ in headers]) + '|\n')
# Rows
for row in table[1:]:
markdown_lines.append('| ' + ' | '.join(str(cell) if cell else '' for cell in row) + ' |\n')
markdown_lines.append('\n')
except Exception as e:
print(f"Warning: Table extraction failed: {e}", file=sys.stderr)
result = ''.join(markdown_lines)
# Write to file
output_path.parent.mkdir(parents=True, exist_ok=True)
with open(output_path, 'w', encoding='utf-8') as f:
f.write(result)
print(f"PDF converted to Markdown: {output_path}", file=sys.stderr)
return result
def convert_to_json(
input_path: Path,
output_path: Path,
extract_metadata_flag: bool = True
) -> Dict[str, Any]:
"""
Convert PDF to structured JSON format.
Args:
input_path: Path to input PDF
output_path: Path to output JSON file
extract_metadata_flag: Whether to include metadata
Returns:
JSON data structure
"""
data = {
'source': str(input_path),
'content': []
}
# Add metadata
if extract_metadata_flag:
data['metadata'] = extract_metadata(input_path)
# Extract text content by page
with open(input_path, 'rb') as f:
pdf_reader = PyPDF2.PdfReader(f)
for page_num, page in enumerate(pdf_reader.pages, start=1):
page_data = {
'page': page_num,
'type': 'text',
'content': page.extract_text()
}
data['content'].append(page_data)
# Extract tables if available
if pdfplumber:
try:
with pdfplumber.open(input_path) as pdf:
for page_num, page in enumerate(pdf.pages, start=1):
tables = page.extract_tables()
for table_idx, table in enumerate(tables):
table_data = {
'page': page_num,
'type': 'table',
'table_index': table_idx,
'headers': table[0] if table else [],
'rows': table[1:] if len(table) > 1 else []
}
data['content'].append(table_data)
except Exception as e:
print(f"Warning: Table extraction failed: {e}", file=sys.stderr)
# Write to file
output_path.parent.mkdir(parents=True, exist_ok=True)
with open(output_path, 'w', encoding='utf-8') as f:
json.dump(data, f, indent=2, ensure_ascii=False)
print(f"PDF converted to JSON: {output_path}", file=sys.stderr)
return data
def convert_to_text(
input_path: Path,
output_path: Path
) -> str:
"""
Convert PDF to plain text.
Args:
input_path: Path to input PDF
output_path: Path to output text file
Returns:
Text content
"""
text_lines = []
with open(input_path, 'rb') as f:
pdf_reader = PyPDF2.PdfReader(f)
for page_num, page in enumerate(pdf_reader.pages, start=1):
text_lines.append(f"\n{'='*80}\n")
text_lines.append(f"PAGE {page_num}\n")
text_lines.append(f"{'='*80}\n\n")
text_lines.append(page.extract_text())
text_lines.append('\n\n')
result = ''.join(text_lines)
# Write to file
output_path.parent.mkdir(parents=True, exist_ok=True)
with open(output_path, 'w', encoding='utf-8') as f:
f.write(result)
print(f"PDF converted to text: {output_path}", file=sys.stderr)
return result
def main():
parser = argparse.ArgumentParser(
description="Convert PDF files to various formats",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Convert to Markdown
%(prog)s --input document.pdf --output document.md --format markdown
# Convert to JSON with metadata
%(prog)s --input document.pdf --output document.json --format json --extract-metadata
# Convert to plain text
%(prog)s --input document.pdf --output document.txt --format text
"""
)
parser.add_argument('--input', '-i', required=True, type=Path,
help='Input PDF file path')
parser.add_argument('--output', '-o', required=True, type=Path,
help='Output file path')
parser.add_argument('--format', '-f', choices=['markdown', 'json', 'text'],
default='markdown',
help='Output format (default: markdown)')
parser.add_argument('--preserve-images', action='store_true',
help='Preserve and extract images (Markdown only)')
parser.add_argument('--extract-metadata', action='store_true',
help='Include PDF metadata in output')
args = parser.parse_args()
try:
if args.format == 'markdown':
convert_to_markdown(
args.input,
args.output,
preserve_images=args.preserve_images,
extract_metadata=args.extract_metadata
)
elif args.format == 'json':
convert_to_json(
args.input,
args.output,
extract_metadata_flag=args.extract_metadata
)
elif args.format == 'text':
convert_to_text(args.input, args.output)
except Exception as e:
print(f"Failed to convert PDF: {e}", file=sys.stderr)
sys.exit(1)
if __name__ == '__main__':
main()
#!/usr/bin/env python3
"""
Decode base64-encoded PDF content and save as a PDF file.
"""
import argparse
import base64
import sys
from pathlib import Path
def decode_base64_pdf(input_path: Path, output_path: Path) -> None:
"""
Decode base64-encoded PDF data.
Args:
input_path: Path to file containing base64-encoded PDF data
output_path: Path to save the decoded PDF file
"""
try:
# Read base64 content
with open(input_path, 'r', encoding='utf-8') as f:
base64_content = f.read().strip()
# Remove data URI prefix if present
if base64_content.startswith('data:'):
# Format: data:application/pdf;base64,<base64-data>
if ';base64,' in base64_content:
base64_content = base64_content.split(';base64,')[1]
# Decode base64
print("Decoding base64 content...", file=sys.stderr)
pdf_data = base64.b64decode(base64_content)
# Verify it's a PDF by checking magic bytes
if not pdf_data.startswith(b'%PDF'):
print("Warning: Decoded data doesn't appear to be a valid PDF", file=sys.stderr)
# Create output directory if needed
output_path.parent.mkdir(parents=True, exist_ok=True)
# Write PDF file
with open(output_path, 'wb') as f:
f.write(pdf_data)
print(f"PDF decoded successfully: {output_path}", file=sys.stderr)
print(f"File size: {len(pdf_data) / 1024:.1f} KB", file=sys.stderr)
except base64.binascii.Error as e:
print(f"Error: Invalid base64 encoding: {e}", file=sys.stderr)
raise
except Exception as e:
print(f"Error decoding PDF: {e}", file=sys.stderr)
raise
def decode_base64_string(base64_string: str, output_path: Path) -> None:
"""
Decode base64 string directly (not from file).
Args:
base64_string: Base64-encoded PDF data
output_path: Path to save the decoded PDF file
"""
try:
# Remove data URI prefix if present
if base64_string.startswith('data:'):
if ';base64,' in base64_string:
base64_string = base64_string.split(';base64,')[1]
# Decode base64
print("Decoding base64 string...", file=sys.stderr)
pdf_data = base64.b64decode(base64_string.strip())
# Verify it's a PDF
if not pdf_data.startswith(b'%PDF'):
print("Warning: Decoded data doesn't appear to be a valid PDF", file=sys.stderr)
# Create output directory if needed
output_path.parent.mkdir(parents=True, exist_ok=True)
# Write PDF file
with open(output_path, 'wb') as f:
f.write(pdf_data)
print(f"PDF decoded successfully: {output_path}", file=sys.stderr)
print(f"File size: {len(pdf_data) / 1024:.1f} KB", file=sys.stderr)
except Exception as e:
print(f"Error decoding PDF: {e}", file=sys.stderr)
raise
def main():
parser = argparse.ArgumentParser(
description="Decode base64-encoded PDF content",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Decode from file
%(prog)s --input base64_content.txt --output document.pdf
# Decode from string
%(prog)s --string "JVBERi0xLjQKJ..." --output document.pdf
"""
)
parser.add_argument('--input', '-i', type=Path,
help='Input file containing base64-encoded PDF data')
parser.add_argument('--string', '-s', type=str,
help='Base64-encoded PDF string')
parser.add_argument('--output', '-o', required=True, type=Path,
help='Output PDF file path')
args = parser.parse_args()
if not args.input and not args.string:
print("Error: Either --input or --string must be provided", file=sys.stderr)
sys.exit(1)
if args.input and args.string:
print("Error: Only one of --input or --string should be provided", file=sys.stderr)
sys.exit(1)
try:
if args.input:
decode_base64_pdf(args.input, args.output)
else:
decode_base64_string(args.string, args.output)
except Exception as e:
print(f"Failed to decode PDF: {e}", file=sys.stderr)
sys.exit(1)
if __name__ == '__main__':
main()
#!/usr/bin/env python3
"""
Download PDF files from URLs.
"""
import argparse
import sys
from pathlib import Path
try:
import requests
except ImportError:
print("Error: requests library required. Install with: pip install requests", file=sys.stderr)
sys.exit(1)
def download_pdf(url: str, output_path: Path, verify_ssl: bool = True) -> None:
"""
Download a PDF from a URL.
Args:
url: URL of the PDF file
output_path: Path to save the downloaded PDF
verify_ssl: Whether to verify SSL certificates
"""
try:
print(f"Downloading PDF from: {url}", file=sys.stderr)
# Make request with streaming
response = requests.get(url, stream=True, verify=verify_ssl, timeout=30)
response.raise_for_status()
# Check content type
content_type = response.headers.get('Content-Type', '')
if 'application/pdf' not in content_type and not url.endswith('.pdf'):
print(f"Warning: Content-Type is '{content_type}', expected 'application/pdf'", file=sys.stderr)
# Create output directory if needed
output_path.parent.mkdir(parents=True, exist_ok=True)
# Download with progress
total_size = int(response.headers.get('Content-Length', 0))
downloaded = 0
with open(output_path, 'wb') as f:
for chunk in response.iter_content(chunk_size=8192):
if chunk:
f.write(chunk)
downloaded += len(chunk)
if total_size > 0:
progress = (downloaded / total_size) * 100
print(f"\rProgress: {progress:.1f}%", end='', file=sys.stderr)
if total_size > 0:
print(file=sys.stderr) # New line after progress
print(f"PDF downloaded successfully: {output_path}", file=sys.stderr)
print(f"File size: {output_path.stat().st_size / 1024:.1f} KB", file=sys.stderr)
except requests.exceptions.RequestException as e:
print(f"Error downloading PDF: {e}", file=sys.stderr)
raise
except Exception as e:
print(f"Unexpected error: {e}", file=sys.stderr)
raise
def main():
parser = argparse.ArgumentParser(
description="Download PDF files from URLs",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Download PDF
%(prog)s --url "https://example.com/document.pdf" --output document.pdf
# Download without SSL verification (use with caution)
%(prog)s --url "https://example.com/doc.pdf" --output doc.pdf --no-verify-ssl
"""
)
parser.add_argument('--url', '-u', required=True, type=str,
help='URL of the PDF file')
parser.add_argument('--output', '-o', required=True, type=Path,
help='Output file path')
parser.add_argument('--no-verify-ssl', action='store_true',
help='Disable SSL certificate verification')
args = parser.parse_args()
try:
download_pdf(args.url, args.output, verify_ssl=not args.no_verify_ssl)
except Exception as e:
print(f"Failed to download PDF: {e}", file=sys.stderr)
sys.exit(1)
if __name__ == '__main__':
main()
#!/usr/bin/env python3
"""
Extract tables from PDF files and convert to CSV, JSON, or Markdown format.
This script uses pdfplumber for robust table detection and extraction.
"""
import argparse
import json
import sys
from pathlib import Path
from typing import List, Dict, Any
try:
import pdfplumber
except ImportError:
print("Error: pdfplumber library required. Install with: pip install pdfplumber", file=sys.stderr)
sys.exit(1)
def extract_tables_from_pdf(
input_path: Path,
output_format: str = 'csv',
output_dir: Path = None,
output_file: Path = None
) -> List[Dict[str, Any]]:
"""
Extract all tables from a PDF file.
Args:
input_path: Path to input PDF file
output_format: Output format ('csv', 'json', 'markdown')
output_dir: Directory to save individual table files
output_file: Single file to save all tables (JSON only)
Returns:
List of extracted tables with metadata
"""
if not input_path.exists():
raise FileNotFoundError(f"PDF file not found: {input_path}")
extracted_tables = []
try:
with pdfplumber.open(input_path) as pdf:
total_pages = len(pdf.pages)
print(f"Processing {total_pages} pages for tables...", file=sys.stderr)
table_count = 0
for page_num, page in enumerate(pdf.pages, start=1):
tables = page.extract_tables()
if tables:
print(f"Found {len(tables)} table(s) on page {page_num}", file=sys.stderr)
for table_idx, table in enumerate(tables):
table_count += 1
# Store table with metadata
table_data = {
'page': page_num,
'table_number': table_count,
'headers': table[0] if table else [],
'rows': table[1:] if len(table) > 1 else [],
'raw_data': table
}
extracted_tables.append(table_data)
# Save individual table files
if output_dir:
output_dir.mkdir(parents=True, exist_ok=True)
save_table(table_data, output_format, output_dir, table_count)
print(f"Extracted {table_count} tables total", file=sys.stderr)
# Save all tables to a single JSON file
if output_file and output_format == 'json':
output_file.parent.mkdir(parents=True, exist_ok=True)
with open(output_file, 'w', encoding='utf-8') as f:
json.dump(extracted_tables, f, indent=2, ensure_ascii=False)
print(f"All tables saved to: {output_file}", file=sys.stderr)
return extracted_tables
except Exception as e:
print(f"Error extracting tables: {e}", file=sys.stderr)
raise
def save_table(table_data: Dict[str, Any], format: str, output_dir: Path, table_num: int):
"""Save a single table to a file in the specified format."""
filename = f"table_{table_num}_page_{table_data['page']}"
if format == 'csv':
save_as_csv(table_data, output_dir / f"{filename}.csv")
elif format == 'json':
save_as_json(table_data, output_dir / f"{filename}.json")
elif format == 'markdown':
save_as_markdown(table_data, output_dir / f"{filename}.md")
def save_as_csv(table_data: Dict[str, Any], output_path: Path):
"""Save table as CSV file."""
import csv
with open(output_path, 'w', newline='', encoding='utf-8') as f:
writer = csv.writer(f)
# Write headers
if table_data['headers']:
writer.writerow(table_data['headers'])
# Write rows
for row in table_data['rows']:
writer.writerow(row)
print(f" Saved: {output_path}", file=sys.stderr)
def save_as_json(table_data: Dict[str, Any], output_path: Path):
"""Save table as JSON file."""
with open(output_path, 'w', encoding='utf-8') as f:
json.dump({
'page': table_data['page'],
'table_number': table_data['table_number'],
'headers': table_data['headers'],
'rows': table_data['rows']
}, f, indent=2, ensure_ascii=False)
print(f" Saved: {output_path}", file=sys.stderr)
def save_as_markdown(table_data: Dict[str, Any], output_path: Path):
"""Save table as Markdown file."""
lines = []
lines.append(f"# Table {table_data['table_number']} (Page {table_data['page']})\n\n")
# Headers
if table_data['headers']:
lines.append('| ' + ' | '.join(str(h) if h else '' for h in table_data['headers']) + ' |\n')
lines.append('|' + '|'.join(['---' for _ in table_data['headers']]) + '|\n')
# Rows
for row in table_data['rows']:
lines.append('| ' + ' | '.join(str(cell) if cell else '' for cell in row) + ' |\n')
with open(output_path, 'w', encoding='utf-8') as f:
f.writelines(lines)
print(f" Saved: {output_path}", file=sys.stderr)
def main():
parser = argparse.ArgumentParser(
description="Extract tables from PDF files",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Extract all tables to CSV files
%(prog)s --input document.pdf --format csv --output-dir ./tables/
# Extract tables to JSON
%(prog)s --input document.pdf --format json --output tables.json
# Extract tables to Markdown
%(prog)s --input document.pdf --format markdown --output-dir ./tables/
"""
)
parser.add_argument('--input', '-i', required=True, type=Path,
help='Input PDF file path')
parser.add_argument('--format', '-f', choices=['csv', 'json', 'markdown'],
default='csv',
help='Output format (default: csv)')
parser.add_argument('--output-dir', '-d', type=Path,
help='Output directory for individual table files')
parser.add_argument('--output', '-o', type=Path,
help='Output file for all tables (JSON format only)')
args = parser.parse_args()
if not args.output_dir and not args.output:
print("Error: Either --output-dir or --output must be specified", file=sys.stderr)
sys.exit(1)
if args.output and args.format != 'json':
print("Warning: --output only works with JSON format. Use --output-dir for CSV/Markdown", file=sys.stderr)
try:
extract_tables_from_pdf(
input_path=args.input,
output_format=args.format,
output_dir=args.output_dir,
output_file=args.output
)
except Exception as e:
print(f"Failed to extract tables: {e}", file=sys.stderr)
sys.exit(1)
if __name__ == '__main__':
main()
#!/usr/bin/env python3
"""
Extract text content from PDF files with optional formatting preservation.
This script provides flexible text extraction from PDFs including:
- Full document or specific pages
- Formatting preservation
- Password-protected PDFs
- Chunked processing for large files
"""
import argparse
import sys
from pathlib import Path
from typing import Optional, List
try:
import PyPDF2
except ImportError:
try:
import pypdf as PyPDF2
except ImportError:
print("Error: PyPDF2 or pypdf library required. Install with: pip install pypdf", file=sys.stderr)
sys.exit(1)
def parse_page_range(page_spec: str, total_pages: int) -> List[int]:
"""
Parse page specification like '1,3,5-10' into list of page numbers.
Args:
page_spec: String specification of pages (e.g., '1,3,5-10')
total_pages: Total number of pages in document
Returns:
List of page numbers (0-indexed)
"""
pages = set()
for part in page_spec.split(','):
if '-' in part:
start, end = part.split('-')
start = int(start.strip())
end = int(end.strip())
pages.update(range(start - 1, min(end, total_pages)))
else:
page = int(part.strip())
if 0 < page <= total_pages:
pages.add(page - 1)
return sorted(list(pages))
def extract_text_from_pdf(
input_path: Path,
output_path: Optional[Path] = None,
pages: Optional[str] = None,
password: Optional[str] = None,
preserve_formatting: bool = True,
chunk_size: Optional[int] = None
) -> str:
"""
Extract text from a PDF file.
Args:
input_path: Path to input PDF file
output_path: Path to output text file (if None, returns text)
pages: Page specification (e.g., '1,3,5-10')
password: Password for encrypted PDFs
preserve_formatting: Whether to preserve spacing and line breaks
chunk_size: Process in chunks of N pages (for large files)
Returns:
Extracted text content
"""
if not input_path.exists():
raise FileNotFoundError(f"PDF file not found: {input_path}")
extracted_text = []
try:
with open(input_path, 'rb') as pdf_file:
# Create PDF reader
pdf_reader = PyPDF2.PdfReader(pdf_file)
# Handle encrypted PDFs
if pdf_reader.is_encrypted:
if password:
pdf_reader.decrypt(password)
else:
raise ValueError("PDF is password-protected. Please provide password with --password")
total_pages = len(pdf_reader.pages)
print(f"Processing PDF with {total_pages} pages...", file=sys.stderr)
# Determine which pages to extract
if pages:
page_list = parse_page_range(pages, total_pages)
print(f"Extracting pages: {[p+1 for p in page_list]}", file=sys.stderr)
else:
page_list = list(range(total_pages))
# Process pages (with optional chunking)
if chunk_size:
for i in range(0, len(page_list), chunk_size):
chunk = page_list[i:i+chunk_size]
print(f"Processing chunk {i//chunk_size + 1}: pages {chunk[0]+1}-{chunk[-1]+1}", file=sys.stderr)
for page_num in chunk:
page = pdf_reader.pages[page_num]
text = page.extract_text()
if preserve_formatting:
extracted_text.append(f"\n{'='*80}\n")
extracted_text.append(f"PAGE {page_num + 1}\n")
extracted_text.append(f"{'='*80}\n\n")
extracted_text.append(text)
extracted_text.append("\n\n")
else:
for page_num in page_list:
page = pdf_reader.pages[page_num]
text = page.extract_text()
if preserve_formatting:
extracted_text.append(f"\n{'='*80}\n")
extracted_text.append(f"PAGE {page_num + 1}\n")
extracted_text.append(f"{'='*80}\n\n")
extracted_text.append(text)
extracted_text.append("\n\n")
result = ''.join(extracted_text)
# Write to file if output path provided
if output_path:
output_path.parent.mkdir(parents=True, exist_ok=True)
with open(output_path, 'w', encoding='utf-8') as f:
f.write(result)
print(f"Text extracted successfully to: {output_path}", file=sys.stderr)
return result
except Exception as e:
print(f"Error extracting text from PDF: {e}", file=sys.stderr)
raise
def main():
parser = argparse.ArgumentParser(
description="Extract text content from PDF files",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Extract all text from PDF
%(prog)s --input document.pdf --output document.txt
# Extract specific pages
%(prog)s --input document.pdf --pages "1,3,5-10" --output selected.txt
# Extract from password-protected PDF
%(prog)s --input secure.pdf --password "secret" --output document.txt
# Process large PDF in chunks
%(prog)s --input large.pdf --chunk-size 10 --output document.txt
"""
)
parser.add_argument('--input', '-i', required=True, type=Path,
help='Input PDF file path')
parser.add_argument('--output', '-o', type=Path,
help='Output text file path (prints to stdout if not specified)')
parser.add_argument('--pages', '-p', type=str,
help='Pages to extract (e.g., "1,3,5-10")')
parser.add_argument('--password', type=str,
help='Password for encrypted PDFs')
parser.add_argument('--preserve-formatting', type=lambda x: x.lower() == 'true',
default=True,
help='Preserve formatting with page markers (default: true)')
parser.add_argument('--chunk-size', type=int,
help='Process in chunks of N pages (for large files)')
args = parser.parse_args()
try:
result = extract_text_from_pdf(
input_path=args.input,
output_path=args.output,
pages=args.pages,
password=args.password,
preserve_formatting=args.preserve_formatting,
chunk_size=args.chunk_size
)
# Print to stdout if no output file specified
if not args.output:
print(result)
except Exception as e:
print(f"Failed to extract text: {e}", file=sys.stderr)
sys.exit(1)
if __name__ == '__main__':
main()
#!/usr/bin/env python3
"""
Summarize PDF content by extracting key information.
This script provides basic summarization by:
- Extracting document metadata
- Identifying headings and sections
- Extracting first/last paragraphs
- Highlighting key sentences
- Providing statistical overview
"""
import argparse
import re
import sys
from pathlib import Path
from typing import List, Dict, Any
try:
import PyPDF2
except ImportError:
try:
import pypdf as PyPDF2
except ImportError:
print("Error: PyPDF2 or pypdf library required. Install with: pip install pypdf", file=sys.stderr)
sys.exit(1)
try:
import pdfplumber
except ImportError:
pdfplumber = None
def extract_metadata(pdf_path: Path) -> Dict[str, Any]:
"""Extract PDF metadata."""
metadata = {}
try:
with open(pdf_path, 'rb') as f:
pdf_reader = PyPDF2.PdfReader(f)
info = pdf_reader.metadata
if info:
metadata['title'] = info.get('/Title', 'Unknown')
metadata['author'] = info.get('/Author', 'Unknown')
metadata['subject'] = info.get('/Subject', 'Unknown')
metadata['creator'] = info.get('/Creator', 'Unknown')
metadata['pages'] = len(pdf_reader.pages)
except Exception as e:
print(f"Warning: Could not extract metadata: {e}", file=sys.stderr)
return metadata
def extract_headings(text: str) -> List[str]:
"""Extract potential headings from text."""
headings = []
lines = text.split('\n')
for line in lines:
line = line.strip()
# Heuristics for headings:
# - Short lines (< 80 chars)
# - All caps or title case
# - Ends without punctuation
if line and len(line) < 80:
if (line.isupper() or line.istitle()) and not line.endswith(('.', ',', ';', ':', '!', '?')):
headings.append(line)
return headings[:20] # Limit to first 20 headings
def extract_key_sentences(text: str, num_sentences: int = 10) -> List[str]:
"""
Extract key sentences using simple heuristics.
Prioritizes:
- First sentence of paragraphs
- Sentences with numbers/statistics
- Sentences with emphasis keywords
"""
# Split into sentences (basic)
sentences = re.split(r'[.!?]+', text)
sentences = [s.strip() for s in sentences if len(s.strip()) > 20]
key_sentences = []
emphasis_words = ['important', 'significant', 'critical', 'key', 'main', 'primary',
'conclude', 'therefore', 'however', 'moreover', 'furthermore']
for sentence in sentences[:50]: # Check first 50 sentences
# Check for numbers/statistics
if re.search(r'\d+', sentence):
key_sentences.append(sentence)
continue
# Check for emphasis words
if any(word in sentence.lower() for word in emphasis_words):
key_sentences.append(sentence)
continue
if len(key_sentences) >= num_sentences:
break
return key_sentences[:num_sentences]
def summarize_pdf(
input_path: Path,
output_path: Path,
style: str = 'concise'
) -> str:
"""
Generate a summary of a PDF document.
Args:
input_path: Path to input PDF
output_path: Path to output summary file
style: Summary style ('concise', 'detailed', 'executive')
Returns:
Summary text
"""
if not input_path.exists():
raise FileNotFoundError(f"PDF file not found: {input_path}")
print(f"Summarizing PDF: {input_path}", file=sys.stderr)
# Extract metadata
metadata = extract_metadata(input_path)
# Extract full text
full_text = []
with open(input_path, 'rb') as f:
pdf_reader = PyPDF2.PdfReader(f)
for page in pdf_reader.pages:
full_text.append(page.extract_text())
all_text = '\n'.join(full_text)
# Extract structural elements
headings = extract_headings(all_text)
key_sentences = extract_key_sentences(all_text, num_sentences=15 if style == 'detailed' else 10)
# Count tables if pdfplumber available
table_count = 0
if pdfplumber:
try:
with pdfplumber.open(input_path) as pdf:
for page in pdf.pages:
table_count += len(page.extract_tables())
except Exception:
pass
# Build summary
summary_lines = []
# Header
summary_lines.append("# PDF Document Summary\n\n")
summary_lines.append(f"**Source:** {input_path.name}\n")
summary_lines.append(f"**Generated:** {Path(__file__).name}\n\n")
# Metadata section
summary_lines.append("## Document Information\n\n")
for key, value in metadata.items():
summary_lines.append(f"- **{key.title()}:** {value}\n")
# Statistics
word_count = len(all_text.split())
char_count = len(all_text)
summary_lines.append(f"- **Words:** ~{word_count:,}\n")
summary_lines.append(f"- **Characters:** ~{char_count:,}\n")
if table_count > 0:
summary_lines.append(f"- **Tables:** {table_count}\n")
summary_lines.append("\n")
# Structure overview
if headings and style in ['detailed', 'executive']:
summary_lines.append("## Document Structure\n\n")
summary_lines.append("Detected headings and sections:\n\n")
for heading in headings[:10]:
summary_lines.append(f"- {heading}\n")
summary_lines.append("\n")
# Key points
if style == 'executive':
summary_lines.append("## Executive Summary\n\n")
summary_lines.append("### Key Points\n\n")
else:
summary_lines.append("## Key Content\n\n")
for i, sentence in enumerate(key_sentences, 1):
if style == 'concise':
summary_lines.append(f"{i}. {sentence}.\n")
else:
summary_lines.append(f"**Point {i}:** {sentence}.\n\n")
summary_lines.append("\n")
# First page excerpt (for detailed/executive)
if style in ['detailed', 'executive'] and full_text:
summary_lines.append("## Opening Content\n\n")
first_page_excerpt = full_text[0][:500]
summary_lines.append(f"{first_page_excerpt}...\n\n")
# Last page excerpt (for detailed)
if style == 'detailed' and len(full_text) > 1:
summary_lines.append("## Closing Content\n\n")
last_page_excerpt = full_text[-1][:500]
summary_lines.append(f"{last_page_excerpt}...\n\n")
# Footer
summary_lines.append("---\n\n")
summary_lines.append("*This summary was automatically generated. ")
summary_lines.append("For complete information, please refer to the original document.*\n")
result = ''.join(summary_lines)
# Write to file
output_path.parent.mkdir(parents=True, exist_ok=True)
with open(output_path, 'w', encoding='utf-8') as f:
f.write(result)
print(f"Summary saved to: {output_path}", file=sys.stderr)
return result
def main():
parser = argparse.ArgumentParser(
description="Summarize PDF documents",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Concise summary
%(prog)s --input document.pdf --output summary.md --style concise
# Detailed summary
%(prog)s --input document.pdf --output summary.md --style detailed
# Executive summary
%(prog)s --input document.pdf --output summary.md --style executive
"""
)
parser.add_argument('--input', '-i', required=True, type=Path,
help='Input PDF file path')
parser.add_argument('--output', '-o', required=True, type=Path,
help='Output summary file path (Markdown)')
parser.add_argument('--style', '-s', choices=['concise', 'detailed', 'executive'],
default='concise',
help='Summary style (default: concise)')
args = parser.parse_args()
try:
summarize_pdf(args.input, args.output, args.style)
except Exception as e:
print(f"Failed to summarize PDF: {e}", file=sys.stderr)
sys.exit(1)
if __name__ == '__main__':
main()