Searxng Search

SearXNG Search

SearXNG is a privacy-respecting metasearch engine that you can run locally. It aggregates results from multiple search engines and package repositories, returning clean JSON output.

Quick Start

Start SearXNG:

start-searxng --detach

This will:

• Auto-detect podman or docker
• Create a minimal config with JSON output enabled
• Start SearXNG on http://localhost:8888
• Wait until ready

Stop SearXNG:

podman stop searxng  # or: docker stop searxng

Custom port:

start-searxng --port 9999 --detach

Quick Reference

Available Categories

Run to see all categories:

curl -s "http://localhost:8888/config" | jq '.categories'

Notable categories:

• general: General web search (default)
• cargo: Rust crates from crates.io
• packages: Multi-repo (npm, rubygems, haskell/hoogle, hex, packagist, metacpan, pub.dev, pkg.go.dev, docker hub, alpine, etc.)
• it: IT/tech resources (includes GitHub, Docker Hub, crates.io)
• repos: Code repositories
• code: Code search
• scientific publications: Academic papers
• news, videos, images, books, etc.

See [package-engine-status.md](references/package-engine-status.md) for comprehensive package search testing results.

JSON Response Structure

{
  "query": "search term",
  "number_of_results": 0,
  "results": [
    {
      "url": "https://example.com",
      "title": "Result Title",
      "content": "Snippet of content...",
      "publishedDate": "2025-01-01T00:00:00",
      "engine": "duckduckgo",
      "engines": ["duckduckgo", "startpage"],
      "score": 3.0,
      "category": "general"
    }
  ],
  "answers": [],          // Direct answers/infoboxes
  "suggestions": [],      // Search suggestions
  "corrections": [],      // Query corrections
  "infoboxes": [],       // Knowledge panels
  "unresponsive_engines": []
}

Common Usage Patterns

1. Package Repository Searches

Cargo/Rust crates:

curl -s "http://localhost:8888/search?q=tokio&format=json&categories=cargo" | \
  jq '.results[] | {title, url, content}'

npm packages:

curl -s "http://localhost:8888/search?q=express&format=json&categories=packages" | \
  jq '.results[] | select(.engines[] == "npm") | {title, url, content}'

PyPI packages (workaround - see below):

# PyPI engine is enabled but not returning results in current SearXNG config
# Use direct API or qypi CLI instead (see PyPI Workaround section)

2. Web Search with Filtering

IT/Tech search:

curl -s "http://localhost:8888/search?q=rust+async&format=json&categories=it" | \
  jq '.results[0:5] | .[] | {title, url, engines}'

GitHub repositories:

curl -s "http://localhost:8888/search?q=machine+learning&format=json&categories=repos" | \
  jq '.results[] | select(.engines[] == "github") | {title, url}'

3. Extracting Specific Information

Get top 3 results:

curl -s "http://localhost:8888/search?q=rust+ownership&format=json" | \
  jq '.results[0:3] | .[] | {title, url, content}'

Check which engines returned results:

curl -s "http://localhost:8888/search?q=python&format=json" | \
  jq '.results[0].engines'

Get answer boxes/infoboxes:

curl -s "http://localhost:8888/search?q=rust+language&format=json" | \
  jq '.infoboxes, .answers'

PyPI Workaround

Since PyPI is not returning results in SearXNG (despite being enabled), use these alternatives:

Option 1: Direct PyPI JSON API

# Search (limited to simple package name matching)
curl -s "https://pypi.org/pypi/<package>/json" | jq '.info | {name, summary, version, home_page}'

# Example:
curl -s "https://pypi.org/pypi/requests/json" | jq '.info.summary'

Option 2: qypi CLI tool

# Install
uvx qypi search pandas --json

# Get package info
uvx qypi info requests --json

# List releases
uvx qypi releases flask --json

See references/pypi-direct-search.md for more details.

Integration with Nushell

Create a helper function:

def searx [
  query: string,
  --category (-c): string = "general",
  --limit (-l): int = 10
] {
  http get $"http://localhost:8888/search?q=($query | url encode)&format=json&categories=($category)"
  | get results
  | first $limit
  | select title url content engines
}

Usage:

searx "tokio async" --category cargo --limit 5
searx "flask tutorial" --category general

Debugging

Check SearXNG config:

curl -s "http://localhost:8888/config" | jq '.engines[] | select(.name == "pypi")'

Check for engine errors:

curl -s "http://localhost:8888/search?q=test&format=json" | jq '.unresponsive_engines'

Test specific engine:

curl -s "http://localhost:8888/search?q=flask&format=json&engines=pypi" | jq .

Known Issues

• PyPI engine enabled but not working: Use direct API or qypi CLI as workaround
• Cargo category sometimes returns empty: Try categories=packages or categories=it which also include crates.io
• Rate limiting: SearXNG may rate-limit if too many requests in quick succession

Configuration

Using the Helper Script (Recommended)

The start-searxng script creates a minimal configuration automatically:

start-searxng --help

Default config includes:

• use_default_settings: true (inherits all SearXNG defaults)
• JSON format enabled
• Rate limiting disabled (for local use)
• Secret key (change in production!)

Using Your Own Config

start-searxng --config /path/to/your/config/dir

Your config directory should contain settings.yml.

Manual Container Start

# Create config
mkdir -p /tmp/searxng-config
cat > /tmp/searxng-config/settings.yml << 'EOF'
use_default_settings: true
search:
  formats:
    - html
    - json
server:
  secret_key: "change-me-in-production"
  bind_address: "0.0.0.0"
  port: 8080
EOF

# Start with podman
podman run --rm -d --name searxng \
  -p 8888:8080 \
  -v /tmp/searxng-config:/etc/searxng:Z \
  docker.io/searxng/searxng:latest

# Or with docker
docker run --rm -d --name searxng \
  -p 8888:8080 \
  -v /tmp/searxng-config:/etc/searxng \
  docker.io/searxng/searxng:latest

Check Logs

podman logs searxng  # or: docker logs searxng

Advanced Config

See SearXNG Settings Documentation for all options.

Minimal config to add JSON output to defaults:

use_default_settings: true
search:
  formats:
    - html
    - json

SearXNG Search Skill

Enhanced web and package repository search capabilities using SearXNG metasearch engine.

Fully portable - works on any machine with podman or docker, no external dependencies.

What This Provides

• Unified search interface across multiple package repositories (npm, Cargo, Docker Hub, etc.)
• Category-based filtering for targeted searches (IT, repos, scientific publications, etc.)
• JSON output for programmatic consumption
• Workarounds for PyPI (direct API + qypi CLI tool)
• Nushell helper script for convenient command-line usage

Quick Start

1. Start SearXNG:

start-searxng --detach

2. Use the search helper:

searx "tokio" --category cargo
searx "express" --category packages
searx "rust async" --category it --limit 5

3. Or use curl directly:

curl -s "http://localhost:8888/search?q=serde&format=json&categories=cargo" | jq '.results[0:3]'

4. Stop when done:

podman stop searxng  # or: docker stop searxng

Files

• SKILL.md: Main documentation with quick reference and common patterns
• references/package-engine-status.md: Test results for all 14 package repositories ⭐
• references/category-guide.md: Comprehensive guide to all search categories
• references/pypi-direct-search.md: PyPI workarounds (API + qypi CLI)
• references/agent-usage.md: Guide for AI agents using SearXNG
• scripts/start-searxng: Bash script to start SearXNG container (portable!)
• scripts/searx: Nushell helper script with colored output

What Works

✅ 13/14 package repositories working, including:

• Haskell (Hoogle/Hackage) - packages & functions ⭐
• JavaScript (npm)
• Rust (crates.io, lib.rs)
• Ruby (RubyGems)
• PHP (Packagist)
• Erlang/Elixir (Hex)
• Perl (MetaCPAN)
• Dart/Flutter (pub.dev)
• Go (pkg.go.dev)
• Docker Hub
• Alpine/Void Linux packages

✅ General web search: Multiple engines (DuckDuckGo, Startpage, etc.) ✅ GitHub/GitLab: Repository and code search ✅ Academic papers: arXiv, PubMed, Google Scholar, etc.

❌ PyPI (Python): Broken due to bot protection - use direct API or qypi CLI instead

See `references/package-engine-status.md` for detailed test results!

Requirements

• podman or docker (auto-detected)
• curl (for API access)
• jq (optional, for JSON parsing)
• nushell (optional, for the searx helper script)

No installation needed - runs in a container!

Next Steps

1. Try the helper script for common searches 2. Explore different categories (see category-guide.md) 3. For PyPI searches, use uvx qypi search <term> or direct API

Tips

• Use categories=packages for multi-repo package search
• Use categories=cargo specifically for Rust crates
• Combine categories: categories=packages,it,repos
• Filter results by engine in jq: select(.engines[] == "npm")

SearXNG for AI Agents

This guide explains how AI agents (like Claude) can use SearXNG for enhanced search capabilities.

Why Use SearXNG?

Problem: Built-in WebSearch tools often have limitations:

• Limited to ~10 results per query
• Can't search specific package repositories (PyPI, npm, cargo)
• No direct control over search engines used
• May have rate limits or restrictions

Solution: SearXNG provides:

• Direct access to 100+ search engines
• Specialized package repository search
• Full control over categories and filters
• Unlimited local queries
• JSON API for programmatic access

Agent Workflow

1. Start SearXNG (if not running)

# Check if running
curl -sf http://localhost:8888/ > /dev/null || \
  start-searxng --detach

When to start:

• At the beginning of a search-intensive task
• When user explicitly requests package searches
• When WebSearch tool returns insufficient results

2. Choose the Right Category

3. Execute Search

Option A: Direct curl (fast, scriptable)

curl -s "http://localhost:8888/search?q=tokio&format=json&categories=cargo" | \
  jq '.results[0:5] | .[] | {title, url, content}'

Option B: Nushell helper (formatted output)

searx "tokio" --category cargo --limit 5

4. Parse and Present Results

Extract key information:

# Get titles and URLs
jq '.results[] | {title, url}'

# Filter by specific engine
jq '.results[] | select(.engines[] == "npm")'

# Get top N results
jq '.results[0:N]'

# Check which engines returned results
jq '.results[0].engines'

Common Agent Use Cases

Package Discovery

User: "Find a Rust crate for HTTP requests"

Agent workflow: 1. Start SearXNG if needed 2. Search cargo: ?q=http+requests&categories=cargo 3. Parse top 5 results 4. Present: title, URL, description 5. Optionally fetch details from crates.io API

Implementation:

curl -s "http://localhost:8888/search?q=http+requests&format=json&categories=cargo" | \
  jq -r '.results[0:5] | .[] | "[\(.title)](\(.url))\n  \(.content)\n"'

Multi-Repository Search

User: "What are the best logging libraries across different languages?"

Agent workflow: 1. Search categories=packages for general term 2. Group results by engine (npm, crates.io, hex, etc.) 3. Present organized by ecosystem

Implementation:

curl -s "http://localhost:8888/search?q=logging&format=json&categories=packages" | \
  jq 'group_by(.results[].engines[0]) |
      map({engine: .[0].engines[0], packages: map(.title)})'

Academic Research

User: "Find recent papers on transformer architectures"

Agent workflow: 1. Search scientific publications category 2. Filter by date if needed 3. Extract: title, URL, published date 4. Provide links to arXiv/papers

Implementation:

curl -s "http://localhost:8888/search?q=transformer+architecture&format=json&categories=scientific+publications" | \
  jq '.results[] | {title, url, date: .publishedDate, source: .engines}'

Code Examples Search

User: "Show me examples of async/await in Rust"

Agent workflow: 1. Use code category for code search 2. Filter GitHub results 3. Extract repository URLs

Implementation:

curl -s "http://localhost:8888/search?q=rust+async+await&format=json&categories=code" | \
  jq '.results[] | select(.engines[] == "github") | {title, url}'

Best Practices for Agents

1. Always Check If SearXNG is Running

if ! curl -sf http://localhost:8888/ > /dev/null 2>&1; then
  echo "Starting SearXNG..."
  start-searxng --detach
fi

2. Use Appropriate Categories

• Don't use general for package searches - use packages, cargo, etc.
• Use it for broad tech searches
• Use repos specifically for GitHub/GitLab
• Use code for searching within code files

3. Handle Empty Results

RESULTS=$(curl -s "..." | jq '.results | length')
if [ "$RESULTS" -eq 0 ]; then
  echo "No results found. Trying broader search..."
  # Try different category or broader terms
fi

4. Combine with Other Tools

• Use SearXNG to find packages
• Then use package-specific APIs (crates.io, npm) for details
• For PyPI: always use direct API or qypi (PyPI engine doesn't work)

5. Respect Resources

• Don't spam queries in tight loops
• Reuse results when possible
• Stop SearXNG when done with search-intensive tasks:

  podman stop searxng

Error Handling

SearXNG Not Responding

# Check if container is running
podman ps | grep searxng

# Check logs
podman logs searxng

# Restart
podman stop searxng
start-searxng --detach

Empty Results

1. Check unresponsive_engines in response 2. Try broader search terms 3. Try different category 4. Check if specific engine is down

PyPI Not Working

PyPI engine is enabled but returns no results. Always use workaround:

# Option 1: Direct API
curl -s "https://pypi.org/pypi/requests/json" | jq '.info | {name, summary, version}'

# Option 2: qypi CLI
uvx qypi search pandas --json
uvx qypi info requests --json

Performance Tips

Limit Results

# Default returns many results
curl -s "...&format=json" | jq '.results[0:10]'

Parallel Searches

For multiple queries, run in parallel:

curl -s "...cargo..." > cargo.json &
curl -s "...packages..." > packages.json &
wait

Cache Results

Store frequently-used searches:

# Cache popular packages
curl -s "...?q=tokio&categories=cargo" > /tmp/tokio-search.json

Integration Examples

Bash Function

searx_pkg() {
  local query="$1"
  local category="${2:-packages}"
  curl -s "http://localhost:8888/search?q=${query}&format=json&categories=${category}" | \
    jq -r '.results[0:5] | .[] | "\(.title): \(.url)"'
}

# Usage
searx_pkg "express" "packages"
searx_pkg "tokio" "cargo"

Python Integration

import requests

def searxng_search(query, category='general', limit=10):
    resp = requests.get('http://localhost:8888/search', params={
        'q': query,
        'format': 'json',
        'categories': category
    })
    results = resp.json()['results'][:limit]
    return [{'title': r['title'], 'url': r['url'], 'content': r.get('content', '')}
            for r in results]

# Usage
packages = searxng_search('async', category='cargo', limit=5)
for pkg in packages:
    print(f"{pkg['title']}: {pkg['url']}")

Nushell Function

def searx-api [
  query: string,
  --category (-c): string = "general",
  --limit (-l): int = 10
] {
  http get $"http://localhost:8888/search?q=($query | url encode)&format=json&categories=($category)"
  | get results
  | first $limit
  | select title url content engines
}

# Usage
searx-api "serde" --category cargo --limit 5

When to Use SearXNG vs. Built-in Tools

Use SearXNG When:

• Searching package repositories (cargo, npm, etc.)
• Need more than 10 results
• Need to filter by specific engines
• Searching academic papers
• Need full control over search categories
• Rate limits hit on other tools

Use Built-in WebSearch When:

• General web queries
• Current events/news
• Quick fact-checking
• Don't need specialized filtering
• SearXNG not available/running

Cleanup

When done with search tasks:

# Stop SearXNG
podman stop searxng

# Container auto-removes (--rm flag)
# Temp config auto-deleted by OS

SearXNG Category Search Guide

This document details all available categories and which engines serve them.

Available Categories

Get the full list:

curl -s "http://localhost:8888/config" | jq -r '.categories[]'

Current categories (as of testing):

• general
• videos
• social media
• images
• music
• packages ⭐
• it ⭐
• files
• books
• news
• apps
• software wikis
• science
• scientific publications
• web
• repos ⭐
• other
• currency
• weather
• map
• dictionaries
• shopping
• lyrics
• code ⭐
• icons
• cargo ⭐
• movies
• translate
• radio

(⭐ = Most useful for development work)

Development-Focused Categories

1. packages - Multi-Repository Package Search

Engines included:

• npm (JavaScript/Node.js)
• crates.io (Rust)
• hex (Erlang/Elixir)
• hoogle (Haskell)
• metacpan (Perl)
• packagist (PHP/Composer)
• docker hub (Container images)
• alpine linux packages
• lib.rs (Rust alternative registry)
• pypi (Python - configured but not working, see workarounds)

Example:

curl -s "http://localhost:8888/search?q=express&format=json&categories=packages" | \
  jq '.results[] | {title, url, engine: .engines[0], content}'

Use cases:

• Finding packages across multiple ecosystems
• Comparing implementations in different languages
• Discovering container images for tools

2. cargo - Rust Crates Only

Engines included:

• crates.io

Example:

curl -s "http://localhost:8888/search?q=tokio&format=json&categories=cargo" | \
  jq '.results[] | {title, url, content}'

Use cases:

• Finding Rust crates
• Browsing crates.io search results
• Getting crate descriptions

3. it - IT/Tech Resources

Engines included:

• GitHub
• Docker Hub
• Stack Overflow
• crates.io
• GitLab
• And many more tech-focused sources

Example:

curl -s "http://localhost:8888/search?q=kubernetes+helm&format=json&categories=it" | \
  jq '.results[0:5] | .[] | {title, url, engines}'

Use cases:

• Broad tech searches
• Finding GitHub repos, Docker images, and tech docs in one query
• Stack Overflow Q&A

4. repos - Code Repositories

Engines included:

• GitHub
• GitLab
• Codeberg
• Gitea instances

Example:

curl -s "http://localhost:8888/search?q=machine+learning&format=json&categories=repos" | \
  jq '.results[] | select(.engines[] == "github") | {title, url, content}'

Use cases:

• Finding source code repositories
• Discovering open-source projects
• Searching for code examples

5. code - Code Search

Engines included:

• GitHub Code Search
• Sourcehut
• Other code-specific engines

Example:

curl -s "http://localhost:8888/search?q=async+fn+main&format=json&categories=code" | \
  jq '.results[] | {title, url, content}'

Use cases:

• Searching within code files
• Finding specific function implementations
• Discovering code patterns

Research-Focused Categories

scientific publications

Engines included:

• arXiv
• CrossRef
• Google Scholar
• PubMed
• Semantic Scholar
• And more

Example:

curl -s "http://localhost:8888/search?q=neural+networks&format=json&categories=scientific+publications" | \
  jq '.results[0:3] | .[] | {title, url, content, publishedDate}'

science

General science resources and databases.

Example:

curl -s "http://localhost:8888/search?q=quantum+computing&format=json&categories=science"

Multi-Category Searches

Combine categories with commas:

curl -s "http://localhost:8888/search?q=docker&format=json&categories=packages,it,repos" | \
  jq '.results[] | {title, url, engines, category}'

This searches across Docker Hub, GitHub, and other IT resources simultaneously.

Filtering Results by Engine

After searching, filter by specific engine:

# Search packages, filter to npm only
curl -s "http://localhost:8888/search?q=react&format=json&categories=packages" | \
  jq '.results[] | select(.engines[] == "npm")'

# Search IT, filter to GitHub only
curl -s "http://localhost:8888/search?q=rust&format=json&categories=it" | \
  jq '.results[] | select(.engines[] == "github")'

# Search packages, filter to crates.io only
curl -s "http://localhost:8888/search?q=serde&format=json&categories=packages" | \
  jq '.results[] | select(.engines[] == "crates.io")'

Checking Engine Availability

See which engines are configured for a category:

# Check all engines in packages category
curl -s "http://localhost:8888/config" | \
  jq '.engines[] | select(.categories[] | contains("packages")) | .name'

# Check all engines in cargo category
curl -s "http://localhost:8888/config" | \
  jq '.engines[] | select(.categories[] | contains("cargo")) | .name'

Check if specific engine is enabled:

curl -s "http://localhost:8888/config" | \
  jq '.engines[] | select(.name == "pypi")'

Advanced: Engine-Specific Search

Force search using only specific engines:

# Use only npm
curl -s "http://localhost:8888/search?q=typescript&format=json&engines=npm" | \
  jq '.results[]'

# Use only crates.io
curl -s "http://localhost:8888/search?q=async&format=json&engines=crates.io" | \
  jq '.results[]'

Nushell Helpers

# Search packages and group by engine
def search-packages [query: string] {
  http get $"http://localhost:8888/search?q=($query | url encode)&format=json&categories=packages"
  | get results
  | group-by { $in.engines | first }
  | transpose engine results
  | each { |row|
      {
        engine: $row.engine,
        count: ($row.results | length),
        results: ($row.results | select title url content)
      }
    }
}

# Search specific category
def searx-cat [
  query: string,
  category: string,
  --limit (-l): int = 10
] {
  http get $"http://localhost:8888/search?q=($query | url encode)&format=json&categories=($category)"
  | get results
  | first $limit
  | select title url content engines
}

# Multi-category search
def searx-multi [
  query: string,
  categories: list<string>,
  --limit (-l): int = 10
] {
  let cats = ($categories | str join ',')
  http get $"http://localhost:8888/search?q=($query | url encode)&format=json&categories=($cats)"
  | get results
  | first $limit
  | select title url content engines category
}

Usage:

search-packages "express"
searx-cat "tokio" "cargo" --limit 5
searx-multi "docker" ["packages", "it", "repos"] --limit 10

Common Patterns

Finding a package across all ecosystems:

curl -s "http://localhost:8888/search?q=http+client&format=json&categories=packages" | \
  jq '.results | group_by(.engines[0]) | map({engine: .[0].engines[0], packages: map(.title)})'

Tech documentation search:

curl -s "http://localhost:8888/search?q=rust+async+programming&format=json&categories=it" | \
  jq '.results[] | select(.url | contains("doc")) | {title, url}'

Academic research:

curl -s "http://localhost:8888/search?q=transformer+architecture&format=json&categories=scientific+publications" | \
  jq '.results[] | {title, url, date: .publishedDate, content}'

SearXNG Package Search Engine Status

Comprehensive test results for all package repository search engines in SearXNG.

Test date: December 2025 SearXNG version: 2025.10.23+e363db970

Summary

Detailed Results

✅ Haskell - Hoogle (Hackage)

Status: ✅ Working perfectly

Test results:

# Query: aeson
Results: 25 packages/functions
First result: package aeson

Example search:

curl -s "http://localhost:8888/search?q=lens&format=json&engines=hoogle" | jq .

What it returns:

• Package listings
• Function signatures
• Module documentation
• Links to Hackage documentation

Notes:

• Searches both package names and function names
• Returns direct links to Hackage
• Very comprehensive results

---

✅ JavaScript/Node.js - npm

Status: ✅ Working perfectly

Test results:

# Query: express
Results: 25 packages
First result: express

Example search:

curl -s "http://localhost:8888/search?q=react&format=json&engines=npm"

---

✅ Rust - crates.io

Status: ✅ Working perfectly

Test results:

# Query: tokio
Results: 10 crates
First result: tokio

Example search:

curl -s "http://localhost:8888/search?q=serde&format=json&engines=crates.io"

Notes:

• Also accessible via lib.rs engine (alternative Rust registry frontend)

---

✅ Ruby - RubyGems

Status: ✅ Working perfectly

Test results:

# Query: rails
Results: 30 gems
First result: rails 8.1.1

Example search:

curl -s "http://localhost:8888/search?q=sinatra&format=json&engines=rubygems"

---

✅ PHP - Packagist

Status: ✅ Working perfectly

Test results:

# Query: symfony
Results: 15 packages
First result: symfony/yaml

Example search:

curl -s "http://localhost:8888/search?q=laravel&format=json&engines=packagist"

---

✅ Erlang/Elixir - Hex

Status: ✅ Working perfectly

Test results:

# Query: http
Results: 10 packages
First result: cowboy

Example search:

curl -s "http://localhost:8888/search?q=phoenix&format=json&engines=hex"

---

✅ Perl - MetaCPAN

Status: ✅ Working perfectly

Test results:

# Query: http
Results: 20 modules
First result: HTTP

Example search:

curl -s "http://localhost:8888/search?q=mojolicious&format=json&engines=metacpan"

---

✅ Dart/Flutter - pub.dev

Status: ✅ Working perfectly

Test results:

# Query: http
Results: 10 packages

Example search:

curl -s "http://localhost:8888/search?q=flutter&format=json&engines=pub.dev"

---

✅ Go - pkg.go.dev

Status: ✅ Working perfectly

Test results:

# Query: http
Results: 50 packages

Example search:

curl -s "http://localhost:8888/search?q=gin&format=json&engines=pkg.go.dev"

Notes:

• Very comprehensive results (up to 50)
• Official Go package registry

---

✅ Docker - Docker Hub

Status: ✅ Working perfectly

Test results:

# Query: docker
Results: 10 images
First result: docker

Example search:

curl -s "http://localhost:8888/search?q=nginx&format=json&engines=docker+hub"

---

✅ Alpine Linux Packages

Status: ✅ Working perfectly

Test results:

# Query: linux
Results: 50 packages

Example search:

curl -s "http://localhost:8888/search?q=python&format=json&engines=alpine+linux+packages"

---

✅ Void Linux Packages

Status: ✅ Working (enabled by default)

Example search:

curl -s "http://localhost:8888/search?q=vim&format=json&engines=voidlinux"

---

✅ lib.rs (Rust Alternative)

Status: ✅ Working perfectly

Notes:

• Alternative frontend for crates.io
• Provides enhanced search and categorization

---

❌ Python - PyPI

Status: ❌ BROKEN

Issue: PyPI returns JavaScript "Client Challenge" page for bot protection

Details:

• Engine scrapes HTML from https://pypi.org/search/
• PyPI now requires JavaScript to display results
• Parser finds no expected HTML elements
• Returns 0 results (no error shown)
• Reported: SearXNG issue #4093 (December 2024)
• Status: OPEN as of December 2025, no fix yet

Workarounds:

• Use PyPI JSON API: https://pypi.org/pypi/<package>/json (exact name only)
• Use qypi CLI: uvx qypi search <term>

See pypi-direct-search.md for detailed workarounds.

---

Multi-Language Search

You can search across all package repositories at once:

# Search all package repos
curl -s "http://localhost:8888/search?q=http&format=json&categories=packages" | \
  jq '.results | group_by(.engines[0])'

This searches:

• npm (JS)
• crates.io (Rust)
• rubygems (Ruby)
• packagist (PHP)
• hoogle (Haskell)
• hex (Erlang/Elixir)
• metacpan (Perl)
• pub.dev (Dart)
• pkg.go.dev (Go)
• docker hub (containers)
• alpine/void (Linux packages)
• ~~pypi (Python)~~ - broken

Usage Tips

Search Specific Language

# Haskell packages
curl -s "http://localhost:8888/search?q=aeson&format=json&engines=hoogle"

# Ruby gems
curl -s "http://localhost:8888/search?q=rails&format=json&engines=rubygems"

# Go packages
curl -s "http://localhost:8888/search?q=gin&format=json&engines=pkg.go.dev"

Filter Multi-Category Search by Engine

# Search packages category, filter to specific engine
curl -s "http://localhost:8888/search?q=web&format=json&categories=packages" | \
  jq '.results[] | select(.engines[] == "hoogle")'

Nushell Helper Function

def search-pkg [
  query: string,
  language: string  # hoogle, npm, rubygems, etc.
] {
  http get $"http://localhost:8888/search?q=($query | url encode)&format=json&engines=($language)"
  | get results
  | select title url content
}

# Usage
search-pkg "lens" "hoogle"
search-pkg "express" "npm"

Configuration Notes

All engines tested with default SearXNG configuration (use_default_settings: true).

Some engines may be disabled by default in certain configs. To enable:

engines:
  - name: hoogle
    disabled: false

Check current engine status:

curl -s "http://localhost:8888/config" | jq '.engines[] | select(.name == "hoogle")'

Why Most Engines Work

Unlike PyPI, most package registries either: 1. Provide stable HTML structures that haven't changed 2. Offer search APIs that SearXNG uses 3. Don't have aggressive bot protection

Only PyPI added JavaScript challenges that break HTML scraping.

Monitoring for Updates

• PyPI fix: Watch issue #4093
• SearXNG releases: https://github.com/searxng/searxng/releases
• Engine changes: Check searx/engines/ in SearXNG repo

Related Documentation

• category-guide.md - All search categories
• pypi-direct-search.md - PyPI workarounds
• agent-usage.md - AI agent integration guide

PyPI Direct Search

Why SearXNG's PyPI Engine Doesn't Work

Root cause: PyPI now serves a JavaScript "Client Challenge" page for bot protection, breaking SearXNG's HTML scraper.

Details:

• SearXNG's PyPI engine scrapes HTML from https://pypi.org/search/?q=...
• As of late 2024/early 2025, PyPI returns a JS challenge page instead of search results
• The HTML parser can't find expected elements (XPath selectors fail)
• Reported in SearXNG issue #4093 (December 2024)
• Status: OPEN as of December 2025 - no fix yet

The engine needs to be rewritten to use PyPI's JSON API instead of web scraping.

Alternative Methods

Since SearXNG's PyPI engine is broken, use these direct methods:

Method 1: PyPI JSON API

PyPI provides a JSON API for package metadata.

Get Package Info

curl -s "https://pypi.org/pypi/<package>/json"

Response structure:

{
  "info": {
    "name": "package-name",
    "version": "1.2.3",
    "summary": "Package description",
    "description": "Long description (often in markdown)",
    "author": "Author Name",
    "author_email": "author@example.com",
    "home_page": "https://github.com/...",
    "license": "MIT",
    "keywords": "keyword1, keyword2",
    "classifiers": [...],
    "requires_python": ">=3.8",
    ...
  },
  "urls": [...],  // Download URLs for wheels, sdist, etc.
  "releases": {...},  // All versions
  "vulnerabilities": [...]
}

Extract Specific Fields

# Get summary
curl -s "https://pypi.org/pypi/requests/json" | jq '.info.summary'

# Get version
curl -s "https://pypi.org/pypi/requests/json" | jq '.info.version'

# Get homepage
curl -s "https://pypi.org/pypi/requests/json" | jq '.info.home_page'

# Check Python version requirement
curl -s "https://pypi.org/pypi/requests/json" | jq '.info.requires_python'

Nushell Helper

def pypi [package: string] {
  http get $"https://pypi.org/pypi/($package)/json"
  | get info
  | select name version summary home_page license requires_python
}

# Usage
pypi requests

Method 2: qypi CLI

Install via uvx:

uvx qypi search <term>
uvx qypi info <package>

Search

# Search with JSON output
uvx qypi search pandas --json

# Search with boolean operators
uvx qypi search --and machine learning --json
uvx qypi search --or pandas numpy --json

# Search for packages or releases
uvx qypi search --packages scikit --json
uvx qypi search --releases torch --json

Output format:

[
  {
    "name": "package-name",
    "version": "1.2.3",
    "summary": "Description"
  },
  ...
]

Get Package Info

uvx qypi info requests --json

Output includes:

• Name, version, summary
• Author, maintainer
• License
• Homepage, documentation, source URLs
• Dependencies
• Keywords, classifiers

List Releases

uvx qypi releases flask --json

Returns all available versions of a package.

List Package Owners

uvx qypi owner requests --json

Method 3: pip search Alternative (Disabled)

pip search has been disabled by PyPI since 2021 due to abuse. Use the methods above instead.

Comparison

Troubleshooting SearXNG PyPI (Archived)

Note: These troubleshooting steps won't fix the PyPI engine because the issue is on PyPI's side (bot protection), not SearXNG's.

If you want to verify the issue yourself:

1. Test PyPI search page directly:

   curl -s "https://pypi.org/search/?q=requests" | head -50

You'll see a "Client Challenge" page requiring JavaScript, not search results.

2. Check SearXNG logs (no errors will appear):

   podman logs searxng | grep -i pypi

3. The engine won't be listed as "unresponsive" because it successfully fetches the page - it just can't parse it:

   curl -s "http://localhost:8888/search?q=test&engines=pypi&format=json" | jq '.unresponsive_engines'
   # Returns: []

Monitoring for a Fix

Watch SearXNG issue #4093 for updates. The fix will likely involve:

• Rewriting the engine to use PyPI's JSON API (https://pypi.org/pypi/<package>/json)
• Removing HTML scraping entirely
• Possibly requiring package name exact match instead of search

Configuration Location

Your SearXNG config: /home/ypares/Config/.config/searxng/settings.yml

PyPI engine config (lines 1742-1744):

- name: pypi
  shortcut: pypi
  engine: pypi

To modify engine settings, add parameters like:

- name: pypi
  shortcut: pypi
  engine: pypi
  timeout: 10.0  # Increase timeout
  disabled: false  # Ensure it's enabled

Then restart SearXNG container.