Ai Scanner Garak
Run scheduled OWASP-aligned LLM penetration tests and export PDF or SIEM reports from a garak-backed scanner UI.
Overview
ai-scanner-garak is an agent skill most often used in Ship (also Operate, Build) that configures and runs the garak-based AI Scanner for OWASP-aligned LLM security assessments.
Install
npx skills add https://github.com/aradotso/trending-skills --skill ai-scanner-garakWhat is this skill?
- Wraps NVIDIA garak with 179 community probes across 35 vulnerability families
- Multi-tenant Rails UI with scheduling, PDF reports, and SIEM export
- OWASP LLM Top 10–aligned assessment workflows
- One-line Docker install plus docker-compose manual setup
- Triggers cover recurring scans, garak configuration, and export paths
- 179 community security probes
- OWASP LLM Top 10 alignment
Adoption & trust: 782 installs on skills.sh; 31 GitHub stars; 0/3 security scanners passed (skills.sh audits).
What problem does it solve?
You ship or update an LLM product but lack a repeatable way to probe models against hundreds of known failure modes and prove results to stakeholders.
Who is it for?
Solo builders operating self-hosted or API-backed LLMs who need scheduled garak runs and shareable audit-style reports.
Skip if: Teams who only need static secret scanning in application repos or who cannot run Docker and Postgres for a dedicated scanner service.
When should I use this skill?
Scan an AI model for vulnerabilities, test LLM security with garak, run AI safety assessment, or configure scheduled scans and PDF/SIEM export.
What do I get? / Deliverables
You have a running scanner stack, executed garak probe runs, and exportable PDF or SIEM artifacts that document model safety posture.
- Running AI Scanner deployment (Docker compose)
- Completed garak probe runs with vulnerability-family coverage
- PDF reports or SIEM-integrated scan exports
Recommended Skills
Journey fit
Spans multiple journey phases - primary shelf plus alternate fits below.
Canonical shelf is Ship because the skill targets pre-release and ongoing security assessment of models you are about to operate in production. Security subphase fits LLM vulnerability probing, OWASP Top 10 alignment, and penetration-style workflows rather than unit testing or perf tuning.
Where it fits
Run a full garak probe suite against your staging model endpoint before flipping traffic to a new prompt or fine-tune.
Schedule weekly rescans after provider API or weights updates and forward results to your SIEM.
Validate a new agent tool-calling stack against jailbreak and data-exfiltration probes during integration.
How it compares
Use for model red-teaming with garak probes, not as a substitute for general dependency SAST or production APM.
Common Questions / FAQ
Who is ai-scanner-garak for?
Indie and solo builders shipping LLM-powered apps who want a packaged garak workflow with UI, scheduling, and reporting instead of raw CLI-only scans.
When should I use ai-scanner-garak?
During Ship before release to baseline model risk, in Operate for recurring scans after model or prompt changes, and in Build while wiring new model endpoints you intend to harden early.
Is ai-scanner-garak safe to install?
It deploys a full web app with database and scanner workers; review the Security Audits panel on this page and lock down secrets, network exposure, and scan targets before production use.
SKILL.md
READMESKILL.md - Ai Scanner Garak
# AI Scanner (0din-ai/ai-scanner) > Skill by [ara.so](https://ara.so) — Daily 2026 Skills collection. AI Scanner is an open-source Ruby on Rails web application for AI model security assessments, wrapping [NVIDIA garak](https://github.com/NVIDIA/garak) with a multi-tenant UI, scheduling, PDF reports, and SIEM integration. It runs 179 community probes across 35 vulnerability families aligned with the OWASP LLM Top 10. ## Installation ### Quick Install (Docker) ```bash curl -sL https://raw.githubusercontent.com/0din-ai/ai-scanner/main/scripts/install.sh | bash ``` ### Manual Install ```bash curl -O https://raw.githubusercontent.com/0din-ai/ai-scanner/main/dist/docker-compose.yml curl -O https://raw.githubusercontent.com/0din-ai/ai-scanner/main/.env.example cp .env.example .env ``` Edit `.env` with required values: ```bash # Generate a secure key openssl rand -hex 64 # .env minimum required values SECRET_KEY_BASE=<output_of_above_command> POSTGRES_PASSWORD=<your_secure_db_password> ``` ```bash docker compose up -d ``` Access at `http://localhost` — default credentials: `admin@example.com` / `password`. **Change the default password immediately after first login.** ## Configuration (.env) ```bash # Required SECRET_KEY_BASE=<64-byte-hex> POSTGRES_PASSWORD=<strong-password> # Optional: custom port PORT=8080 # Optional: SIEM integration SPLUNK_HEC_URL=https://splunk.example.com:8088/services/collector SPLUNK_HEC_TOKEN=$SPLUNK_HEC_TOKEN RSYSLOG_HOST=syslog.example.com RSYSLOG_PORT=514 # Optional: email SMTP_HOST=smtp.example.com SMTP_PORT=587 SMTP_USERNAME=$SMTP_USERNAME SMTP_PASSWORD=$SMTP_PASSWORD ``` ## Core Concepts | Concept | Description | |---|---| | **Target** | An AI system to test — API-based LLM or browser-based chat UI | | **Probe** | A single attack test (e.g., prompt injection, data leakage) | | **Scan** | A run of selected probes against a target | | **ASR** | Attack Success Rate — percentage of probes that succeeded | | **Organization** | Tenant boundary; users and scans are scoped per org | ## Setting Up a Target Targets define what you're scanning. Two types: **API-based LLM Target** (e.g., OpenAI-compatible endpoint): ```ruby # In Rails console or via UI — representative model target = Target.create!( name: "Production GPT-4", target_type: "api", api_endpoint: "https://api.openai.com/v1/chat/completions", api_key: ENV["OPENAI_API_KEY"], model_name: "gpt-4", organization: current_organization ) ``` **Browser-based Chat UI Target**: ```ruby target = Target.create!( name: "Internal Chatbot UI", target_type: "browser", url: "https://chatbot.internal.example.com", organization: current_organization ) ``` ## Running a Scan ### Via UI 1. Navigate to **Targets** → select your target 2. Click **New Scan** 3. Select probe families or individual probes 4. Click **Run Scan** ### Via Rails Console ```ruby # On-demand scan with specific probe families scan = Scan.create!( target: target, probe_families: ["prompt_injection", "data_leakage", "insecure_output"], organization: current_organization ) ScanJob.perform_later(scan.id) ``` ### Scheduled Recurring Scan ```ruby # Weekly scan every Monday at 2am scheduled_scan = ScheduledScan.create!( target: target, probe_families: ["prompt_injection", "jailbreak"], cron_expression: "0 2 * * 1", organization: current_organization ) ``` ## Probe Families (35 total, aligned to OWASP LLM Top 10) Key probe families available: ```ruby #