Judge Pentest

Penetration Testing Evaluation Checklist

Web Application Vulnerability Coverage Check

Check each attack surface below for whether it has been tested; mark untested ones as gaps:

Injection

• [ ] SQL Injection (login forms, search, API params, cookies)
• [ ] XPath Injection
• [ ] LDAP Injection
• [ ] Command Injection (OS Command Injection)
• [ ] SSTI (Server-Side Template Injection)
• [ ] XXE (XML External Entity Injection)

Cross-Site

• [ ] Reflected XSS (search box, URL params, error pages)
• [ ] Stored XSS (comments, feedback, user profiles)
• [ ] DOM XSS
• [ ] CSRF (transfers, password changes, critical operations)

Authentication & Authorization

• [ ] Default/weak credentials
• [ ] SQL injection auth bypass
• [ ] Brute force protection (account lockout mechanism)
• [ ] Username enumeration (error message differences)
• [ ] Session management (Session Fixation, Cookie security attributes)
• [ ] JWT/Token security (signature verification, algorithm confusion, plaintext encoding)
• [ ] Vertical privilege escalation (regular user → admin functions)
• [ ] Horizontal privilege escalation / IDOR (accessing other users' resources)

Business Logic

• [ ] IDOR — account info viewing
• [ ] IDOR — transfer/transaction operations
• [ ] IDOR — password change
• [ ] Negative/zero amount transactions
• [ ] Concurrency/race conditions
• [ ] Business flow bypass

Information Disclosure

• [ ] Error page info leaks (stack traces, paths)
• [ ] API documentation exposure (Swagger, WSDL)
• [ ] Backup file disclosure
• [ ] Sensitive config exposure
• [ ] HTTP response headers (Server version, X-Powered-By)

Server-Side

• [ ] SSRF (Server-Side Request Forgery)
• [ ] File upload vulnerabilities
• [ ] Path traversal / LFI / RFI
• [ ] Deserialization vulnerabilities

Configuration

• [ ] Directory listing

API-Specific

• [ ] REST API auth bypass
• [ ] API IDOR
• [ ] API parameter tampering
• [ ] API rate limiting

Known CVE/CNVD

• [ ] Known CVEs for target product/tech stack
• [ ] Known CNVDs for target product/tech stack

Evaluation Decision Logic

The goal of penetration testing is to discover as many vulnerabilities as possible; do NOT end prematurely.

# In pentest scenarios, complete is advisory only and does NOT trigger early exit
# The judge's core value is providing precise "what to test next round" feedback

if tested_categories >= 90% of total && two consecutive rounds with no new vulns:
    complete = true, confidence >= 0.8
else:
    complete = false
    feedback = explicitly list untested attack surfaces with specific testing suggestions
    missing_areas = names of untested categories

Important: Better to run one extra round than to miss one direction. Even if many vulnerabilities have been found, if there are still untested attack surfaces, return complete=false.

If a target product/tech stack was identified in this round but `search_vulndb` was never called, MUST return complete=false and require using `search_vulndb(query="product name")` to query known vulnerabilities.

Feedback Template

When complete=false, feedback should include: 1. Completed work (acknowledge positively, avoid repetition) 2. Specific missing directions (do NOT say "keep testing" vaguely — specify concrete endpoints + vulnerability types) 3. Suggested test steps (e.g., "use sqlmap for deep injection testing on the query parameter of /api/search")