Api Load Tester

API Load Tester

Stress-test HTTP endpoints under increasing load, identify breaking points, and produce a report with actionable recommendations.

Contents

• references/tool-commands.md -- tool invocations (hey/wrk/ab/curl), default concurrency stages, per-stage data to capture.
• references/metrics-interpretation.md -- latency, throughput, error, breaking-point, and bottleneck classification.
• references/output-template.md -- exact structure for api-load-report.md, including ASCII charts and scaling table.
• references/rules-and-examples.md -- safety rules, error handling, and example invocations.

Inputs

Collect from the user. Ask before proceeding if a required input is missing.

Required: endpoint URL(s) (with method, headers, body as needed); expected latency thresholds. Default thresholds if unspecified: p50 < 100ms, p95 < 300ms, p99 < 1000ms.

Optional: concurrent users or range (default ramp 1 to 100); authentication; request payloads; custom headers; test duration (default 10s per stage); ramp pattern (default step ramp, doubling each stage); success criteria (default 2xx); known rate limits; environment label (prod/staging/dev).

Workflow

Follow these steps in order.

1. Select a tool. Check in priority order: which hey, which wrk, which ab, which curl. If none of hey/wrk/ab exist, install hey (brew install hey on macOS, go install github.com/rakyll/hey@latest on Linux with Go) or fall back to curl with bash background processes and wait. Verify with a single trivial request against a provided endpoint; diagnose connectivity or auth before continuing.

2. Validate endpoints. Send one request per endpoint with the specified method, headers, auth, and body. Confirm the status matches the success criteria and record baseline single-request latency. On failure, surface the error and ask whether to skip or fix.

3. Design the test plan. Build progressive concurrency stages (see references/tool-commands.md for the default progression), trimming or extending to the user's concurrency range. Define per-endpoint method, URL, headers, body, success codes, and timeout (default 30s). Print the plan for review before executing.

4. Execute stages. For each endpoint, run every concurrency stage sequentially with the selected tool, waiting 2 seconds between stages. Capture and store the per-stage metrics. See references/tool-commands.md for commands, request-count formula, and the metrics list.

5. Interpret metrics. Compute latency percentiles and profile, throughput curve and ceiling, error rates and onset, the breaking point, and the bottleneck classification. See references/metrics-interpretation.md.

6. Generate the report. Write api-load-report.md to the current working directory following references/output-template.md exactly, including ASCII throughput and latency charts.

7. Post-report actions. Print a 3-5 line summary to the console, state the report path, explicitly highlight any critical issues, and offer to re-run specific stages with different parameters.

Rules

Apply the safety rules, error handling, and example invocations in references/rules-and-examples.md. Key constraints: never load-test production without explicit confirmation, only test GET by default, mask auth tokens, respect 429 rate limits, count timeouts as failures, and never extrapolate beyond tested ranges.

Metrics Interpretation

After all stages complete, interpret the collected metrics as follows.

Latency

For each endpoint, compute:

• Latency by percentile: p50, p75, p90, p95, p99 at each concurrency level.
• Latency trend: How median latency changes as concurrency increases. Compute the slope.
• Latency stability: Standard deviation at each stage. Flag stages where stddev > 2x the median.
• Latency threshold violations: At which concurrency level each percentile exceeded the target.

Classify the latency profile:

• Flat: Latency stays within 20% of baseline up to max concurrency. Excellent.
• Linear degradation: Latency increases proportionally with concurrency. Acceptable up to a point.
• Exponential degradation: Latency increases faster than concurrency. Bottleneck detected.
• Cliff: Latency suddenly spikes at a specific concurrency level. Hard limit found.

Throughput

For each endpoint, compute:

• Peak throughput: Maximum requests/second achieved and at which concurrency level.
• Throughput ceiling: The concurrency level where adding more users no longer increases throughput (saturation point).
• Throughput curve shape: Linear growth, logarithmic growth, or plateau.
• Efficiency ratio: Throughput per concurrent user at each stage.

Errors

For each endpoint, compute:

• Error rate by stage: Percentage of non-success responses at each concurrency level.
• Error onset: The concurrency level where errors first appear above 0.1%.
• Error types: Categorize into timeout, connection refused, 4xx, 5xx, and other.
• Error rate trend: Stable, growing linearly, or growing exponentially.

Breaking Point

The breaking point is the concurrency level where ANY of the following first occurs. State it clearly and name which condition triggered it.

1. Error rate exceeds 1%. 2. p95 latency exceeds 5x the baseline single-user p95. 3. Throughput decreases compared to the previous stage (throughput cliff). 4. More than 5% of connections are refused or reset.

Bottleneck Classification

Classify the likely bottleneck and provide supporting evidence from the data.

• CPU-bound: Latency increases linearly, throughput plateaus, no connection errors.
• Memory-bound: Latency is stable then suddenly spikes, often with connection resets.
• I/O-bound (database): Latency variance is high, throughput has a hard ceiling, errors are timeouts.
• I/O-bound (network): Connection refused errors, high timeout rate, latency spikes correlate with error spikes.
• Connection pool exhaustion: Sudden onset of connection errors at a specific concurrency level, latency cliff.
• Rate limiting: Consistent 429 status codes above a threshold, latency stable but errors spike.
• Thread/process pool exhaustion: Throughput plateaus, latency grows linearly, no errors until a hard cliff.

Output Template

Create the file api-load-report.md in the current working directory using this exact structure.

# API Load Test Report

**Date**: <YYYY-MM-DD HH:MM:SS timezone>
**Environment**: <prod/staging/dev or as specified>
**Tool**: <hey/wrk/ab/curl>
**Test Duration**: <total wall-clock time>

---

## Executive Summary

<2-3 sentences summarizing the overall findings. State the key throughput number, the breaking point, and the most critical recommendation.>

---

## Endpoints Tested

| # | Method | URL | Auth | Payload |
|---|--------|-----|------|---------|
| 1 | GET | https://... | Bearer | N/A |
| 2 | POST | https://... | Bearer | JSON (245 bytes) |

---

## Test Configuration

- **Concurrency stages**: <list of concurrency levels>
- **Duration per stage**: <seconds>
- **Total requests per stage**: <number>
- **Request timeout**: <seconds>
- **Success criteria**: <status codes>
- **Ramp pattern**: <step/linear/spike>

---

## Results by Endpoint

### Endpoint 1: <METHOD> <URL>

#### Latency Percentiles (ms)

| Concurrency | p50 | p75 | p90 | p95 | p99 | Max |
|-------------|-----|-----|-----|-----|-----|-----|
| 1 | ... | ... | ... | ... | ... | ... |
| 5 | ... | ... | ... | ... | ... | ... |
| ... | ... | ... | ... | ... | ... | ... |

#### Throughput

| Concurrency | Req/sec | Transfer (KB/s) | Avg Latency (ms) | Error Rate (%) |
|-------------|---------|------------------|-------------------|----------------|
| 1 | ... | ... | ... | ... |
| 5 | ... | ... | ... | ... |
| ... | ... | ... | ... | ... |

#### Error Breakdown

| Concurrency | 2xx | 4xx | 5xx | Timeout | Conn Error | Total Errors |
|-------------|-----|-----|-----|---------|------------|-------------|
| 1 | ... | ... | ... | ... | ... | ... |
| ... | ... | ... | ... | ... | ... | ... |

#### Latency Profile

<Classify as Flat / Linear / Exponential / Cliff with supporting data>

#### Breaking Point

<State the breaking point concurrency, which condition triggered it, and the specific metric values>

---

<Repeat for each endpoint>

---

## Comparative Analysis

<If multiple endpoints were tested, compare their performance profiles. Identify which endpoints are the weakest links.>

| Endpoint | Peak RPS | Breaking Point | Bottleneck Type | p95 at Peak |
|----------|----------|---------------|-----------------|-------------|
| GET /health | ... | ... | ... | ... |
| POST /search | ... | ... | ... | ... |

---

## Throughput Curves (ASCII)

<For each endpoint, render an ASCII chart showing throughput vs concurrency>

Throughput (req/s) ^ 800 | -------- | 600 | | 400 | | 200 | | 0 +--+--+--+--+--+--+--> Concurrency 1 5 10 25 50 100 200

---

## Latency Distribution (ASCII)

<For each endpoint, render an ASCII chart showing p50/p95/p99 vs concurrency>

Latency (ms) ^ 1000 | p99 | 500 | * o p95 | o o 200 |o o . . . . . p50 100 |. . . 0 +--+--+--+--+--+--+--+--> Concurrency 1 5 10 25 50 100 200 500

---

## Bottleneck Analysis

### Primary Bottleneck

<Classification (CPU/Memory/IO/Connection Pool/Rate Limit/Thread Pool) with 3-5 bullet points of supporting evidence from the test data>

### Secondary Observations

<Any additional patterns observed, such as:>
- Garbage collection pauses (periodic latency spikes)
- DNS resolution overhead
- TLS handshake cost at high concurrency
- Keep-alive vs connection-per-request behavior
- Response body size variation under load

---

## Recommendations

### Critical (Address Immediately)

1. **<Recommendation title>**: <Detailed explanation with specific numbers from the test. E.g., "Add connection pooling -- connection errors begin at 50 concurrent users, suggesting the server is opening a new database connection per request. A pool of 20-30 connections should handle up to 200 concurrent users based on the observed throughput ceiling.">

2. **<Recommendation title>**: <...>

### Important (Address Before Scaling)

3. **<Recommendation title>**: <...>

4. **<Recommendation title>**: <...>

### Nice to Have (Optimization)

5. **<Recommendation title>**: <...>

6. **<Recommendation title>**: <...>

---

## Capacity Estimate

Based on the observed performance profile:

- **Current safe operating capacity**: <X concurrent users> (<Y req/sec>)
- **Maximum tested capacity**: <X concurrent users> (<Y req/sec, Z% error rate>)
- **Estimated capacity with recommended fixes**: <X concurrent users> (projected)

### Scaling Projections

| Target Users | Current Status | After Fixes | Additional Infra Needed |
|-------------|---------------|-------------|------------------------|
| 50 | OK | OK | None |
| 100 | Degraded (p95 > target) | OK (projected) | None |
| 500 | Breaking point | OK (projected) | Add replica |
| 1000 | Not viable | Marginal | Load balancer + 3 replicas |

---

## Methodology Notes

- Tool: <name and version>
- Each concurrency stage ran for <N> seconds with a <N>-second cooldown between stages
- Latency measurements include full round-trip time (DNS + connect + TLS + TTFB + transfer)
- All tests were run from <location/machine description>
- Results may vary based on network conditions, server load, and time of day
- For production capacity planning, repeat tests at different times and from multiple geographic locations

---

## Raw Data Reference

Raw output files are stored in: `<temp_directory_path>`

<List the files with brief descriptions>

Rules, Error Handling, and Examples

Safety and Conduct Rules

1. Never test production endpoints without explicit user confirmation. If the environment is "prod" or the URL contains "prod", "production", or appears to be a production domain, warn the user and ask for confirmation before proceeding. 2. Respect rate limits. If 429 responses are detected, reduce concurrency and note the rate limit. Do not continue hammering an endpoint that is returning 429s. 3. Handle authentication carefully. Never log or include full auth tokens. Mask them (e.g., "Bearer eyJ...**"). 4. No destructive testing by default. Only test GET endpoints by default. For POST/PUT/DELETE, confirm the endpoint is safe to call repeatedly (idempotent, uses a test database, or has no side effects). 5. Clean up temporary files. Store raw results in a clearly named temp directory but do not delete them automatically; the user may want to inspect them. 6. Report in consistent units. Use milliseconds for latency, requests/second for throughput, and percentages for error rates. Always label units. 7. ASCII charts are mandatory in the output. Even though approximate, they give immediate visual understanding without external tools. 8. Test from the same machine consistently. Do not distribute load across machines unless the user specifically asks for distributed testing. 9. Timeouts count as failures. A timed-out request is a failed request, not excluded from the data. 10. Do not extrapolate beyond tested ranges.** The scaling projections table must clearly mark projected values vs observed values.

Error Handling

• If a tool installation fails, fall back to the next tool in the priority list. If all preferred tools fail, use the curl fallback.
• If an endpoint becomes completely unresponsive during testing (100% timeout for 30+ seconds), stop testing that endpoint at that concurrency level and move to the next stage or endpoint. Note this in the output as "endpoint became unresponsive."
• If the machine runs out of file descriptors or hits OS-level connection limits, detect the error message, surface it, and suggest increasing ulimit -n before retrying.
• If the run is interrupted (Ctrl+C or timeout), save whatever data has been collected so far and generate a partial output clearly marked as incomplete.

Example Invocations

Simple single endpoint:

Load test https://api.example.com/health
Expected response time: p95 < 200ms
Concurrent users: up to 100

Multiple endpoints with auth:

Endpoints:
  - GET https://api.example.com/users (Bearer token: abc123)
  - POST https://api.example.com/search (Bearer token: abc123, body: {"query": "test"})
Expected: p95 < 300ms
Concurrency: 10 to 500
Environment: staging

Quick smoke test:

Quick load test https://api.example.com/health with 50 concurrent users

For quick/smoke tests, reduce to 3 stages: baseline (1), target concurrency (50), and 2x target (100). Shorten duration to 5 seconds per stage.

Tool Commands

Run each concurrency stage with the best available tool. Wait 2 seconds between stages so the server stabilizes and carryover effects are avoided.

hey (preferred)

hey -n <total_requests> -c <concurrency> -t <timeout> \
    -m <METHOD> \
    -H "Authorization: Bearer <token>" \
    -H "Content-Type: application/json" \
    -d '<body>' \
    <url>

Calculate total requests as concurrency * (duration / estimated_response_time), with a minimum of concurrency * 10 requests per stage.

wrk

wrk -t <threads> -c <concurrency> -d <duration>s \
    -s <lua_script> \
    <url>

Generate a Lua script when custom methods, headers, or bodies are needed.

ab (Apache Bench)

ab -n <total_requests> -c <concurrency> -t <timeout> \
    -H "Authorization: Bearer <token>" \
    -T "application/json" \
    -p <body_file> \
    <url>

curl fallback

for i in $(seq 1 $CONCURRENCY); do
  (for j in $(seq 1 $REQUESTS_PER_USER); do
    curl -o /dev/null -s -w "%{http_code} %{time_total}\n" \
      -X <METHOD> \
      -H "Authorization: Bearer <token>" \
      -H "Content-Type: application/json" \
      -d '<body>' \
      <url>
  done) &
done
wait

Default concurrency stages

Adjust to the user-specified concurrency range. If the user specifies a max of 50, stop there. If they specify a max of 1000, add stages beyond 500.

Per-stage data to capture

• Total requests sent
• Successful responses (by status code)
• Failed responses (by status code or error type)
• Latency: min, max, mean, median (p50), p90, p95, p99
• Requests per second (throughput)
• Transfer rate (bytes/sec if available)
• Connection errors, timeouts, and resets
• Stage start and end timestamps

Store raw results in a temp directory like /tmp/api-load-test-<timestamp>/, one file per stage: raw_<endpoint_name>_c<concurrency>.txt.