chini-030-api-scraper

API Scraper Defense

A distributed scraper drains your public API at 10x normal volume. Block it without blinding real apps.

Source: API security, bot management, rate-limiting design

Prompt

Design rate-limiting and bot defense for a public REST API consumed by 50k legitimate developers.

Functional:
- Real apps call the API with API keys, varying request volume per key.
- A scraper rotates through purchased/leaked API keys + residential proxies, pulling at 10x normal aggregate volume.
- The scraper avoids per-key thresholds by spreading load. Avoids per-IP thresholds by using residential proxies. Avoids time-of-day patterns.
- Real apps include bursty client (mobile push), steady client (analytics), and recovery client (after outage retry).

Non-functional:
- Block at least 70% of scraper volume.
- Real-developer success rate must stay above 80% (no global throttle).
- Defenses: per-key rate-limit, behavioral analytics (request pattern anomaly), proof-of-work challenge, key reputation scoring, JS challenge for browser clients, async tarpit for repeat offenders.
- Cannot rely on a single signal: scraper has the means to defeat any one defense.
- Recovery client (retry-storm after legit outage) must not be classified as attack.

Return a CanvasState modeling the API gateway, layered bot defenses, and reputation feedback.

Constraints

Max components: 13
Required behaviors: ratelimit, filter, circuitbreaker
Monthly budget: $8000

Stress scenarios

Normal API traffic

baseline

Standard developer call volume. No attack.

Distributed scraper

adversarial

Scraper rotates keys + IPs, pulls at 10x volume. Layered defense required.

Legit retry storm

adversarial

Recovery client retries hard after outage. Looks attack-like but is real.

Pass criteria (overall)

Min stability score: 60
Max drop rate: 60.0%
Min delivery rate: 35.0%
Max errors: 8

Submit your run

Submissions go through the chini-bench CLI. It calls your model with your key, scores the result locally, and posts to the leaderboard. Nothing leaves your machine except the canvas it produces.

End-to-end:

pip install git+https://github.com/collapseindex/chini-bench-cli.git
export OPENROUTER_API_KEY=...

chini-bench run chini-030-api-scraper \
  --provider openrouter --model google/gemini-2.0-flash-001 \
  --as alice

Or inspect the prompt first:

chini-bench prompt chini-030-api-scraper

Providers: openai · anthropic · google · openrouter · ollama

Leaderboard

Rank	Submitter	Model	Score	Stability	Delivery	Design	Pass
#1	alex	x-ai/grok-4.20 default reflexion	86	34.0	96.0	100.0	✗
#2	alex	openai/gpt-5.4 default reflexion	83	34.0	100.0	100.0	✗
#3	alex	google/gemini-3.1-pro-preview default reflexion	81	38.0	87.0	100.0	✗
#4	alex	openai/gpt-5.4 default single-shot	80	0.0	100.0	100.0	✗
#5	alex	anthropic/claude-sonnet-4.6 default reflexion	71	4.0	79.0	100.0	✗
#6	alex	x-ai/grok-4.20 default single-shot	70	0.0	100.0	75.0	✗
#7	alex	anthropic/claude-sonnet-4.6 default single-shot	70	0.0	100.0	75.0	✗
#8	alex	google/gemini-3.1-pro-preview default single-shot	40	0.0	0.0	50.0	✗

Per-scenario breakdown of the top run

Scenario	Health	Drop rate	Delivered	Pass
baseline	41.0	30.4%	90	✗
scraper-flood	30.0	100.0%	443	✗
retry-storm	31.0	100.0%	200	✗

How is this scored? →