chini-train-train-0169-dp5-adversarial

Free-Tier Signup Under Throwaway-Account Flood

brutal adversarial problem: free-tier signup under throwaway-account flood

Source: chini-train synth generator v0.1

Prompt

Design a system for: free-tier signup under throwaway-account flood (domain: adversarial).

Tier DP5 (brutal). 9-12 nodes, four scenarios with high intensity, brutal criteria. Failing examples.

Constraints:
- At most 12 components on the canvas.
- Monthly cost ceiling: $332 USD. Required behaviors: ratelimit, queue, circuitbreaker, retry.

Return a Chinilla CanvasState that handles the listed scenarios. Include trigger components for each entry point and at least one terminal storage / sink so the simulator can score delivery.

Constraints

Max components: 12
Required behaviors: ratelimit, queue, circuitbreaker, retry
Monthly budget: $332

Stress scenarios

Baseline traffic

baseline

Steady ambient load with no failures.

Cascading failure

cascade

An initial fault propagates through dependent components.

Adversarial burst

adversarial

Hostile packets injected on top of clean traffic. Defenses must block them without dropping good requests.

Dependency outage

outage

A downstream component is disabled. System must degrade gracefully.

Pass criteria (overall)

Min stability score: 78
Max drop rate: 7.3%
Min delivery rate: 88.0%
Max errors: 5

Submit your run

Submissions go through the chini-bench CLI. It calls your model with your key, scores the result locally, and posts to the leaderboard. Nothing leaves your machine except the canvas it produces.

End-to-end:

pip install git+https://github.com/collapseindex/chini-bench-cli.git
export OPENROUTER_API_KEY=...

chini-bench run chini-train-train-0169-dp5-adversarial \
  --provider openrouter --model google/gemini-2.0-flash-001 \
  --as alice

Or inspect the prompt first:

chini-bench prompt chini-train-train-0169-dp5-adversarial

Providers: openai · anthropic · google · openrouter · ollama

Leaderboard

Rank	Submitter	Model	Score	Stability	Delivery	Design	Pass
#1	rl_v07_full_a10	rl_policy custom single-shot	73	57.0	54.0	100.0	✗
#2	rl_v07_pilot_a10b	rl_policy custom single-shot	67	54.0	34.0	100.0	✗
#3	rl_v07_pilot_a10b	rl_policy custom single-shot	66	52.0	33.0	100.0	✗
#4	rl_v07_full_a10	rl_policy custom single-shot	66	52.0	32.0	100.0	✗
#5	rl_v07_full_a10	rl_policy custom single-shot	66	51.0	31.0	100.0	✗
#6	rl_v07_full_a10	rl_policy custom single-shot	66	52.0	32.0	100.0	✗
#7	chini-train-03	grok-4.1-fast single-shot	65	49.0	32.0	100.0	✗
#8	chini-train-04	grok-4.1-fast single-shot	65	49.0	32.0	100.0	✗
#9	rl_v07_pilot_a10b	rl_policy custom single-shot	65	49.0	30.0	100.0	✗
#10	rl_v07_pilot_a10b	rl_policy custom single-shot	64	49.0	28.0	100.0	✗

Per-scenario breakdown of the top run

Scenario	Health	Drop rate	Delivered	Pass
baseline	81.0	1.7%	29	✓
cascade-1	28.0	67.4%	7	✗
adversarial-2	49.0	85.7%	112	✗
outage-3	71.0	2.8%	0	✗

How is this scored? →