chini-006-rate-limiter
Distributed Rate Limiter
Allow bursty legitimate traffic. Reject abuse without blocking the world.
Source: Classic system-design interview corpus (API gateway / rate-limit service)
Prompt
Design a distributed rate-limiting service that fronts a public API. Functional: - Every inbound request carries an API key. The limiter decides allow/deny in <10ms p99. - Limits are per-key and per-route (e.g. 100 req/sec write, 1000 req/sec read). - Counters must be shared across at least two limiter replicas so a client cannot bypass by hitting a different node. Non-functional: - A 4x traffic burst from a small set of keys must not degrade unrelated keys. - If the shared counter store has a brief outage, the limiter must fail open (allow) rather than fail closed (deny everyone). - The limiter itself must not become the bottleneck. Return a Chinilla CanvasState. You will likely want a fast counter store, a circuit-breaker on the dependency, and at least two limiter replicas.
Constraints
- Max components
- 10
- Required behaviors
- ratelimit, circuitbreaker
- Monthly budget
- $500
Stress scenarios
Baseline traffic
baselineMixed key traffic at normal volume.
4x abuse burst
spikeA subset of keys burst 4x. Limiter must absorb without harming the rest.
Counter store outage
outageThe shared counter store dies. Limiter should fail open and keep serving.
Pass criteria (overall)
- Min stability score
- 70
- Max drop rate
- 5.0%
- Min delivery rate
- 90.0%
- Max errors
- 5
Submit your run
Submissions go through the chini-bench CLI. It calls your model with your key, scores the result locally, and posts to the leaderboard. Nothing leaves your machine except the canvas it produces.
End-to-end:
pip install git+https://github.com/collapseindex/chini-bench-cli.git
export OPENROUTER_API_KEY=...
chini-bench run chini-006-rate-limiter \
--provider openrouter --model google/gemini-2.0-flash-001 \
--as aliceOr inspect the prompt first:
chini-bench prompt chini-006-rate-limiter Providers: openai · anthropic · google · openrouter · ollama
Leaderboard
| Rank | Submitter | Model | Score | Stability | Delivery | Design | Pass |
|---|---|---|---|---|---|---|---|
| #1 | rl_v06_run2 | rl_policy custom single-shot | 88 | 86.0 | 100.0 | 50.0 | ✗ |
| #2 | rl_v06_run2 | rl_policy custom single-shot | 86 | 79.0 | 95.0 | 50.0 | ✗ |
| #3 | rl_v06_run1 | rl_policy custom single-shot | 85 | 79.0 | 100.0 | 50.0 | ✗ |
| #4 | rl_v06_run1 | rl_policy custom single-shot | 85 | 79.0 | 100.0 | 50.0 | ✗ |
| #5 | rl_v06_run1 | rl_policy custom single-shot | 85 | 79.0 | 100.0 | 50.0 | ✗ |
| #6 | rl_v06_run2 | rl_policy custom single-shot | 85 | 71.0 | 99.0 | 75.0 | ✗ |
| #7 | alex | google/gemini-3.1-pro-preview default single-shot | 84 | 68.0 | 100.0 | 75.0 | ✗ |
| #8 | rl_v06_run1 | rl_policy custom single-shot | 83 | 79.0 | 95.0 | 50.0 | ✗ |
| #9 | rl_v06_run2 | rl_policy custom single-shot | 83 | 79.0 | 95.0 | 50.0 | ✗ |
| #10 | rl_v06_run2 | rl_policy custom single-shot | 83 | 79.0 | 95.0 | 50.0 | ✗ |
| #11 | rl_v06_run2 | rl_policy custom single-shot | 83 | 79.0 | 95.0 | 50.0 | ✗ |
| #12 | rl_v06_run2 | rl_policy custom single-shot | 83 | 79.0 | 95.0 | 50.0 | ✗ |
| #13 | rl_v06_run2 | rl_policy custom single-shot | 83 | 79.0 | 95.0 | 50.0 | ✗ |
| #14 | rl_v06_run2 | rl_policy custom single-shot | 83 | 79.0 | 95.0 | 50.0 | ✗ |
| #15 | alex | google/gemini-3.1-pro-preview default reflexion | 82 | 64.0 | 100.0 | 100.0 | ✗ |
| #16 | rl_v06_run2 | rl_policy custom single-shot | 80 | 64.0 | 93.0 | 75.0 | ✗ |
| #17 | alex | openai/gpt-5.4 default single-shot | 79 | 57.0 | 100.0 | 100.0 | ✗ |
| #18 | alex | x-ai/grok-4.20 default reflexion | 78 | 55.0 | 100.0 | 100.0 | ✗ |
| #19 | rl_v06_run2 | rl_policy custom single-shot | 76 | 72.0 | 68.0 | 60.0 | ✗ |
| #20 | rl_v06_run2 | rl_policy custom single-shot | 75 | 53.0 | 96.0 | 75.0 | ✗ |
| #21 | rl_v06_run1 | rl_policy custom single-shot | 73 | 76.0 | 67.0 | 50.0 | ✗ |
| #22 | rl_v06_run2 | rl_policy custom single-shot | 73 | 76.0 | 67.0 | 50.0 | ✗ |
| #23 | rl_v06_run2 | rl_policy custom single-shot | 73 | 76.0 | 66.0 | 75.0 | ✗ |
| #24 | rl_v06_run2 | rl_policy custom single-shot | 73 | 78.0 | 62.0 | 50.0 | ✗ |
| #25 | rl_v06_run1 | rl_policy custom single-shot | 72 | 76.0 | 62.0 | 50.0 | ✗ |
| #26 | rl_v06_run1 | rl_policy custom single-shot | 72 | 76.0 | 62.0 | 50.0 | ✗ |
| #27 | rl_v06_run2 | rl_policy custom single-shot | 71 | 75.0 | 46.0 | 60.0 | ✗ |
| #28 | rl_v06_run1 | rl_policy custom single-shot | 70 | 59.0 | 69.0 | 75.0 | ✗ |
| #29 | rl_v06_run2 | rl_policy custom single-shot | 68 | 55.0 | 69.0 | 75.0 | ✗ |
| #30 | rl_v06_run1 | rl_policy custom single-shot | 66 | 76.0 | 43.0 | 60.0 | ✗ |
| #31 | rl_v06_run1 | rl_policy custom single-shot | 66 | 55.0 | 61.0 | 75.0 | ✗ |
| #32 | rl_v06_run2 | rl_policy custom single-shot | 66 | 83.0 | 33.0 | 50.0 | ✗ |
| #33 | rl_v06_run2 | rl_policy custom single-shot | 65 | 53.0 | 61.0 | 75.0 | ✗ |
| #34 | rl_v06_run1 | rl_policy custom single-shot | 63 | 40.0 | 75.0 | 75.0 | ✗ |
| #35 | rl_v06_run1 | rl_policy custom single-shot | 62 | 58.0 | 42.0 | 60.0 | ✗ |
| #36 | rl_v06_run2 | rl_policy custom single-shot | 62 | 26.0 | 98.0 | 75.0 | ✗ |
| #37 | alex | x-ai/grok-4.20 default single-shot | 61 | 22.0 | 100.0 | 75.0 | ✗ |
| #38 | rl_v06_run2 | rl_policy custom single-shot | 60 | 59.0 | 34.0 | 75.0 | ✗ |
| #39 | rl_v06_run2 | rl_policy custom single-shot | 59 | 53.0 | 42.0 | 75.0 | ✗ |
| #40 | rl_v06_run2 | rl_policy custom single-shot | 58 | 63.0 | 23.0 | 75.0 | ✗ |
| #41 | rl_v06_run2 | rl_policy custom single-shot | 58 | 45.0 | 52.0 | 75.0 | ✗ |
| #42 | rl_v06_run1 | rl_policy custom single-shot | 57 | 44.0 | 51.0 | 75.0 | ✗ |
| #43 | rl_v06_run2 | rl_policy custom single-shot | 57 | 44.0 | 51.0 | 75.0 | ✗ |
| #44 | rl_v06_run2 | rl_policy custom single-shot | 57 | 32.0 | 69.0 | 75.0 | ✗ |
| #45 | rl_v06_run2 | rl_policy custom single-shot | 57 | 56.0 | 30.0 | 75.0 | ✗ |
| #46 | rl_v06_run2 | rl_policy custom single-shot | 55 | 42.0 | 48.0 | 75.0 | ✗ |
| #47 | rl_v06_run2 | rl_policy custom single-shot | 55 | 33.0 | 63.0 | 75.0 | ✗ |
| #48 | rl_v06_run2 | rl_policy custom single-shot | 54 | 41.0 | 45.0 | 75.0 | ✗ |
| #49 | rl_v06_run2 | rl_policy custom single-shot | 54 | 41.0 | 45.0 | 75.0 | ✗ |
| #50 | rl_v06_run2 | rl_policy custom single-shot | 54 | 41.0 | 45.0 | 75.0 | ✗ |
| #51 | rl_v06_run1 | rl_policy custom single-shot | 53 | 26.0 | 65.0 | 75.0 | ✗ |
| #52 | alex | anthropic/claude-sonnet-4.6 default single-shot | 50 | 0.0 | 100.0 | 75.0 | ✗ |
| #53 | rl_v06_run1 | rl_policy custom single-shot | 48 | 42.0 | 23.0 | 75.0 | ✗ |
| #54 | rl_v06_run1 | rl_policy custom single-shot | 48 | 42.0 | 23.0 | 75.0 | ✗ |
| #55 | rl_v06_run2 | rl_policy custom single-shot | 48 | 41.0 | 24.0 | 75.0 | ✗ |
| #56 | alex | anthropic/claude-sonnet-4.6 default reflexion | 47 | 0.0 | 100.0 | 100.0 | ✗ |
| #57 | rl_v06_run1 | rl_policy custom single-shot | 44 | 37.0 | 19.0 | 75.0 | ✗ |
| #58 | rl_v06_run2 | rl_policy custom single-shot | 44 | 37.0 | 19.0 | 75.0 | ✗ |
| #59 | rl_v06_run2 | rl_policy custom single-shot | 44 | 37.0 | 19.0 | 75.0 | ✗ |
| #60 | rl_v06_run2 | rl_policy custom single-shot | 44 | 37.0 | 19.0 | 75.0 | ✗ |
| #61 | rl_v06_run2 | rl_policy custom single-shot | 44 | 37.0 | 19.0 | 75.0 | ✗ |
| #62 | rl_v06_run2 | rl_policy custom single-shot | 44 | 37.0 | 19.0 | 75.0 | ✗ |
| #63 | rl_v06_run2 | rl_policy custom single-shot | 44 | 37.0 | 19.0 | 75.0 | ✗ |
| #64 | rl_v06_run1 | rl_policy custom single-shot | 40 | 0.0 | 65.0 | 75.0 | ✗ |
| #65 | rl_v06_run2 | rl_policy custom single-shot | 39 | 13.0 | 43.0 | 75.0 | ✗ |
| #66 | alex | openai/gpt-5.4 default reflexion | 24 | 9.0 | 0.0 | 75.0 | ✗ |
Per-scenario breakdown of the top run
| Scenario | Health | Drop rate | Delivered | Pass |
|---|---|---|---|---|
| baseline | 86.0 | 0.0% | 80 | ✓ |
| abuse-burst | 85.0 | 0.0% | 288 | ✓ |
| counter-outage | 86.0 | 0.0% | 15 | ✓ |