chini-008-search-autocomplete

Search Autocomplete

Suggest as you type. Stay snappy when one shard goes dark.

Source: Classic system-design interview corpus (Google / Amazon search suggest)

Prompt

Design a search autocomplete service.

Functional:
- GET /suggest?q=<prefix> returns up to 10 ranked suggestions in <50ms p99.
- The suggestion index is updated nightly from a batch job; queries hit the live index.
- Popular queries should be served from cache; tail queries fall through to the index.

Non-functional:
- A 6x query spike (a celebrity event) must not collapse latency.
- If one index shard fails, queries that would have hit it should degrade to results from neighboring shards rather than 5xx.
- The nightly batch update must not interrupt live serving.

Return a Chinilla CanvasState. Expect a cache, an index store (storage), and replication / routing for shard failures.

Constraints

Max components: 12
Required behaviors: storage, replicate
Monthly budget: $800

Stress scenarios

Baseline queries

baseline

Steady prefix-suggest traffic.

6x query spike

spike

Trending event causes 6x suggest traffic.

Index shard outage

outage

An index shard goes offline. Queries should degrade, not fail.

Pass criteria (overall)

Min stability score: 70
Max drop rate: 5.0%
Min delivery rate: 90.0%
Max errors: 5

Submit your run

Submissions go through the chini-bench CLI. It calls your model with your key, scores the result locally, and posts to the leaderboard. Nothing leaves your machine except the canvas it produces.

End-to-end:

pip install git+https://github.com/collapseindex/chini-bench-cli.git
export OPENROUTER_API_KEY=...

chini-bench run chini-008-search-autocomplete \
  --provider openrouter --model google/gemini-2.0-flash-001 \
  --as alice --x alice --linkedin alice-builds

Or inspect the prompt first:

chini-bench prompt chini-008-search-autocomplete

Providers: openai · anthropic · google · openrouter · ollama

Leaderboard

Rank	Submitter	Model	Score	Stability	Delivery	Design	Pass	Links
#1	alex default	G google/gemini-3.1-pro-preview	88	75.0	100.0	75.0	✗	X
#2	alex default	A anthropic/claude-sonnet-4.6	84	67.0	100.0	100.0	✓	X
#3	alex default	X x-ai/grok-4.20	54	67.0	0.0	25.0	✗	X
#4	alex default	O openai/gpt-5.4	53	65.0	0.0	50.0	✗	X

Per-scenario breakdown of the top run

Scenario	Health	Drop rate	Delivered	Pass
baseline	78.0	3.5%	334	✓
celeb-spike	73.0	5.3%	1598	✗
shard-outage	73.0	1.4%	44	✓

How is this scored? →