Cody

Cody AI

Cody AI

Cody AI

Cody AI

Cody AI

Cody AI

Cody AI

Cody AI

Cody AI

Cody AI

With Claude Code I built an AI interrogation game, 200+ players in a week, 1,400 questions asked so far. Here’s what happened.

I’ve been building a browser game called **The Last Question**. The idea: You interrogate AI suspects trying to make them confess. Each suspect has hidden internal state (pressure, trust, story consistency), so they react differently depending on your approach. Some players try logic. Some threaten. Some obviously try to flirt with the suspects (but I have already put in measures for this!) Built fast with: * lots of Claude Code * AI-generated suspect content (including images) * cheap infra Current stats: * 258 players * 1,471 interrogation messages * 23% confession rate Biggest surprise: People quit WAY earlier than I expected. Top dropoffs: * Message #1 → 22.5% * Message #2 → 12.3% * Message #8 → 12.3% (this is where free credits end) Which probably means: * opening experience is weak * players don’t understand the game fast enough * monetization is way too early Now I’m experimenting with: * visual novel style intros * community-created suspects * sharing interrogation transcripts * daily credits * making suspects feel more “alive” Curious: If you tried this, what would make you stay and play another suspect? Here is how it looks like! [https://thelastquestion.io](https://thelastquestion.io)

I measured my Claude Code MCP stack on two axes — byte savings AND cache-friendliness. My "best" byte-saver was defeating Anthropic's prompt cache (counter-example + open benchmark)

**TL;DR** — Single-axis benchmarks for MCPs, compressors, and retrieval layers can recommend a system that's *strictly worse* in production. The missing axis: **cache-friendliness** — whether the same input produces byte-identical bytes across runs, so Anthropic's prompt cache hits. In my coding-agent stack, my biggest byte-saver (retrieval MCP, 60–70% reduction) was defeating the 5-min TTL prompt cache on every call. Two runs of the same query produced different bytes because of `rg --files-with-matches` output order leaking through a `Map` insertion sequence into the final context. The fix was 2 lines: sort the rg hits before slicing, sort the `Map` entries by path. Byte savings unchanged, `cache_friendly_score` went from \~0% to 100%. https://preview.redd.it/x5foipotq93h1.png?width=1600&format=png&auto=webp&s=c0930422e882e23d1fc34ded25934c74db692a21 **Article + open benchmark harness:** * Article: [https://gregshevchenko.com/research/mcp-stack-token-economy/](https://gregshevchenko.com/research/mcp-stack-token-economy/) * Harness (stdlib-only Python, offline): [https://github.com/g-shevchenko/mcp-token-savers](https://github.com/g-shevchenko/mcp-token-savers) — see `methods/` for formal definitions, cluster-bootstrap CIs, Wilson CIs, preregistration, real-data Cohen's κ. **What the harness measures:** * `mean_ratio` \+ CV across N≥5 runs per fixture → byte-saving axis * `unique_md5_count == 1` check → cache-friendliness axis (0–100%) * 12-anti-pattern audit on tool definitions (DSA reference) **What named alternatives publicly disclose:** I surveyed the public docs for Cursor codebase index, Sourcegraph Cody, Aider repo-map, Microsoft LLMLingua / LLMLingua-2, Firecrawl / Jina Reader, RouteLLM / Martian (May 2026). https://preview.redd.it/ailemo1wq93h1.png?width=1600&format=png&auto=webp&s=4732f5d03f53ba95d2b5aaac0c7f21f1858a36a4 **Limitations:** * I hypothesized that the prep layer triggers more downstream cache hits on subsequent turns. It didn't reach significance: Welch p=0.32, Cohen's d ≈ 0.18, N=137. * Two-judge Cohen's κ on the corpus (cerebras-llama × groq-llama, N=25): κ = 0.5955 (moderate, below the 0.7 substantial threshold). 4 of 5 inter-judge disagreements concentrate on one task with an ambiguous acceptance criterion. Sharpening the spec would push κ to \~0.83. **Disclosure:** I'm the author. No commercial affiliation with the listed tools. The harness is MIT-licensed and takes any compressor as `(str) -> str`. Curious what `cache_friendly_score` looks like on others' Claude Code stacks.