Garrit argues from direct experience building agents: models behave brilliantly on the first 20K tokens of a 200K session, then start forgetting tool definitions, hallucinating already-read file paths, and contradicting earlier decisions. The number on the model card describes what you can stuff in, not what the model can reliably use.
Synthesizing the HN thread's linked research, the editorial notes that RULER shows frontier models above 90% on simple retrieval up to ~32K tokens then degrading sharply on multi-hop reasoning, while NoLiMa shows performance falling to half the short-context baseline at 32K for models advertising 128K–1M windows. The shape is consistent across tests even as specific numbers vary.
The editorial frames the two-year context-length race as treating tokens like RAM when the reality is closer to a noisy disk with read errors that climb with seek distance. This matters most for the exact workloads — codebase-wide agents, multi-document legal review, long-running chats — that justified the long-context race in the first place.
A short, blunt post by developer Garrit titled "Don't trust large context windows" hit 110 on Hacker News this week, and the comment thread filled with senior engineers nodding along. The argument is one sentence long: the number on the model card is a storage spec, not a performance spec. Everything past that is evidence.
Garrit's case is built on a pattern any agent builder will recognize: the model behaves brilliantly on the first 20K tokens of a 200K-token session, then starts forgetting tool definitions, hallucinating file paths it already read, and contradicting decisions it made twenty turns earlier. The post points at the gap between marketing ("1M tokens!") and the lived experience of running a coding agent against a real repo for an afternoon. The HN thread surfaced links to Adobe/NVIDIA's RULER benchmark, the NoLiMa paper from January, and Greg Kamradt's original Needle-in-a-Haystack work — all converging on the same uncomfortable result.
The specific numbers vary by model and test, but the shape is consistent. RULER shows most frontier models scoring above 90% on simple retrieval up to about 32K tokens, then degrading sharply on multi-hop reasoning beyond that. NoLiMa, which strips out lexical hints so the model has to actually reason about position, shows performance falling to half its short-context baseline at 32K for models advertising 128K–1M windows. The window is real. The competence inside it is not uniform.
The industry has spent two years racing on context length as if it were RAM — bigger is better, ship it, charge for it. The benchmarks tell a different story: it's more like a noisy disk where the read error rate climbs the further you seek. You can put a million tokens in. You cannot rely on the model finding the right ones.
This matters most for the workloads that motivated the long-context race in the first place. Codebase-wide agents. Multi-document legal review. Long-running chat sessions with tool use. These are exactly the cases where the failure mode is silent — the model doesn't say "I forgot," it confidently produces an answer based on the half of the context it still has working memory for. Senior engineers who've built agent loops describe the same arc: weeks of feeling like the model is "getting dumber," followed by the realization that context bloat, not the model, was the regression.
The HN comments split into two camps, both correct. Camp one points out that frontier labs have made real progress — Gemini's long-context numbers on Kamradt's needle test were genuinely impressive, and Anthropic's recent work on context management is shipping. Camp two points out that needle-in-a-haystack is the easiest possible long-context task (find this one weird sentence), and that the moment you ask for multi-hop reasoning across the window — "summarize how the auth flow evolved across these 40 commits" — everything falls apart again.
The more interesting subthread was about cost. A 200K-token prompt at current Anthropic pricing is several dollars per call. If half of it is being effectively ignored, you're paying retail for tokens the model treats as background noise. The economics of agent loops — which can rack up 50-100 calls per task — make this an actual line item, not a rounding error.
Stop treating the context window as a dumping ground. The pattern that works, repeatedly cited in the thread and in production write-ups from Cognition, Cursor, and Replit, is the opposite of what "1M tokens" implies: keep the working context small, curate aggressively, and use retrieval or sub-agents to fetch detail on demand.
Concrete moves worth running this week: instrument your prompts and log the actual token count per call — most teams discover their "short" prompts are 40K+ once tool definitions and history accumulate. Add a summarization step that compresses turn history older than N exchanges into a structured digest. For codebase agents, replace "dump the repo" with a file-level retrieval index and let the model request files by path. For RAG, run RULER-style evals on your own corpus before trusting that the model can find the answer in chunk 47 of 50.
The deeper lesson is that "context engineering" is becoming the new prompt engineering — and it's a real discipline, not a buzzword. What you put in the window, in what order, with what compression, is now a more important lever than which frontier model you picked. The teams shipping reliable agents are the ones who treat the window as expensive scratch space, not as a database.
The next 12 months will probably see two things happen in parallel. Labs will keep improving genuine long-context reasoning — there's real research progress on positional encoding, attention sparsity, and training data that rewards distant recall. And the practitioners building on top will continue to assume the worst, because the cost of believing the marketing is silent regression in production. Bet on the curation layer, not on the next window-size bump. The model card number is a permission, not a promise.
Top 10 dev stories every morning at 8am UTC. AI-curated. Retro terminal HTML email.