The essay argues that raw model weights have converged to near-parity, and the differentiator between products like Claude Code, Cursor, and Devin is the harness — the loop, tools, prompts, context management, verification, and retry logic wrapping the API. Giving this scaffolding a crisp name reframes the entire industry conversation away from benchmark leaderboards and toward the surrounding software engineering.
The piece points to SWE-bench Verified as evidence: the same base model scores in the low 20s with a naive one-shot prompt but reaches ~70% inside a proper harness with file navigation, scratchpads, test-runner loops, and error recovery. That ~50-point delta is entirely attributable to scaffolding, meaning leaderboards like MMLU, HumanEval, and GPQA systematically mismeasure real-world capability.
The essay treats this as a settled, load-bearing observation behind every serious agent product shipped in the last 18 months, not a controversial claim. The individual components of a harness — the loop, tool interfaces, context management, verification, retry logic — are individually boring but collectively decisive in determining whether an agent actually works.
A post titled *What Is a Harness?* landed on the Hacker News front page and clocked 367 points, which is unusual for a definitional essay. The reason it resonated: the AI community has spent two years arguing about model benchmarks while the actual gap between products has been almost entirely explained by something people don't have a clean name for.
The essay's term of art is harness — the loop, tools, prompts, context-management, verification, and retry logic that sit between a raw model API and a working product. Claude Code is a harness. Cursor is a harness. Devin is a harness. The chat window on claude.ai is a harness. Same underlying weights; radically different capability surfaces.
The essay's core claim is that the model is now the commodity and the harness is where all the interesting engineering has migrated. That's not a new observation in agent circles, but this piece is the first to give it a crisp, teachable name — which is probably why it went viral.
For two years, the industry has evaluated AI by leaderboard: MMLU, HumanEval, SWE-bench, GPQA. Those benchmarks measure the model in a vacuum. But nobody uses a model in a vacuum. They use it inside a harness, and the harness quietly does most of the work.
Consider SWE-bench Verified, where the top scores hover around 70%. The same base model running with a naive one-shot prompt scores in the low 20s. The delta is entirely harness: file navigation tools, a scratchpad, a test-runner loop, a plan-then-execute structure, error recovery when a tool call malforms. A mediocre model with a great harness routinely beats a great model with a mediocre harness, and this is not a controversial claim anymore — it's the load-bearing observation behind every serious agent product shipped in the last 18 months.
The essay is careful to break the harness down into components that are individually boring but collectively decisive:
- The loop. How the model gets called repeatedly, when it stops, what triggers a retry. - The context. What goes into the prompt on turn N — recent messages, tool results, retrieved docs, summaries of earlier turns, system instructions. - The tools. Function definitions, their schemas, how errors surface, how results are formatted for the model to read next turn. - The verification. Does the harness re-read its own output? Run the tests? Check the diff compiles? Ask a second model to grade? - The escape hatches. What happens when the model hallucinates a tool call, produces malformed JSON, hits a rate limit, times out.
Each of these is a design decision. None of them are the model. All of them determine whether your product actually works.
The HN discussion drove this home with practitioner examples. One commenter noted that swapping from GPT-4 to Claude 3.5 in their agent moved the success rate by ~4 points, but rewriting the tool-error-handling in the harness moved it by 22. Another pointed out that Cursor's edge over vanilla Copilot isn't a better model — it's a smarter context window: which files to include, which to summarize, which to skip. The harness is where product taste gets encoded, and product taste is not a solved problem you can buy from an API.
If you're building on top of LLMs, this reframes the roadmap. The question isn't *which model should we use* — it's *what's our harness architecture, and where are the biggest wins hiding in it.*
A few concrete implications:
Stop A/B-testing models first. The instinct when your agent underperforms is to try GPT-5 or Opus 4.7 or whatever shipped last week. That's usually the wrong first move. Instrument your harness: log every tool call, every retry, every context assembly. The failure modes will almost always be there, not in the model's raw reasoning. A malformed tool call that the harness silently drops will destroy your success rate more thoroughly than a 3-point drop in MMLU.
Treat the harness as first-class code. Most teams have thousands of lines of harness logic buried in a `utils.py` that nobody owns, and it shows in the output. The harness deserves the same test coverage, version control, and design review as any other core system. If your product's differentiation lives here, act like it.
The moat is portable in one direction. Model APIs are near-perfect substitutes. Your harness code is not. This is good news: it means your investment in scaffolding compounds across model upgrades. A well-designed harness gets a free performance bump every time Anthropic or OpenAI ships a new SOTA. A poorly-designed one squanders those upgrades entirely.
Beware harness-shaped SaaS. A wave of startups is selling "agent frameworks" that are essentially opinionated harnesses wrapped in a subscription. Some are genuinely useful. Many are one weekend of hand-rolled code, priced at $2K/month. Read what's actually in the box before you buy.
The next frontier of AI product engineering is going to be fought over harness architecture, not model weights — and the teams that internalize this early will build durable advantages while their competitors keep chasing benchmark scores. Expect the term to enter general vocabulary within the next quarter, and expect the interview question *"walk me through your harness design"* to start showing up in senior AI eng loops shortly after.
Does anyone have a suggestion for a harness that is good at handoff?When I say handoff, I mean: * handoff from a terminal CLI to webui (on a phone)? * handoff from one team member, to another? * handoff from one communication modality, like writing a prompt in a TUI, to email? * handoff from one mod
Author here. It’s ironic because this post was clearly geared towards non-hackers. But now that we’re here.. the other analogy I considered presenting was:harness = chassis, model = engine, fuel = tokens, agent = carI’m curious what y’all might think and whether that analogy carries more explanatory
Harnesses are the next frontier. If LLMs are electricity, harnesses are the “electronics.” Right now, it’s like an AC vs DC between Claude and ChatGPT, but once that settles, the harnesses will be the actual value providers.And Pi is the best harness because of the amazing extension system. You can
The ai hype word for 2026 after agent in 2025 for any LLM powered application.Well kind of, I wouldn't be surprised to see that some things marketed as agents are actually good old deterministic software.
Top 10 dev stories every morning at 8am UTC. AI-curated. Retro terminal HTML email.
I’ve been working on a harness for accounting agents at my job recently and it’s been a pretty interesting experience.We originally started with building a CLI tool so our LLMs could more easily interact with our platform. I cannot recommend enough the value of having an internal CLI. It’s both fun