The productivity mirage: why AI coding feels fast but ships slow

5 min read 1 source clear_take
├── "AI coding assistants create a dopamine-driven illusion of productivity while actually slowing developers down"
│  └── frantic.im (The Productivity Mirage) → read

The essay argues that accepting Copilot suggestions and watching Claude Code fill in files feels like shipping but isn't. The author cites METR's controlled study showing developers using AI tools were 19% slower while believing they were 20% faster — evidence that the tools deliver the sensation of progress rather than actual throughput.

├── "AI assistants optimize the wrong stage of software engineering — typing was never the bottleneck"
│  └── top10.dev editorial (top10.dev) → read below

The editorial reframes software work as a lean manufacturing line where throughput is set by the slowest stage — review, staging, flaky tests, decision latency — not code generation. LLMs are optimized to skip past reading, remembering, and deciding, which were the actual hard parts, and by accelerating typing they simply pile more work-in-progress onto stages that were already the constraint.

└── "The productivity gains show up locally but are eaten downstream in review, incidents, and coordination"
  └── @Hacker News thread (Hacker News, 218 pts) → view

Commenters piled on with a consistent pattern: PRs land faster but reviews take longer, incidents linger, and teams avoid measuring the actual outcome. The gains are visible at the individual keystroke level and invisible — or negative — at the team throughput level.

What happened

A short essay titled *The Productivity Mirage* by frantic.im hit the front page of Hacker News (218 points) with an argument most senior engineers have been quietly making at standups for a year: coding assistants are a dopamine machine that we've mistaken for a productivity machine. The author's framing is that the pleasant hum of accepting Copilot suggestions or watching Claude Code fill in a file is the *feeling* of shipping, not the *fact* of shipping.

The piece doesn't come with a benchmark table. It comes with the observation that we already have one — METR's controlled study of experienced open-source maintainers from earlier this year, which found developers using AI tools were 19% slower at completing real tasks, while believing they were roughly 20% faster. That gap between perceived and actual speed is the entire subject of the essay. The HN thread piled on with variants of the same story: PRs land faster, review takes longer, incidents linger, and nobody quite wants to run the numbers on their own team.

The mirage is not that the tools don't work. It's that they optimize the part of the job that was never the bottleneck. Typing was never why software is hard. Reading, remembering, deciding, and *not* writing the thing were why software is hard, and LLMs are optimized to skip past exactly those steps with a confident tone of voice.

Why it matters

There are three distinct failure modes bundled inside "AI made me more productive," and it's worth pulling them apart because the fix for each is different.

The first is the inventory problem. Software engineering has always looked more like a lean manufacturing line than a typing contest — throughput is set by the slowest stage, which is almost never code generation. It's review, it's staging environments, it's the one flaky integration test, it's the Slack thread waiting on a decision. Assistants dump more work-in-progress onto that line. A 3× increase in PR volume against an unchanged review capacity doesn't produce 3× the shipped features; it produces a queue, and queues degrade quality because reviewers start skimming.

The second is the comprehension debt problem. When you write code, you build a mental model of the system as a byproduct; when you accept generated code, you get the artifact without the model. This is the same delta you feel between a codebase you wrote and one you inherited, except now you're inheriting a codebase from yourself, weekly. The bill comes due at debug time. Every senior engineer who's spent an afternoon spelunking through Copilot-suggested error handling that *almost* matches the surrounding conventions knows this tax intimately.

The third is the confidence problem, and it's the one METR's study captured most cleanly. Developers were *worse* and *sure they were better*. That's not a small calibration error — that's the tools actively degrading your ability to estimate your own throughput. If you can't trust your gut on "am I moving fast this sprint," you can't trust it on "is this refactor worth it," "is this ticket a two-day job or a two-week job," or "should we cut scope." The whole engineering-judgment stack sits on top of accurate self-assessment.

Community reaction on the HN thread is bifurcated in a telling way. The engineers doing greenfield, well-scoped, well-tested work (side projects, prototypes, isolated services) report enormous wins. The engineers doing brownfield work in large codebases with real users report a much muddier picture — some acceleration on boilerplate, a lot of extra review load, and a nagging sense that they're spending their brain-time on the wrong problems. That split isn't a disagreement about the tools; it's a disagreement about which job we're talking about.

What this means for your stack

Don't rip out Copilot. Do run the numbers.

The cheapest useful experiment is a two-week honest self-audit: track cycle time (ticket opened → merged → deployed → not-rolled-back) with the assistant on and off, on comparable tickets. Not "how did it feel" — how long did it actually take, and how many follow-up PRs did the original produce. If your team ships to production continuously, this is instrumentation you probably already have; you just need to bucket by "AI-assisted" and look. Most teams haven't done this because the answer is uncomfortable and the vendor's dashboard already tells them a happier story.

Second, treat generated code as an input to review, not an output of engineering. The one habit that separates the developers who genuinely get faster with LLMs from the ones who get mired is that the former read every line before it lands, and the latter don't. If your PR is 400 lines and you skimmed 380 of them, you have shifted the debugging work into next month at compound interest. The teams reporting real wins are running the assistant, then re-reading, then deleting a third of it, then shipping.

Third, notice where the tools *do* clearly win, and lean in there: writing tests for code you already understand, one-shot scripts, migrations, boilerplate CRUD, exploring an unfamiliar API in a scratch file. These are the tasks where comprehension debt doesn't compound because you were never going to maintain the code, or where the correctness bar is low and testable. Reserve unassisted flow-state for the parts of the system where being wrong is expensive and being subtly wrong is worse.

Looking ahead

The honest read of the current moment is that we're roughly a year into a technology whose productivity story is still mostly vibes and vendor decks, and the first rigorous studies are landing on the unflattering side of the ledger. That doesn't mean the tools are bad — it means the industry has been grading itself on the wrong metric. The next interesting move won't be a smarter model; it'll be the first engineering org that publishes a real, honest before/after on shipped-feature velocity, defect rate, and incident load. Until then, the productivity mirage is going to keep looking like a productivity oasis, right up until someone measures the water.

Hacker News 314 pts 137 comments

The Productivity Mirage

→ read on Hacker News

// share this

// get daily digest

Top 10 dev stories every morning at 8am UTC. AI-curated. Retro terminal HTML email.