Does AI actually make you code faster?
In July 2025 a research group called METR ran something almost nobody in this argument had bothered to run: an actual randomised controlled trial. They took 16 experienced open-source developers, gave them 246 real tasks on repositories they already maintained, and randomly allowed or forbade AI tools on each task.
The developers with AI took 19% longer. Afterwards, those same developers estimated AI had made them about 20% faster.
You can read the write-up at METR's report on the study.
Read the caveats before you quote the number
This result gets waved around by both camps, and both usually skip the fine print.
- 16 developers is a small sample. It is a real trial, not a survey, but it is small.
- The tasks were on large, mature codebases the developers already knew well. That is close to the worst case for AI assistance and nothing like greenfield work.
- The tools were early-2025 tools, mostly Cursor with Claude 3.5 or 3.7 Sonnet.
- METR themselves have since published an update saying they are changing the experiment design and that the result should be treated as historical.
So the honest reading is not "AI makes developers slower". The honest reading is narrower and more useful.
The gap is the finding, not the 19%
The number that should worry you is not 19%. It is the 39-point swing between how fast people were and how fast they felt.
Speed is not something you can sense from the inside. Typing feels like progress. Waiting for a suggestion, reading it, deciding whether it is right, and then fixing it does not feel like work in the same way, so your brain does not bill you for it. You remember the moment the tool wrote a working function. You do not remember the twenty minutes spent proving that a different suggestion was subtly wrong.
That is a measurement problem, and it does not go away as the models get better. It gets worse, because a suggestion that is 95% right is far more expensive to check than one that is obviously garbage.
What this changes about how you practise
If you cannot feel your own speed, you have to keep the skills that let you judge output instead of just accepting it.
Keep the ability to read code cold. The expensive part of AI-assisted work is deciding whether the thing in front of you is correct. That is a reading skill, and it decays if you never read anything you did not write. Working through JavaScript multiple-choice questions with the explanation shown afterwards is deliberately this exercise: look at code, predict what it does, find out if you were right.
Practise the judgment calls, not the syntax. AI can wire up a payment SDK in minutes. It cannot decide how much staleness your product tolerates, or which failure mode you are willing to accept. Those are the parts of the job that stay yours, and we wrote up four of them in what to own and what to hand to AI.
Get graded on reasoning, not answers. Estimation questions are the cleanest test of whether you can still structure a problem without a tool holding your hand. Our AI-graded guesstimates score the reasoning against a published rubric, so you can see exactly where the structure broke rather than just whether the number matched.
Measure something, even badly. The trial worked because someone held a stopwatch. You will not run a randomised trial on yourself, but you can note how long a task took and compare it to your estimate. The first few times are usually uncomfortable.
Where this leaves the tools
Nothing here says stop using AI. It says stop trusting your own sense of how much it helped, and keep the skills that let you check its work. Those are the same skills a technical interview tests, which is convenient, because it means practising for one is practising for the other.
If you want to argue with any of this, the study is linked above and so is METR's own update walking parts of it back. Read both.