AI Can Write the Code. Who Is Responsible for Proving It Works?

By Neel Shah · August 21, 2026

We've spent two years asking the wrong question. "Can AI write the code?" is settled — it can, and in 2026 roughly 42% of new code is AI-generated or AI-assisted, on its way past half by next year. The question that actually matters now is quieter and much harder: when an agent implements a feature, who is responsible for proving the implementation is correct?

That responsibility didn't disappear when generation got cheap. It moved. And most teams haven't noticed where it landed.

The bottleneck moved — it didn't vanish

For decades, writing code was the expensive part of software. AI collapsed that cost. But software was never bottlenecked on typing — it was bottlenecked on being sure the thing is right. Generation got 10× cheaper; verification didn't get cheaper at all. So the constraint simply relocated from "write it" to "prove it works."

You can see this in the data that should have been the productivity victory lap. A METR randomized controlled trial in mid-2025 found experienced open-source developers using AI tools took 19% longer to finish tasks — not because the AI couldn't produce code, but because of the overhead of reviewing and debugging code that interacted with the system in unexpected ways. The generation was fast. The verification ate the savings.

GitHub, which has every incentive to be optimistic about agents, said the quiet part out loud in 2026: agents can produce changes faster than any human can review them, and code now lands in pull requests "without a clear trail of what the agent tried, what it validated, or where human judgment is needed." When the company selling the agents tells you review is the problem, the problem is real.

The uncomfortable data on correctness

Here's the finding that reframes everything, from Veracode's Spring 2026 study of over 150 models: AI coding tools now write syntactically correct code more than 95% of the time — but only about 55% of it is secure. Nearly half of AI-generated code introduces a known OWASP-class vulnerability when no explicit security guidance is given. And that security number has stayed essentially flat for two years, even as the models' coding benchmarks climbed.

Sit with that decoupling: capability went up, correctness didn't. The models got better at producing code that looks right and compiles, without getting better at code that is right. As the researchers put it, the model learns from frequency, not from correctness — it reproduces the most common patterns on the internet, security holes and all.

The downstream numbers match. Blended 2026 analysis puts AI-authored pull requests at roughly 1.7× the defect rate and 2.74× the vulnerability rate of human-written code. That multiplier sounds survivable until you remember it's multiplying half your codebase. The most dangerous output of an AI agent isn't code that fails — that gets caught. It's code that is plausible, passes a shallow read, and is subtly wrong.

Why agent code is harder to verify than a junior's

We already know how to review human code. AI code breaks several assumptions that review quietly depends on:

• There's no intent to interrogate. When a colleague writes something odd, you ask "what were you thinking here?" and their answer reveals whether it's a bug or a reason you missed. An agent has no thinking to recover — only output. The single most useful signal in code review, author intent, is gone.
• Confidence is decoupled from correctness. Human uncertainty leaks into hedged comments and tentative structure. Agents produce wrong code with exactly the same fluent confidence as right code. There's no tell.
• The tests can be theater. If the same agent writes the implementation and the tests, green checks prove only that it was internally consistent with its own misunderstanding. Tests written to fit the code will pass the code.
• Architecture gets violated politely. Agents optimize for making the local change work, not for honoring the conventions, boundaries, and implicit contracts of your codebase. ICSE 2026 research confirmed the flip side: feeding agents explicit architectural context measurably improves not just correctness but architectural conformance — which tells you how often, without it, they quietly break your structure.

So the reviewer of AI code has a harder job than the reviewer of a junior's code, while the volume of code to review has gone up. That's the squeeze.

So who is responsible?

Strip away the tooling and the answer is old and unglamorous: the human who approves the merge owns the code. Accountability can't be delegated to an agent, because an agent can't be accountable — it can't be on call at 2 a.m., can't answer to a customer, can't be the name in the incident review. Responsibility is a property of people, and in software it attaches at the moment of approval.

What AI changed is that it collapsed the author and the reviewer into the same problem. Historically those were two people providing two independent judgments. When the "author" is an agent, that independence is an illusion unless a human reconstructs it — genuinely understanding the change well enough to have written it, not just skimming a diff and trusting the green check. If you approve a PR you don't understand, you haven't been assisted by AI; you've laundered unverified code through your own credibility.

This is the shift the industry is slow to price in: reviewing is no longer the cheap part of the job. It's the job. The person who can look at plausible-looking agent output and find the subtle wrongness is now doing the highest-value work on the team — and only about 18% of organizations have any review process specific to AI-generated code. The other four in five are pointing a human-scale review process at a machine-scale firehose.

What the tooling is (finally) doing about it

The good news: the platforms have recognized that the fix isn't better generation, it's better legibility — making an agent's work reviewable by a human. GitHub's 2026 agent-native direction is aimed squarely here:

• Provenance and trails — surfacing what the agent tried, what it ran, and what it validated, so a reviewer can reconstruct intent instead of guessing.
• Stacked pull requests (public preview, mid-2026) — a direct answer to agents dumping one enormous, unreviewable diff; they break a change into ordered, independently reviewable layers.
• Review gates for bots — agent-created PRs require human approval before they can run privileged workflows, closing an obvious accountability and security hole.
• Architectural context files (AGENTS.md) — giving agents the conventions up front, which the research shows improves conformance and correctness.

None of this removes the human from the loop. All of it is designed to make the human's verification possible at the speed agents now produce. That's the tell for where this is heading: not "trust the agent," but "make the agent auditable."

The test I'm running (and the scorecard)

Opinions about agent reliability are cheap. So I'm doing what the data begs for: giving a coding agent a real task in a real codebase — not a toy kata — and scoring what actually happens end to end.

Requirement → Agent → Code → Tests → Review → Issues → Human correction

The scorecard I'm filling in for each task:

That last row is the whole essay compressed into one question. Not "did it produce code" (it will) but "would I put my name on it." I'll publish the filled-in scorecard as a follow-up — the point is to replace "AI agents are unreliable" (a vibe) with "here is a real task, here's exactly where it succeeded, where it failed, and what I had to catch" (evidence).

A verification playbook for right now

Until the tooling fully catches up, this is the discipline that keeps AI a multiplier instead of a liability:

• Treat agent PRs like a new contributor's — with more scrutiny, not less. The fluency earns it less trust, not more.
• Keep the author and the verifier separate. Never let the agent that wrote the code be the only thing that "reviewed" it. If it wrote the implementation, a human (or a different, independent check) writes or audits the tests.
• Make the agent show its work. Prefer workflows that expose what it ran and validated. If you can't reconstruct intent, treat the change as unverified by default.
• Never merge what you don't understand. "The tests pass" is a necessary condition, not a sufficient one — especially when the tests may share the code's blind spots.
• Make security review non-negotiable. With ~45% of AI code carrying an OWASP-class flaw, a security pass isn't optional polish; it's the baseline assumption.
• Write it down. An explicit AI-code review policy is what separates the 18% who are managing this from the 80% who are hoping.

Takeaways

• AI made generation cheap; it made verification the bottleneck — the constraint moved, it didn't disappear.
• The data is blunt: ~95% syntactically correct, ~55% secure, flat for two years; AI PRs carry ~1.7× defects and ~2.74× vulnerabilities vs human code.
• Agent code is harder to review than human code — no intent to interrogate, confidence decoupled from correctness, and tests that can validate a misunderstanding.
• Accountability attaches at the merge. An agent can't be responsible; the human who approves is. Reviewing is now the job, not the chore.
• The tooling response is about legibility — provenance, stacked PRs, review gates — making agent work auditable at machine speed.
• The professionals who thrive in this era aren't the fastest prompters. They're the best verifiers.

Sources & further reading

• Veracode — "Spring 2026 GenAI Code Security Update" (95% syntax vs ~55% security pass rate)
• METR — randomized controlled trial on AI tooling and developer speed (July 2025)
• GitHub Blog — agent-native workflows, canvases, and "how to review agent pull requests" (2026)
• Sonar — developer survey on the share of AI-generated code
• ICSE 2026 — research on architectural context and agent code conformance
• Blended defect/vulnerability multipliers — Veracode, GitClear, and CodeRabbit 2026 analyses