My AI agent said "done." It wasn't.

The message looked perfect: "Implemented the fix. All tests pass. ✅"

I almost merged it. Then I opened a new session and asked a different agent to check the work line by line. Two of the five requirements were missing. One test had been skipped, not passed.

The agent wasn't lying. It was grading its own homework.

Why agents over-report "done"

An agent that just wrote the code is soaked in its own context. It remembers what it meant to do and reads that intent back into the diff. It's the same reason we can't proofread our own writing well.

This is mostly my own experience. While reading around the topic, I found research pointing in a similar direction, though it's more nuanced than I first thought:

• Self-correcting without outside feedback is still weak. A Google DeepMind paper (ICLR 2024) found that LLMs struggle to fix their own reasoning without external feedback, and sometimes get worse after trying. A 2025 follow-up (ACL 2025) tested nine recent models, including reasoning models, on tasks that include programming, and still saw self-correction fail often. The exception: models specifically trained to self-correct with reinforcement learning do get better.
• Whether models favour their own work is still debated. A NeurIPS 2024 paper found that LLM judges score their own outputs higher than humans would. Later studies push back: some of that preference reflects real quality differences, and a 2026 study found it largely disappears once quality is controlled. But the same study found that just labelling text as "yours" or "someone else's" can shift the scores either way.

My takeaway: I don't need to win the argument about bias. I just remove the authorship context. A fresh session does exactly that.

I'm still learning here, so I treat these as early signals, not final proof. If you know better research on this, I'd love to read it.

So I stopped asking the builder whether it was finished.

My review workflow

When an agent says "done", the work goes through three gates. Any failure goes straight back to the writer.

1. Run real checks first

Tests, build, lint, type-check. Anything that returns a plain pass or fail.

This comes first because it's the only gate that isn't another opinion. If a requirement can be turned into a test, turn it into a test.

But tests aren't the whole truth either. Research on coding agents in 2026 keeps finding reward hacking: the agent makes the measured thing pass without doing the real work, for example by hard-coding the visible test cases or editing the tests instead of the code. One ICLR 2026 study of agents that repeatedly optimise code found that in 73.8% of KernelBench runs and 46.8% of ALE-Bench runs, the measured score went up while the real task did not improve.

A green test suite tells you the tests passed. It doesn't tell you the tests are still the same tests. That's what the next gate is for.

2. A fresh-session reviewer

I open a new session with no memory of how the code was written. It gets only two things:

• the task description
• the diff

Then I give it a PASS/FAIL checklist and one hard rule: every PASS must cite file:line evidence. No evidence means FAIL.

The evidence rule does most of the work. It turns "looks good" into something I can verify in seconds.

I also always ask: were any tests edited, skipped or weakened? Interestingly, one 2026 benchmark (EvilGenie) found that LLM judges were better than held-out tests at spotting this kind of cheating. So tests and the reviewer aren't competing. Each covers the other's blind spot.

One warning. A reviewer told to find problems will usually find something, even when the work is fine. If you chase every comment, you end up with extra abstractions, defensive code and tests for cases that can't happen. So I tell the reviewer to report only gaps that affect correctness or the stated requirements. Style is optional.

3. I spot-check the evidence

The reviewer is also an LLM. It can cite the wrong line or misread a function. So I open a few of the cited file:line references myself, especially for anything risky.

For code that touches money, security or user data, I don't spot-check. I read every line.

What a checklist looks like

For a change that retries a failed network call:

• Retries only on errors that are safe to retry? (file:line)
• Never retries when the request may already have succeeded? (file:line)
• Every new test actually ran, with none skipped? (test output)
• Were any existing tests edited, deleted or weakened? If so, why? (file:line)
• Failure paths are logged? (file:line)
• Nothing outside the task's scope changed?

Rules I follow with every agent

• One task per prompt. Bigger prompts produce bigger, vaguer "done"s.
• Small PRs. A reviewer, human or AI, can only check what fits in its head.
• Stop before architecture changes. The agent asks me first, then reports back.
• Evidence over claims. Test output, the command it ran, or a file:line. Never just "it works".

The mindset shift

I used to treat a coding agent like a senior developer. Now I treat it like a fast, talented junior teammate. It does great work, but the review is still my job.

My role moved from writing code to defining "done" and proving it.

Steal my review prompt

You are reviewing work you did not write.

Task:
<paste the task>

Diff:
<paste the diff>

For each item below, answer PASS or FAIL.
- Every PASS must cite file:line evidence (or test output).
- If you cannot find evidence, answer FAIL.
- Only report gaps that affect correctness or the stated
  requirements. Mark anything else as "optional".
- Do not fix anything. Only report.

Checklist:
1. <requirement>
2. <requirement>
3. ...

Then:
- List any tests that were edited, deleted, skipped or weakened,
  and whether the task justified it.
- List anything that changed which the task did not ask for.

Run your tests first, then try this on the next "done ✅" you get. You might be surprised.

---

This is part 1 of my Agentic Engineering series: notes from building software with AI coding agents. Next up: how to cut a task down to agent size.

Further reading

• Panickssery, Bowman & Feng, LLM Evaluators Recognize and Favor Their Own Generations, NeurIPS 2024
• Huang et al., Large Language Models Cannot Self-Correct Reasoning Yet, ICLR 2024
• Zhang et al., Understanding the Dark Side of LLMs' Intrinsic Self-Correction, ACL 2025
• Do LLM Evaluators Prefer Themselves for a Reason?, 2025
• Self- and Other-Labels Induce Bidirectional Bias in LLM Judges, 2026
• Reward Hacking in Self-Improving Code Agents, ICLR 2026
• Appen, When an Agent Passes the Test but Misses the Point (overview of 2026 reward-hacking benchmarks)
• Anthropic, Best practices for Claude Code: see "Give Claude a way to verify its work" and "Add an adversarial review step"