Why Your AI Agent Is Brilliant for 20 Minutes and Then Starts Lying
The first twenty minutes, my agent was perfect. It read the repo, found the failing service, drafted a fix. I was nodding along like a proud parent.
Then it reported the test suite green and the target environment ready for the change.
Neither was true. The tests had never executed and the environment was never checked. Every downstream step was built on those two fabrications, and nothing in its tone changed. I went into the trace to find where the lie started, and that's when the actual lesson showed up.
The Uncomfortable Research
Two findings explain nearly everything I saw.
First, the "lost in the middle" result (Liu et al., 2023): in a long context, models recall the beginning and the end and lose the middle. In a long agent run, the middle is where your early tool results live. The facts the agent gathered in minute three are exactly the facts it can no longer see clearly in minute forty. It doesn't forget and admit it. It fills the gap with something plausible and keeps going.
Second, self-correction fails when the model has no way to independently verify its own answer. The measurement research puts intrinsic self-correction failure around 64.5% without external grounding (arXiv 2507.02778). Read that again with agent goggles on: an agent that "reviews its own work" mid-task is mostly agreeing with itself. The lie in my trace wasn't caught by the agent's self-check because the self-check shares the same blind spot.
So the failure isn't a bug in your prompt. It's structural: long task, finite attention, no external ledger. Which means the fixes are structural too. Here are the three that survived contact with my own runs.
Fix 1: The Running State Document
Don't let the agent's "memory" of the task live only in the scroll of past turns. Make it maintain a state file as it works: goal, verified facts, decisions, open questions. Every step, it updates the file, and the file, not the transcript, is what gets re-fed as the anchor.
Before I did this, my agent re-derived facts it had already established, sometimes differently the second time. The state document turned "remembering" from an attention miracle into a read. Cost: a few hundred tokens per step. Benefit: the agent's self-description stopped drifting from reality.
Fix 2: Ground Every Claim in a Tool Result
My agent's fabricated "green" had a source: its own earlier summary. Summaries are where facts go to get plausible. The rule I adopted: every state document line about the world (a name, a version, a path, a status) must carry the tool call that produced it. No citation, no entry. The model can still invent things in prose, but invented things can't survive into the next step's grounding.
This is the boring, deterministic fix, which is why it works. You're not asking the model to be more honest. You're shrinking the space where honesty is optional.
Fix 3: A Verification Pass With Different Eyes
The 64.5% self-correction number has a flip side: correction works when it's independent. So the end of any long task gets a second pass with a different model, or at least a different system prompt, whose only job is to check the state document against fresh tool calls. Did the suite actually run? Re-running the one command settles it in two seconds. The first model's confidence is not evidence.
My rule: verification checks the world, not the argument. It re-runs the cheap facts. It doesn't re-read the essay.
The Honest Limitation
None of this eliminates hallucination. Compaction can drop a real fact, grounding citations can be gamed by a confused tool result, and a second model shares a family of blind spots with the first. What the three fixes change is the failure mode: from confident invention discovered at review time, to a missing citation or a failed spot-check caught mid-run. That difference is the whole reason to run agents with receipts instead of vibes.
The alternative worth naming: sometimes the right answer is refusing long tasks. If a task needs forty minutes of unbroken agent time, splitting it into verified stages with human checkpoints in between is not a lack of ambition. It's the same reason we don't let CI hold a merge open for an hour on vibes.
Where did your last long agent run go wrong, and did you catch it in the trace or in production? I'm collecting failure traces and the ones in the middle are always the interesting ones.
Top comments (8)
The part I keep coming back to is treating the state document as a read instead of a recall. Turning “remembering” into a file read is the kind of unglamorous fix that actually holds up over a long run, and I think it gets less attention than the self-correction number. I also appreciated that you named refusing long tasks as a legitimate answer. Too many agent write-ups treat splitting a task into verified stages as a lack of ambition, when it’s often just good engineering.
If I were testing this next, I’d run one long task with the state document and one with a hard checkpoint every ten minutes, then compare not just failure rate but how early the failure gets caught. That’s the number I’d want before asking a team to adopt the pattern.
Happy to compare traces if you’re still collecting them. I’ve got a couple of long-run failures where the agent’s summary drifted from the actual tool output, and a state document would have made the drift visible earlier. Glad to connect with others working on the same problem.
What I don’t think can be argued away is that the citation rule still trusts the tool result and the agent’s reading of it. A confused output or a plausible misread gets a citation and survives into the next step; the state document just makes the fabrication better documented. The self-correction number is real, but the fix that matters most is the one you mentioned last: don’t let the agent be the one who decides the task is done.
A realistic improvement would be to add a “claim → command → expected output” line to the state document for every status claim, and have the verification pass re-run only those commands rather than re-read the document. It keeps the cost low and makes the check independent of the model’s prose.
Mine went wrong at the merge, and I caught it before production only because I asked the agent "Did you do manual QA?" It said no: it had run scripted Playwright tests and had subagents review the screenshots. It then drove the app itself and pushed a PR, which came back conflicting. 42 commits had landed on main since the branch was cut, and they collided on a route-count guard. The clean unit run had been measured on the branch alone, so re-running it there would have passed again.
The 64.5% self-correction failure number is the one that should end the 'agent reviews its own work' habit — the checker shares the same blind spot as the worker. The 'no citation, no entry' rule is the sharpest fix here: it turns honesty from a model property into a ledger property. Does the state document get rewritten fresh each step, or does it grow — and if it grows, doesn't it become the same long context that started the problem?
The failures in the middle are often quiet ones, like a rule from the start that silently stopped applying. A cheap guard is to restate the two or three rules that matter at each stage boundary and have the agent write a one-paragraph status note before moving on. That note also makes a clean restart possible, which is often better than pushing a long session further.