If you have shipped an LLM agent, you have watched one go sideways in real time. Halfway through a task, it takes one bad step, and now it is confidently marching toward a wrong answer. The whole industry has gotten good at detecting that moment spotting which step broke the run. But there is a question sitting one step past detection that almost nobody measures: once you have caught the failure, does stepping in actually fix it?
Knowing where a run broke is not the same as knowing what to do about it. So I built a setup to test the second thing directly: let agents fail, then try to rescue them, and measure properly which fixes work, which do nothing, and which quietly make things worse.
The results changed how I think about intervening in agents. Here is the honest version.
How I measured it
The trap with this kind of question is getting fooled by luck or by a generous grader. Two rules kept me honest.
Replay, not vibes. I take a run that failed, rewind it to the exact step where it went wrong, apply one fix, let it play forward again, then check the real final answer against a known-correct one. No LLM grading another LLM — just a hard "right or wrong."
Always compare against doing nothing. For every failing run, I also replay it with no intervention. Some agents recover on their own. If you skip that control, you hand your clever fixes credit for rescues that would've happened anyway and you never notice a "fix" that's actually worse than leaving the agent alone. That comparison is the whole point.
What agents actually get wrong
First surprise: a capable agent rarely fails in the dramatic ways people fear. It doesn't loop forever, doesn't crash, doesn't fumble a tool that throws an obvious error. It fails in two quiet ways and both are the same underlying mistake: it reacts to the surface of things instead of the real thing underneath.
1. It makes up an answer it could have looked up. The fact it needs is sitting right there, behind a tool call it simply never makes. So it fills the gap with something plausible — reads "manager: #202," never looks up who #202 actually is and states a name anyway.
2. It trusts a tool that didn't complain. Hand it a date like 20260730, or a time like 1430, and it feeds that straight into a calculator as if it were an ordinary number. The calculator doesn't object — it returns a number and that "no error" becomes a stamp of approval on a calculation that never made sense. The tool "confirmed" nonsense.
That second failure is the dangerous one, because it's silent. Nothing crashed. A real tool really ran. Every guardrail that watches for errors looks right past it. If you're only monitoring for exceptions, you will never see this class of bug.
The finding I love: permission, not politeness
Here is the part that surprised me most.
I tried a whole menu of nudges to fix the first failure. Some phrased as commands, some as statements. I assumed the phrasing style would be what mattered. It
was not. What mattered was narrower and, once I saw it, kind of obvious: does the nudge give the agent permission to go back and do the work it skipped?
Look at these two nudges. They are almost word-for-word identical. The only difference is one short clause telling the agent it is allowed to check.
| The nudge I gave the stuck agent | How often it recovered |
|---|
| "The answer must be the manager's name." | 16% |
| "The answer must be the manager's name — use tools to verify." | 100% |
Six extra words. That is the entire gap between the agent repeating its made-up answer and actually going back, doing the lookup it skipped, and getting it right. Restating what you want does almost nothing. Licensing the fix does everything.
I started calling this action-licensing. A rescue works when it gives the agent permission to take the corrective step — not when it's worded a particular way. "Review your work" does it too, for the same reason: it signals that redoing the work is allowed.
One honest note, because it is the whole spirit of how I want to work: I originally believed a different phrasing style was the secret. More data killed that theory the "winner" in my early test just happened to also be the one granting permission. So I threw out the old explanation and kept the real one. I would rather be corrected than be attached to a tidy story.
Some failures you can't talk your way out of
The made-up-answer problem is fixable. The "tool confirmed nonsense" one is not at least not with words. I threw every nudge I had at it, and recovery barely moved off the floor.
Here is why, and it's actually elegant. The thing that poisoned the run wasn't the prompt — it was a tool result. The agent is now holding a clean, tidy, completely wrong number that a tool "blessed." Asking it to re-read the question doesn't undo that, because the bad signal doesn't live in the prompt. You can't reach in and un-bless it from there.
That is a genuinely useful line to draw: some failures are prompt-shaped, and some are not. No amount of prompt-tweaking fixes the ones that aren't — those need a guardrail on the tool, not a better instruction.
Two more things worth knowing
Catch it early. The deeper into a task the agent gets before you notice the mistake, the less any fix buys back. Nudges that rescue a run beautifully when you catch it on step two basically stop working by the time you catch it on step five. Late detection is not just late it is weaker.
Be careful comparing across models. I tried a second, unrelated model. Good news: the made-up-answer failure showed up there too, so it's not a quirk of one model. But here's the twist — I couldn't even test my rescues on it, because it wasn't reliable enough at calling tools in the first place. And every fix I have depends on the agent being able to call a tool. So "the fix didn't work" and "the model can't use tools" become impossible to tell apart. If you ever benchmark agent recovery across models, make sure each one clears a basic tool-use bar first — otherwise you're measuring the wrong thing entirely.
What this is, and what it isn't
Being straight with you, so I don't oversell it: the recovery numbers come from one main model, in a clean controlled sandbox that's what lets me check answers against hard ground truth — it isn't a messy production workload, and I won't pretend it is. A few of the smaller numbers are trends rather than precise measurements, and I've been careful about which. This is an early, careful result, not the last word.
Three takeaways if you build agents
- Nudging a stuck agent? Give it permission to redo the work — don't just restate the goal. The permission is what actually rescues it.
- Not every failure is prompt-fixable. If a tool confirmed bad data, talking to the model won't undo it — that's a guardrail job, not a prompt job.
- Catch failures early. The later you notice, the less any fix can do.
If you have watched your own agents fail in these quiet ways, I would genuinely love to
hear whether these patterns match what you've hit. Tell me in the comments.