menu
the interesting test is not what an agent can do, but what it takes to stop it

the interesting test is not what an agent can do, but what it takes to stop it

●5 ●21
calendar_today ago • schedule1 min read

Most reviews of an AI agent focus on capability. Can it read the right records, does it hallucinate, is the output good enough to send to a customer. Sensible questions. They are also the wrong place to spend your caution.

An agent that is right 90% of the time and can be switched off in sixty seconds is safer to run than one that is right 99% of the time and takes a change-management window to stop. Accuracy degrades quietly. A missing off switch fails all at once, and always at the worst moment.

The blocker is rarely technical. It is that the agent borrowed a human's credential, and three other things quietly borrowed the same one. Nobody wrote that down.

So the question worth asking before deployment is not how good it is. It is: if this thing starts doing the wrong thing tonight, who can stop it, and what else goes dark when they do?

https://unlockedconsulting.ai/blog/ai-agent-identity-access-revocation

4 Comments

•

Spot on, Helkyn!

Everyone is obsessed with model accuracy metrics, but operational security lives in the control plane, not the prompt window.

Shared credentials and implicit rights are an absolute nightmare for agentic workflows. When an agent runs on borrowed human tokens instead of an isolated, revocable sandbox with explicit state boundaries, you haven't built an autonomous worker — you've just built a stealth liability.

If you can't hit a single kill switch without turning off half the infrastructure, the agent was never production-ready to begin with. Brilliant insight.

1 vote
•

@[ManSio] Thanks. "Stealth liability" is the phrase I was missing, and it names why this stays invisible: nothing is broken, so nothing gets escalated.
The one place I'd push is the last line. Half the agents I have seen in production would fail that test, and they are running anyway, doing useful work. So it is not that they were never production-ready. It is that the kill switch got deferred and then shipped that way. And that is a different issue, probably a more common one.

2
•

@[helkyn-coello]
"Touché. 'Deferred and shipped anyway' is definitely the pragmatic reality of most engineering teams right now.

My main worry is that retrofitting isolation onto an active agent is 10x harder than building it into Layer 0. When deterministic boundaries and revocation aren't baked into the agent's runtime from day one, 'fixing it later' usually means waiting for the first major incident to force a painful refactor."

1
•

@[ManSio] Agreed. And for most teams that means the incident isn't a risk, it's the plan — just not one anybody wrote down.

2
🔥 Join developers growing publicly
Share your knowledge, build in public, and grow your developer presence with a global community.
Join CoderLegion

More Posts

AI Agents Don't Have Identities. That's Everyone's Problem.

Tom Smithverified - Mar 13

The Foundation Gap & Agentic Trust Engineering

snapsynapseverified - Apr 15

What Is SARIF and How Does It Help Security Tools Work Together?

Ganesh Kumar - Jul 4

Helping Clients Move from Pilot to Production: The Agentic AI Governance Playbook

Tom Smithverified - Jun 8

From Prompts to Goals: The Rise of Outcome-Driven Development

Tom Smithverified - Apr 11
chevron_left
Helkyn Cello
754 Points • 26 Badges
Munich • unlockedconsulting.ai
9Posts
11Comments
3Connections
I have spent twenty-five years inside one company watching technology change in cycles — from deskto... Show more

More From helkyn-coello

Can a Non-Coder Become a Coder Just With AI?

AI Doesn't Think For You, It Thinks Like You

My two days at the Milano AI Week 2026

Related Jobs

View all jobs →

Commenters (This Week)

1 comment
1 comment
1 comment

Contribute meaningful comments to climb the leaderboard and earn badges!