Most of us here have shipped at least one LLM feature that started as a chatbot and quietly grew a job queue behind it. That is the moment the architecture stops being a chat loop, and it is worth naming before the refactor picks you instead.
The Verb Test
The cleanest way to tell which one a requirement describes is to look at the verb in the ticket.
"Answer questions about our returns policy" is a chatbot. The output is a message, and the message is the end of the work.
"Watch the returns queue, spot the repeat offenders, and open a ticket" is an agent. The output is a change in state somewhere outside the conversation.
That one distinction predicts most of what follows, because everything expensive in an agent exists to make that state change safe.
What An Agent Carries That A Chatbot Does Not
A chatbot is three layers: parse the input, call the model, render the reply. You can hold the whole thing in your head.
An agent adds four things a chat loop never needed. Tool use, so it can reach APIs, databases, files and browsers. Memory that survives the session, so it accumulates instead of resetting. Planning, so a goal decomposes into ordered subtasks with real dependencies. And reflection, so it can notice a failed step and try another path instead of returning a confident wrong answer.
Take any one of those out and you are back to a chatbot with extra steps. That is why bolting a function call onto a chat endpoint rarely feels like the demos, the loop is missing the part that decides what to do when the call fails. The full comparison of the two architectures walks through each layer side by side if you want the longer version.
Where The Cost Actually Lands
The token bill is the part everyone estimates and the part that matters least.
A chatbot's cost is roughly linear in conversations. An agent's cost is a function of steps per goal, and steps per goal is a function of how often it has to retry. A planner that picks a bad tool on the first attempt does not cost you one extra call, it costs you the whole branch underneath it.
The other cost is operational and it never shows up on a pricing page. Once a system can modify a database or send mail, you need permissions, an audit trail, and a way to answer "what did it touch last Tuesday". Teams that skip this ship fast and then freeze, because nobody will let the thing near production.
The Hybrid Most Teams Ship
The interesting production systems are rarely pure. A chatbot front end handles the predictable majority, the FAQ, the order lookup, the guided flow, cheaply and at a latency users tolerate. Behind it, an agent picks up the requests that need several steps and real tool access.
The routing decision between them is where the engineering lives. Route too eagerly to the agent and you pay agent prices for questions a lookup would have answered. Route too conservatively and users hit the wall the chatbot cannot cross, which is exactly the experience that makes people stop trusting the feature.
The Takeaway
Do not pick the label first. Read the verb in the requirement, decide whether the output is a message or a state change, and let that choose the architecture. If the answer is a state change, budget for retries, permissions and an audit trail on day one, because those are not features you add later, they are the reason the agent gets to run at all.
