Most of us here have watched an agent confidently rewrite a function nobody asked it to touch, then spent the next twenty minutes rewording the prompt to make it stop. This is the writeup I wish I had read before doing that for the tenth time.
When a coding agent edits the wrong function, the instinct is to blame the model. Usually the model was never shown the file that mattered. That distinction sounds pedantic until you start fixing the right layer and the failures stop.
Context Failures Look Like Reasoning Failures
A language model has access to exactly one thing, the tokens in its context window on this call. It does not remember yesterday's session, it cannot see your repo, and it does not know what your tools returned unless that output is sitting in the prompt right now. Everything it appears to know about your codebase is there because some part of your system decided to put it there.
So when the output is wrong, there are two very different questions. Did the model have the right information and reason badly about it, or did it never have the right information at all? In practice the second one is far more common. The support bot that gave the wrong refund answer usually had the correct policy in its knowledge base. The agent that edited the wrong function was usually never shown the file. Rewording the instruction does not fix a window that is missing the one document the answer depended on.
What A Context Assembler Actually Does
Prompt engineering tunes the fixed wording of an instruction. Context engineering builds the system that decides, on every call, which instructions, retrieved documents, memories, tool outputs and history go into the window, then prunes everything else. One is a string. The other is a runtime component with its own logic, and it deserves the same review and testing you give the rest of your code. The full breakdown of the strategies and failure modes is worth a read if you are building this layer for the first time.
The practical shift is that "what does the agent actually see" becomes something you can inspect, log and diff. Once you can print the assembled context for a failing run, most of the mystery evaporates.
The Token Budget Is The Whole Constraint
The window is finite and more context is not free. Models that advertise very large windows still degrade as those windows fill, because the relevant tokens get diluted by irrelevant ones and attention spreads thinner. Every token you add also costs money and latency on every single call.
That makes this an optimization under a budget, not a collection problem. Get the few thousand tokens that actually matter for this request into the window, leave the rest out, and do it fast enough to run in production. A team that stuffs everything it has into a large window will pay more, respond slower and get worse answers than a team assembling a lean, relevant context per call.
Where Memory Fits
Retrieval solves "find the document." It does not solve "remember what we decided last Tuesday." A memory layer is what lets the assembler pull in prior decisions, established conventions and accumulated project knowledge without re-sending the entire history every time. It is the difference between an agent that starts from zero every session and one that gets more useful the longer you work with it.
The Takeaway
Before you swap models or rewrite the system prompt again, log the exact context the failing call received. If the answer was not in there, no model was going to get it right and no prompt wording was going to save it. Fix the assembler, and a surprising number of "the AI is dumb" bugs turn out to be plumbing.
