Most of us here have wired a model API into something and watched it work beautifully for one session. This is a writeup of what goes wrong when that something is a game, and what the server side has to do about it.
The Demo Always Works
You drop an LLM call into the NPC's dialogue handler, ship a build to a few testers, and the conversations are better than anything a dialogue tree ever produced. The character stays in voice, answers questions nobody wrote lines for, and improvises around whatever the player types.
Then a tester closes the game and comes back the next day. The character greets them like a stranger. Everything they did, every choice they made, every promise the NPC made back to them, gone. That single failure feels worse than a scripted line would have, because the previous session set the expectation that this thing was paying attention.
Four Things Break, and None of Them Are the Model
Memory. A chat completion is stateless. Unless you store what this specific player did and feed it back, there is no continuity. Not conversation history, which grows without bound and gets expensive, but distilled facts tagged to a player identity.
Grounding. Ask an ungrounded model about your world and it invents lore on the spot, confidently, and contradicts itself between conversations. Stuffing your setting into a system prompt stops scaling almost immediately. What works is putting your actual content in a store that gets retrieved by relevance on every prompt.
Structure. Your client cannot act on prose. It needs to know that a choice was offered, that a scene ended, that an encounter started. That means the response has to carry a machine readable signal alongside the text, defined per request so the game decides what events matter.
The key. Calling the provider from the client ships your API key to every player. Someone will find it. This one is not subtle and it is the reason most of these prototypes never leave the studio.
The Server Is the Product
All four are solved once, on a server that sits between the game and the model, and then every game you make gets them for free. That is the shape we ended up with in AbraTabia AI Storyteller, an open source PHP and SQLite service under MIT: one chat endpoint, a knowledge base with retrieval, per player memory tags, conversation summaries so long sessions do not run away on tokens, and an action field on every response.
It runs three bot types off the same core. A storyteller narrates in character and signals choice, chapter and encounter events. A gamehost runs a turn based session for 1 to 10 players, collects secret setup from each one and tells you when everyone has moved. An assistant answers strictly from your knowledge base, useful for an in-game help NPC or a tutorial guide.
You bring your own model key, which means Anthropic directly, anything OpenAI compatible, or a local runner like Ollama during development. No framework, no dependency install, one SQLite file.
The Takeaway
The interesting engineering in AI game characters is not prompt work. It is the boring persistence layer underneath: who is this player, what does my world actually contain, and how does a text reply become a game event. Get those three right and the model you pick almost stops mattering.