Adding LLM-driven dialogue to NPCs
Every NPC that answers a typed question instead of picking from three dialogue options is running a model call behind the scenes. That call has a latency, a cost, and a failure mode, and none of the three go away because the tech demo looked good.
Most games do not need this. A barkeep with twelve hand-written lines and a good writer beats a barkeep running a 7B local model on a shaky system prompt, every time a real player plays the game instead of poking at the tech. Reach for LLM dialogue when the design genuinely needs open-ended conversation, a game about interrogation, negotiation, or a companion who should remember what you told it. Do not reach for it because static dialogue trees feel dated.
Should the NPC run local or in the cloud?
Local and cloud split on four things that matter once the game ships, not in a tech demo: latency, cost per conversation, offline play, and content control.
| Factor | Local model | Cloud API |
|---|---|---|
| Latency | No network round trip, but bound by player hardware; slow on a weak GPU | Fast on good infrastructure, but adds a round trip that spikes under load |
| Cost per conversation | Fixed: hardware you already paid for, or your own server | Variable: billed per token, scales with every player and every message |
| Offline play | Works with no connection | Dialogue system goes dark without one, unless you build a scripted fallback |
| Content control | You choose the model and the version; it does not change under you | The provider can update the model or its filtering behind the same endpoint |
Ollama is the common way developers get a model running locally without writing inference code themselves. Ollama itself is open source under the MIT license and free to run. That license only covers the runner: the model you point it at carries its own separate license, and those differ by model, so check each one before you ship on it.
Cloud APIs bill per token, which means cost per conversation is not a fixed number you can plan around, it is a curve that climbs with player count and with how long each player keeps talking. That curve is where the second failure mode below comes from. Whichever provider you use, read its current commercial terms before launch; they change, and the wiki’s guide to reading tool terms covers what to look for.
How do you keep an NPC in character?
Three things hold a character together across a conversation: a system prompt, lore grounding, and retrieval instead of memorization.
The system prompt sets the persona, the tone, and explicit limits: who the NPC is, what they know, what they refuse to discuss, and how long a reply should run. Write it like a character brief for an actor, not a list of adjectives, and be specific about what the NPC does not know rather than assuming the model will infer it.
A model has no idea what happened in your game’s lore unless you put it in the prompt, and it does not reliably hold a long lore document in mind across a drifting conversation. Retrieval fixes this: instead of stuffing your entire lore bible into every request, pull just the passages relevant to what the player asked and inject those alongside the system prompt. This matters for cost, since a shorter prompt is cheaper per call, and for accuracy, since models pay less attention to the middle of a long context than to its start and end.
For a single NPC with a modest backstory, a tight 500 to 1000 word character sheet in the system prompt outperforms a full retrieval pipeline and is far less code to maintain. Build retrieval once your lore stops fitting in a system prompt, or once several NPCs need to share one world document without you copy-pasting it into each character’s prompt by hand.
What are the four failure modes that actually block a ship?
These four show up in playtests, not in demos, and each one has stopped a real launch.
| Failure mode | What it looks like | What stops it |
|---|---|---|
| Latency spikes | A player asks a question and waits several seconds while the conversation stalls | Stream the response into the UI as it generates, and put a hard timeout on the call |
| Cost blowups at scale | Launch traffic multiplies conversations at once; a bored player loops an NPC for an hour | Response length caps, a message cap per session, caching repeat questions |
| Jailbreaking off-brand | Players deliberately push the NPC out of character for a screenshot or worse | Prompt injection defense and output scrubbing, covered below |
| Inventing quest items | The NPC names an item, quest, or location that does not exist, and a player goes looking for it | Retrieval grounding plus an explicit instruction to say "I don't know" instead of guessing |
Models default to answering rather than admitting they do not know something, so a single system-prompt line telling the NPC to decline unknown questions rarely holds by itself. Test that instruction directly, with prompts designed to bait an invented answer, before you trust it in a build players can reach.
What does prompt injection defense look like in production?
The AI Game Dev Org’s own Discord bot runs in production and gets probed daily, so the defenses below are the ones that have survived contact with actual players.
Delimiter stripping handles the input side. Player text often carries strings designed to look like a system instruction, “ignore the above” or a fake closing tag meant to break out of the prompt template, and you strip or escape those sequences out of player input before it ever reaches the model.
Output scrubbing handles the other side. Check the model’s response before it reaches the player: strip anything that looks like a leaked system prompt, catch responses that trip your topic fence, and swap in the fallback line instead of shipping the raw output. Neither defense lives inside the prompt itself. The enforcement has to run in code, on every call, because a determined player will eventually find the phrasing that gets the model to ignore its own instructions.
What guardrails actually keep a conversation from going wrong?
Three guardrails do most of the work, and all three are cheap to add:
- 01
Response length caps. A hard token limit on output, set for cost and because a rambling NPC reply is what ends up in a bad screenshot.
- 02
Topic fences. Instruct the NPC not to discuss out-of-universe topics, real politics, real brands, then back that instruction with a keyword check on the output, because the model will not always hold the line under pressure.
- 03
A fallback line. When the call fails, times out, or the output scrubber catches something, show a static in-character line instead of an error message or a blank bubble. "The guard looks away like he didn't hear you" reads as intentional; a stalled loading spinner does not.
Going deeper
Stream the response token by token into the dialogue UI instead of waiting for the full reply. This does not change actual latency, but it changes perceived latency: a barkeep who starts talking after half a second and keeps going feels responsive even if the full answer takes three.
Cache common exchanges. Players ask the same handful of questions constantly, where’s the blacksmith, what’s the fee, so hash the incoming question and serve a cached response for near-duplicates instead of paying for a fresh model call every time. This also fixes part of the cost-blowup failure mode above, since a cached exchange costs nothing.
Small local models are worth a serious look once you have a latency or cost problem cloud calls cannot solve. Open-weight model licenses differ by model and by version: some restrict commercial use above a revenue or user-count threshold, others restrict specific use cases outright. Check the current license for the exact model and version you plan to ship with, since terms change between releases and an older version’s license does not carry over.
Character drift is hard to catch by playing the game once. Build a small evaluation harness: a fixed set of test prompts, including a few designed to bait jailbreaks and invented lore, run against the NPC after every prompt or model change, with the responses checked against your character sheet. Teams that skip this catch drift from players instead, in a bug report or a clip.
If an NPC keeps breaking character or blowing your budget, describe the setup in the AI Game Dev Org Discord; production cases like that show up there often. A few of the dialogue systems on the showcase were built past exactly that point.