Every demo of an AI NPC looks the same: a player asks something clever, the NPC answers in character, everyone in the comments says this changes everything. Then you try to ship it and hit the part demos never show, which is what happens on the ten thousandth conversation, not the first one.
Where live generation genuinely helps
Barks, ambient chatter, and any dialogue where being slightly wrong costs nothing, that's the sweet spot. A tavern NPC commenting on the weather or reacting generically to the player's gear doesn't need to be scripted, and live generation gives you infinite variety for near-zero authored content. Players notice and appreciate this kind of texture even when they can't say why.
Where it breaks something you'll regret
Anything load-bearing for plot, a quest-giver's instructions, dialogue that has to reference a specific game-state fact correctly, is the wrong place for an ungrounded model. The failure isn't a crash, it's an NPC that confidently tells the player the wrong objective, or breaks tone by referencing something that doesn't exist in your world. Players forgive a generic line. They don't forgive being sent to the wrong location by an NPC who sounded certain.
Grounding is the whole game
The gap between "cool demo" and "shippable" is almost entirely about grounding: feeding the model your actual quest state, world facts, and character voice as context on every call, and constraining its output to only reference what's true right now. Ungrounded generation is a slot machine. Grounded generation is a writer with a very fast typing speed and a strict style guide.
The honest scope for most teams
Full live generation for every line, the sci-fi demo version, is still expensive, latency-sensitive, and hard to keep on-brand at scale. A more common and much cheaper approach: pre-generate a large bank of barks and reactions offline (where you can review and cut the bad ones), and reserve live generation for the handful of moments where dynamic really does beat scripted. Most players can't tell the difference, and your QA pass gets a lot shorter.