System prompts that hold up in production: structure, refusal behavior, format contracts and testing prompts as code.
Prompts belong in version control, with review and an evaluation set that runs on every change. The biggest quality incidents in production AI come from "just tweaked a word" without a test, exactly what we stopped accepting for code twenty years ago.
Version control
Prompts belong in Git, with history and traceability.
Every change runs the evaluation set
Treat prompts as code: in version control, with review and an evaluation set that runs on every change.
A good system prompt has recognizable blocks: role and context, the task, hard rules (what always and what never), the output format, and examples. Models follow structure better than long running text, and your team can review a structured prompt.
Models follow structure better than long running text, and your team can review a structured prompt.
The most important rule in every enterprise prompt describes what the model does when the answer is not in the context: say so explicitly, and do not fill in from its own knowledge. Without that rule the model fills gaps with plausible nonsense, with a citation attached, which makes it extra convincing.
Without a refusal rule / With explicit refusal
The most important rule describes what the model does when the answer is not in the context.
Two or three good examples steer more than ten. And one good counterexample ("for this question, refuse") prevents a category of errors that positive examples never cover.
Ten examples / Sparing + counterexample
Be sparing with examples, generous with counterexamples. A good counterexample prevents a whole category of errors.
If the application expects JSON, describe the schema exactly and validate the output. Count on it going wrong one in so many times and build a recovery path (ask again with the error message) instead of hoping for perfection.
Request
The user request arrives at the agent or RAG pipeline.
Expect occasional errors, build a retry
If the application expects JSON: describe the schema exactly, validate the output and build a recovery path.
Every extra instruction dilutes attention for the other instructions. Prune periodically: remove a line and see whether your evaluation set notices. Prompts grow organically; quality comes from pruning, not from stacking.
Stacked / Pruned
Every extra instruction dilutes attention for the others. Quality comes from pruning, not stacking.
Click a step for its key decision
Prompts belong in version control, with review and an evaluation set that runs on every change. The biggest quality incidents in production AI come from "just tweaked a word" without a test, exactly what we stopped accepting for code twenty years ago.
Related how-tos
Related use cases
