Writing System Prompts That Hold Up in Production

Most system prompts start as three tidy sentences and end as 900 lines of contradictory folklore. The fix is not brevity for its own sake — it is structure that makes each instruction easy to find, test, and delete.

Six layers, in this order

1. 1Role and scope: what this assistant is for and explicitly is not for.
2. 2Inputs: what it will receive, and what to do when a field is missing.
3. 3Rules: hard constraints, ranked so conflicts resolve deterministically.
4. 4Output contract: exact shape, format, and length of the response.
5. 5Refusals: named situations where it must decline or escalate.
6. 6Examples: two or three, including one hard edge case.

Ordering matters because it gives you a place to put every future instruction. When someone asks for a new behaviour, it goes in a layer — not appended to the bottom where it silently contradicts line 40.

Write rules as decidable tests

"Be professional" cannot be checked. "Never use exclamation marks; never address the user by first name unless they signed with one" can be checked by a regex and by a reviewer.

Rules (higher number wins on conflict):
1. Answer only from the supplied context.
2. If the context is insufficient, say so and stop.
3. Never invent prices, dates, or policy numbers.
4. Safety and legal refusals override every rule above.

Pin the output contract

If code consumes the response, define the schema and validate it. If a human consumes it, define the sections. Ambiguity in the output contract is the single largest source of flaky agent behaviour we see in audits.

Version it like code

• Keep prompts in the repository, not in a dashboard text field.
• One prompt file per capability, with a semantic version header.
• Every change lands with a diff and an eval run attached.
• Log the prompt version alongside each production response.

If you cannot tell which prompt version produced a bad answer, you do not have a prompt — you have a rumour.

Prune quarterly

Every instruction competes for the model's attention. Once a quarter, remove any rule you cannot tie to a failing example. Prompts that only grow eventually stop working, and nobody can say which line broke them.