How to optimise a prompt (and why most prompt enhancers don't)
Prompting for reliable answers · Facts checked 22 October 2026
Improve a prompt by testing it, not rewriting it. Write down what a good answer must contain, collect a dozen real inputs including awkward ones, score the current prompt, then change one thing at a time and rescore. Often the best change is a deletion: OpenAI found leaner system prompts scored roughly 10–15% higher in its internal evals.
Most prompt enhancers make your prompt longer, not better
Paste "analyse our churn" into a typical prompt enhancer and you'll get back three paragraphs. A role ("You are an expert data analyst with 15 years of experience"), a numbered method, a format spec, perhaps a reminder to be thorough. It looks more professional. Whether it works better is a question the tool never asks.
That's the honest limit of one-click enhancers: they can't know your success criteria. They don't know which churn definition your board uses, which edge cases break your data, or what last month's report looked like. So they add what's generic, and generic is mostly length. Sometimes that helps; a vague one-liner often benefits from being told the audience and the output format. Sometimes it hurts, because every extra instruction is something else for the model to weigh.
The vendors are more careful than the tools that borrow their name. OpenAI's own dashboard optimiser rewrites prompts against a dataset of your examples, graded by criteria you set, and its documentation still warns that the results depend on "the quality of your graders", that the result can "perform worse than your original on specific inputs", and that you should "always evaluate and manually review" the output before using it (OpenAI developer docs). OpenAI is also retiring that dataset-backed version along with its Evals platform, which becomes read-only on 31 October 2026 and shuts down on 30 November 2026.
Our position: an enhancer is fine for a first draft. Optimising is a different activity, and it needs test cases.
The five-step loop that improves a prompt
Anthropic's overview says you need three things before you start: "a clear definition of the success criteria", "some ways to empirically test against those criteria", and "a first draft prompt" (Anthropic, Prompt engineering overview). Everything below follows from that.
- Write the success criteria. Make them specific and measurable. Anthropic's example: instead of "good performance", specify "accurate sentiment classification"; and "use quantitative metrics or well-defined qualitative scales" (Anthropic, Define success criteria and build evaluations). For a work prompt, criteria look like: every figure matches the source; the metric definition is stated; missing data is flagged rather than estimated; under 200 words.
- Build a test set. Real inputs, not invented ones. Anthropic's advice is to "design evals that mirror your real-world task distribution" and "don't forget to factor in edge cases". It also prefers more cases with rougher grading to a few perfectly marked ones. A dozen to twenty inputs is plenty for a team prompt.
- Score the current prompt on every case. This is your baseline; without it, "better" is a feeling.
- Change one thing, rerun everything. OpenAI's wording for this is exact: "Remove one group of instructions, examples, or tools at a time, then rerun the same evals" (OpenAI, Using GPT-5.6). Change two things and you won't know which one helped.
- Keep a log. One line per change: what you changed, why, and the score before and after.
If you grade with a model rather than by hand, Anthropic suggests it's "generally best practice to use a different model to evaluate than the model used to generate the evaluated output". A model marking its own homework is generous.
The same logic holds once the prompt ships: something other than the model should mark the answers. Count connects to Claude and audits each answer claim by claim, so you can see whether the new prompt produced better-supported answers.
Test the prompt, then check the answers
A better prompt lowers the error rate; it doesn't tell you which answer is the error. Count shows the reasoning chain under every claim, so you can see what each answer rests on before it ships.
Cut before you add
The most reliable improvement to an old prompt is usually removing something.
OpenAI tested this on its own coding agents. Configurations with leaner system prompts "improved evaluation scores by roughly 10–15%" while "reducing total tokens by 41–66% and cost by 33–67%" (OpenAI, Using GPT-5.6). OpenAI calls the ranges "directional", and they come from coding tasks, so don't expect the same numbers on a reporting prompt. The direction is the point. Its GPT-5.5 guide says the same thing in plainer words: "Avoid carrying over every instruction from an older prompt stack", and save ALWAYS and NEVER for rules that are true invariants.
Anthropic's guidance points the same way from a different angle. Newer Claude models are "more responsive to the system prompt than previous models", so emphatic rules written for older models now overshoot; "CRITICAL: You MUST use this tool when..." becomes "Use this tool when..." (Anthropic, Prompting best practices).
In practice, a cutting pass looks like this:
- Delete any instruction that repeats another one. "State each instruction once" (OpenAI).
- Delete rules added to fix a model you no longer use, then test whether the problem comes back.
- Replace prohibitions with one sentence of reason. Anthropic's example swaps "NEVER use ellipses" for an explanation that the text will be read aloud.
- Delete "think step by step" on reasoning models; the effort setting does that job now. See prompt engineering techniques.
A worked example: a variance-commentary prompt
This example is illustrative: the prompt and test cases are invented to show the method, not results from a real run.
A finance team uses this prompt to draft monthly commentary on budget vs actuals:
You are an expert FP&A analyst. Write insightful, professional commentary on the variances in the attached budget vs actuals file. Be thorough. ALWAYS explain the drivers. NEVER make mistakes. Use bullet points.
Success criteria. Every variance quoted matches the file. Only variances above the materiality threshold (say 5% and £10k) are discussed. Each driver is either supported by a note in the file or labelled as a hypothesis. Favourable and adverse are used correctly for cost lines.
Test set. Twelve past months, chosen to include the awkward ones: a month with a reclassification between cost centres, a month where actuals for one department are missing, a month with no material variances at all, and a month with a one-off accrual reversal.
Changes, one at a time:
| Change | Why we expect it to matter | What to watch on the test set |
|---|---|---|
| Delete "expert", "insightful", "thorough", "NEVER make mistakes" | Adjectives add length, not accuracy | Does length fall without losing any criterion? |
| Add the materiality threshold | The prompt never said which variances count | Fewer immaterial lines discussed |
| Add "label any driver not stated in the file as a hypothesis" | "ALWAYS explain the drivers" invites invention | Unsupported drivers now labelled |
| Add "if a department has no actuals, say so and skip it" | The missing-data month | No figures invented for the gap |
| Ask for calculations in code | Variances are arithmetic | Quoted figures match the file |
The rewritten prompt ends up about the same length as the original, but every line now maps to a criterion and a test case. That's the difference between an optimised prompt and an enhanced one.
Use the model as a critic, not a rewriter
Claude and ChatGPT are useful in this loop, as long as they propose changes rather than rewrite everything. Anthropic suggests asking Claude to evaluate your examples "for relevance and diversity". The same idea works for the whole prompt:
Here is a prompt, the success criteria it must meet, and two test inputs where it failed, with what went wrong.
<prompt>[paste]</prompt>
<criteria>[paste]</criteria>
<failures>[paste the inputs and describe each failure]</failures>
Suggest the single smallest change to the prompt most likely to fix these failures without breaking the other criteria. Explain which failure it addresses and what could get worse. Do not rewrite the whole prompt.
"Single smallest change" keeps you in the one-change-at-a-time loop. "What could get worse" forces the trade-off into the open. And because you supply real failures, the suggestion is about your task, not about prompts in general.
An optimised prompt can be wrong more confidently
Optimisation has a failure mode of its own: you get better at whatever you measure. If your criteria are mostly about format and tone, the prompt will improve on format and tone while accuracy stays flat or slips, and the answers will look more trustworthy than before. OpenAI's warning about graders applies to hand-scoring too.
So put at least one accuracy criterion in every test set, and one case designed to have no good answer. A prompt that has learned to always produce a polished paragraph will produce one for the question the data can't answer.
Two-minute check after any prompt change
- Run your three hardest test cases, not your three easiest.
- Recompute one figure from the source for each.
- Ask a question the material can't answer. The new prompt should still say so. (We cover that "out" in prompts that cite sources.)
- Compare old and new side by side on one case and list what the new version stopped saying.
Point four catches the quiet regressions, such as a caveat that disappeared when you asked for shorter answers. For checking individual answers once a prompt is in use, Count labels each claim as a citation, an assumption, model knowledge or a hallucination, with the reasoning chain behind it, so a prompt change that starts producing unsupported claims shows up in the answers themselves.
Questions people ask
Do prompt optimiser tools work?
For turning a vague one-liner into a clearer brief, often yes. For improving a prompt you already use, only if they test against your own examples: OpenAI's own documentation says the result can perform worse than your original on specific inputs and should be reviewed before use. Without test cases, a rewrite is a guess.
What is the best prompt enhancer?
The one that asks for your test cases. A tool that rewrites a prompt without knowing what a good answer looks like can only add generic structure. Claude or ChatGPT can act as a decent critic if you give them your criteria and real failures and ask for one change at a time.
How do I improve my ChatGPT prompts?
Write down what a good answer must contain, try the prompt on a dozen real inputs, and change one thing at a time. Start by cutting: OpenAI advises against carrying over every instruction from an older prompt, and found leaner prompts scored better in its own evals. Our system prompt guide has a worked example.
How many test cases do I need to optimise a prompt?
Enough to cover your real inputs and their awkward cases; a dozen to twenty is a practical start for a team prompt. Anthropic's eval guidance favours more cases with simpler, automatable grading over a few hand-marked ones.