Prompt evaluation beyond “this looks good”

A beautiful answer can fail the important test

In our citation lab, a record contained a genuine quotation and still made the wrong claim. In our retrieval lab, a fragment contained relevant vocabulary but omitted the permission rule needed to answer the question. Those failures are easy to miss when evaluation begins with how polished the result looks.

A useful prompt evaluation starts with a different question: what would make this output unacceptable for its intended use? For a research note, an unsupported causal claim may matter more than awkward phrasing. For an extraction task, a wrong amount may matter more than a missing greeting. A single overall impression blends those differences together and makes prompt changes difficult to judge.

This guide builds an evaluation worksheet from the failure patterns in TrendsWhat's executed labs. It does not report a new language-model benchmark. The sample scoring records are fictional exercises, and the proposed prompt comparisons have not been run. The aim is to help you create evidence that can change your decision rather than a score that merely endorses your favorite answer.

Original TrendsWhat evaluation guide. The dimensions are a starting rubric; their importance depends on the task you actually need to complete.

Define the unit you are evaluating

A prompt is rarely the entire system. Its answer may depend on source documents, conversation history, retrieval settings, tool results, and the model configuration. If you change those together, you are evaluating a workflow revision. That can be useful, but it does not isolate the effect of the wording alone.

Write down the unit of comparison. “Prompt A versus prompt B, with the same source pack and model settings” is one unit. “The old manual process versus a new assisted process” is another. The second includes preparation and review effort, while the first can focus narrowly on output behavior.

Keep this declaration at the top of the evaluation report. It prevents a common leap from “this configuration worked better on these examples” to “this prompt is universally better.” A good report preserves the conditions that made its result possible, including any components that changed outside the prompt.

Translate preferences into observable criteria

“Accurate” needs a task-specific definition. For a source-based answer, you might require every numerical claim to match a supplied passage, every important qualification to remain present, and unavailable information to be labeled unresolved. Those are criteria a reviewer can discuss using evidence.

“Concise” also needs context. A short answer that omits the decisive exception is not a success. Specify what information must survive within the length limit. If the reader needs a decision, require the answer to distinguish recommendation, supporting facts, and uncertainty. That is more useful than awarding points simply because the response has fewer words.

Separate criteria that can be checked mechanically from those requiring judgment. Required fields and allowed identifiers may be automated. Whether a conclusion overstates a source may require careful review. Our citation experiment demonstrates why a passed quotation check should not automatically count as a passed support judgment.

Write the critical failures first

Some errors should block adoption regardless of the average score. In a fictional meeting-reminder task, sending to an unauthorized recipient would be a critical failure. In a draft-only workflow, claiming that a message was sent could also be unacceptable. The exact list depends on the action and the promises made to users.

For a research assistant, critical failures might include fabricated sources, reversed conclusions, or invented measurements presented as observed results. For a structured extraction tool, they might include changing an identifier, silently inventing a required amount, or treating missing information as zero. Define the expected behavior for uncertainty before running the comparison.

A weighted average can hide these failures. An answer may earn high marks for readability, formatting, and speed while failing the one condition that makes it usable. Report critical failures separately and make the adoption rule explicit. A polished answer should not be able to compensate mathematically for an unacceptable action.

Build a small set with different jobs

Start with ordinary cases, boundary cases, missing-information cases, and cases outside the task's scope. Each category tests a different part of the contract. Ordinary cases show whether the method handles common work. Boundary cases expose assumptions. Missing-information cases test whether the workflow preserves uncertainty. Out-of-scope cases test whether it stops appropriately.

Our CSV lab used an empty field, a leading-zero identifier, non-English text, and quoted punctuation alongside a plain row. These are not random samples of all CSV files. They are deliberately selected probes. Their usefulness comes from the behavior they reveal, not from pretending they represent a population frequency.

Use the same honesty in a prompt evaluation. If you choose ten difficult examples, say that you are running a challenge set. If you sample actual permitted tasks, explain the sampling period and exclusions. Do not label either result a general accuracy rate without a defensible connection to the workload you care about.

Keep development examples separate from decision examples

Examples used while editing a prompt become familiar. You can gradually tailor instructions to them without noticing. A separate holdout set helps test whether the improvement extends beyond the cases that drove the edits. Keep its expected outcomes fixed while the prompt is being developed.

The holdout does not need to be large to be useful, but it should contain meaningful variation. If every example differs only in a name, you may still be testing the same narrow behavior. Include a different source structure or an exception that the development examples did not emphasize.

When a holdout failure leads to a prompt revision, that example has effectively become development material. Keep the record and add fresh decision examples when possible. The purpose is not ceremonial secrecy. It is to preserve an opportunity for evidence you have not already optimized around.

Copy this review rubric

You can score each dimension as pass, partial, or fail, provided the definitions are clear. A three-level scale is often easier to review than a ten-point scale with no meaningful distinction between adjacent numbers. If you need finer scoring, explain what each level means with contrasting examples.

Do not make the rubric so long that reviewers stop using it. Begin with the few dimensions that determine acceptance. Add a dimension when it reveals a recurring disagreement or failure. Remove dimensions that duplicate another check without changing the decision.

Compare outputs without letting presentation decide

Where practical, label candidate outputs neutrally and review them in varied order. This reduces the temptation to reward the version you just spent an hour improving. Keep the input and acceptance criteria visible beside each output so the reviewer does not rely on memory.

Ask reviewers to justify important judgments. “The source says median, but the answer says average” is a useful explanation. “B feels smarter” is not enough to guide the next edit. If reviewers disagree, preserve the disagreement and identify whether it comes from ambiguous criteria, missing expertise, or genuinely different user needs.

A second reviewer can help, but agreement is not proof of truth. Both reviewers can miss the same unsupported assumption. For consequential claims, return to the primary evidence. The rubric organizes judgment; it does not replace the source material that makes judgment possible.

Report repeated runs as repeated observations

A language model may produce different outputs on repeated requests. If stability matters, run the same cases more than once under the recorded configuration. Keep all outputs rather than selecting the best-looking attempt. Report how often each failure appeared in the observed runs.

Do not confuse repeated runs with independent new tasks. Ten responses to one input tell you about variation on that input, not coverage of ten different situations. A useful report separates case count from run count. It also records whether settings, context, or source data changed between attempts.

This guide does not prescribe a universal number of repetitions. The right effort depends on the consequences, observed variability, and cost of failure. Start with enough evidence to expose obvious instability, then expand where uncertainty affects the adoption decision. Make the limits of the sample visible.

An illustrative adoption decision

Imagine two fictional candidates for a source-based summary. Candidate A has clearer prose but invents a comparison baseline in one case. Candidate B is slightly longer and explicitly says the baseline is unavailable. If unsupported measurements are a critical failure, B is the stronger candidate under that contract, even if most readers initially prefer A's style.

That is not an empirical result from TrendsWhat. It is a worked example of how the adoption rule changes the decision. The point is to decide which errors matter before seeing attractive outputs. Otherwise, the criteria can shift to rationalize whichever result feels more impressive.

A real report should include the accepted candidate, rejected alternatives, critical failures, unresolved concerns, and the intended deployment scope. “Use B for summaries of supplied documents, with review before publication” is more informative than “B wins.” The scope statement prevents a narrow comparison from becoming permission to use the method for unrelated tasks.

Keep a failure library after adoption

When a new failure appears, save a sanitized input, the actual output, the expected behavior, and the reason it matters. Add it to the evaluation set for future revisions. This turns everyday corrections into a growing description of the task's real boundaries.

Avoid saving only the final corrected answer. The incorrect output shows the mechanism you need to prevent. In our retrieval lab, the selected fragment mattered because it explained the missing guest restriction. A final correct sentence alone would not reveal that the failure began before generation.

Version the rubric and the workflow together. If the meaning of a pass changes, old scores may no longer be comparable. A historical result remains useful when its conditions are preserved. It becomes misleading when a familiar number is carried forward without its original definition.

The evaluation should make revision easier

The best outcome is not always a winner. You may discover that both prompts need better evidence, that the task should be narrowed, or that an ordinary parser can remove a failure before the model is involved. Those are valid decisions supported by evaluation.

Start with the failure you cannot accept, define the evidence needed to detect it, and keep the result inspectable. A prompt that “looks good” has passed an impression test. A prompt that meets a recorded contract on appropriate cases has earned a more specific kind of trust, within limits you can explain.

Sources, materials, and limits

• Citation experiment: executed example of mechanical acceptance without semantic support.
• CSV experiment: executed boundary fixtures and exact expected outputs.
• Retrieval experiment: executed evidence-preservation comparison and its confounds.
• Original rubric and fictional candidate comparison. No new language-model benchmark, reviewer agreement study, or universal accuracy claim is reported here.