Turning one AI experiment into a reusable skill

Save the reason the result changed

The most reusable artifact from our CSV lab is not the final chart. It is the small collection of inputs, expected tables, and two reader implementations that explains why the chart has those values. Someone can change a fixture, rerun the script, and see whether the conclusion still holds. A screenshot alone cannot provide that path.

The same distinction applies to AI work. Saving a successful prompt may help you repeat its wording, but it does not necessarily preserve the conditions that made it useful. You also need the task, relevant context, acceptance criteria, failed attempts, and the reasoning behind changes. Without those, a future user may copy the prompt into a different situation and mistake resemblance for reliability.

This guide explains how we packaged the four original TrendsWhat labs and how you can adapt the method to an AI-assisted task. It is based on artifacts actually created for this publication. Suggested extensions involving model runs are marked as future work; we did not turn these deterministic labs into an unreported model benchmark.

Original TrendsWhat package guide. Each part answers a different question for the person trying to reuse the method.

Begin with the question, not the winning prompt

Write the smallest question the experiment can answer. Our CSV question concerned preservation of eight known tables under two readers. Our retrieval question concerned whether a selected fragment contained a complete target sentence. Neither experiment asked whether a whole AI product was good.

A narrow question makes reuse easier because readers can compare it with their own need. Someone processing semicolon-delimited files can see that the CSV fixtures do not cover that format. Someone using embeddings can see that the retrieval lab used lexical overlap instead. Those differences tell them what to change or test next.

For an AI writing experiment, a useful question might be whether a revised source pack reduces unsupported factual claims on a defined set of notes. That is a proposed example, not a result we measured. The reusable record should identify the task and metric before preserving the prompt, so that the prompt remains a means rather than the definition of success.

Preserve the input contract

An input contract describes the material the method expects. It can include format, required fields, permitted source types, language, size limits, and the treatment of missing information. A method that assumes a clean table should not be presented as a solution for arbitrary uploaded documents.

In our executable labs, fixtures are included in the script. Expected outputs are visible beside them. This makes the contract unusually inspectable: the reader can see exactly what counts as a plain row, an obfuscated address, or a supported claim. The fixtures are synthetic, so they can be shared without exposing customer records.

For a real project, you may need sanitized examples or a synthetic reconstruction that preserves the relevant behavior. Label that reconstruction clearly. Do not imply it is an untouched production record. The goal is to retain the mechanism needed for learning while removing details that are unnecessary or inappropriate to publish.

Keep the unsuccessful path

A final answer tells you what was accepted. A failed attempt often tells you when the method stops working. In the redaction lab, the email pattern removed conventional addresses but left an obfuscated address, a Korean local-part address, a phone number, and a customer identifier. Those misses define the filter's scope more clearly than a list of its successful matches.

Save the actual failed output, not only a sentence saying it failed. In the retrieval lab, the selected fragment explains the guest-permission miss: it contains relevant terms but stops before the restriction. That artifact lets a reader diagnose the boundary rather than speculate about model reasoning that never occurred in the experiment.

Do not invent a dramatic trial-and-error story to make the package feel personal. If the experiment ran once on a deliberately constructed set, say so. An honest small test can be useful without claiming weeks of struggle, customer impact, or a breakthrough that the evidence does not establish.

Separate raw results from presentation

Our results file contains the per-case data, execution timestamp, and interpreter version. The chart generator reads those recorded values. This creates a traceable relationship between the visual and the experiment. If a result changes, the chart can be regenerated from the new record rather than edited by hand to match a narrative.

Keep that separation in your own work. Raw outputs should remain available in an appropriate location. A report can summarize them, and a diagram can explain the mechanism, but neither should silently replace the original observations. Label calculated metrics and describe how they were computed.

For a model experiment, raw results might include the exact input, model identifier, settings, tool responses, and returned output. Some of those records may require access controls or minimization. Reusability does not mean publishing everything; it means preserving enough evidence, in the right place, to understand and repeat the relevant procedure.

Record decisions as short explanations

A useful decision note has three parts: what changed, why it changed, and what evidence would make you reconsider. For example, the CSV lab supports using a format-aware reader for the tested syntax because the simple splitter failed three fixtures. A different file format would require a configured reader and additional fixtures.

This note is more transferable than “always use this code.” It identifies the failure mechanism and the boundary of the recommendation. Someone with a different environment can preserve the principle while choosing a different library or implementation.

Avoid writing a long diary of every minor edit. Capture decisions that affect meaning, scope, or reproducibility. The point is to help the next person understand why the method looks the way it does, not to make them reconstruct an entire conversation before they can use it.

Copy this experiment package manifest

The manifest can be a short README. It does not require a sophisticated experiment platform. For a small deterministic test, one script and one results file may cover several rows. For a model evaluation, the package may need more structure because inputs and outputs vary across repeated runs.

Make the entry point obvious. A reader should know which command to run and which section of the results to inspect. Our lab script runs all four starter experiments and writes a single JSON report. Each article names the relevant section so readers do not have to guess which result supports its claims.

Turn the package into a reusable procedure

A skill is more than a collection of files. It tells a future user when to apply the method, what inputs to gather, what steps to follow, and when to stop. Start the procedure with its intended use and exclusions. Then put the steps in the order they must happen.

For a source-checking procedure, the sequence might be to extract claims, locate inspected passages, check provenance, judge support, and revise unsupported wording. The stop condition might be an important claim without adequate evidence. That sequence comes from our citation lab's distinction between a real quote and a justified conclusion.

Include a small worked example, but label its facts. A future user should be able to replace the example without accidentally carrying its names, dates, or outcomes into real work. Templates are useful when their structure transfers; they are misleading when their sample facts become invisible defaults.

Test reuse under a changed condition

Repeating the original experiment proves that you can reproduce the original setup. Reuse requires asking what happens when a relevant condition changes. Add a new input format, a different document structure, another language, or an example with missing information. Define the expected outcome before modifying the method to pass it.

For our retrieval lab, a follow-up could add a long paragraph whose exception appears far from its heading. That might reveal a limit of paragraph chunking that the original four-question document does not show. This is a proposed extension, not a result already contained in the recorded data.

Keep the original results and the new configuration distinct. If you overwrite the only evidence file, a later reader may not know which setup produced the published chart. A version label and a short change note can prevent that confusion without creating an elaborate release process.

Make failure handling part of the skill

A reusable procedure should say what to do when a check fails. “Verify the sources” is incomplete if it does not explain how to handle a source that cannot be accessed or a claim that is only partially supported. The useful action may be to narrow the sentence, mark it unresolved, or request missing material.

For a tool-action workflow, failure handling must also distinguish state. The outbox case study shows why a timeout does not prove that an action never happened. A reusable agent procedure should preserve that uncertainty and inspect authoritative state before repeating consequential work.

These instructions are most useful when they name observable conditions. “Stop if the approved draft differs from the version being committed” is clearer than “be careful.” A future user can test the first condition. The second leaves them to invent the actual rule at the moment they need it.

Keep the package maintainable

A reusable method can become stale when its tools, inputs, or assumptions change. Record which parts depend on current software behavior and which express a stable reasoning principle. A command may need updating while the idea of keeping expected outputs remains useful.

Review the package after a new failure, a material dependency change, or a change in the intended task. Do not update dates merely to make the material appear fresh. Explain whether the procedure changed, the evidence was rerun, or only the wording was clarified.

Retire a method when it no longer fits. Reuse is valuable because it saves rediscovery, not because old work must be preserved forever. A concise note explaining why a procedure was replaced can be more useful than a growing stack of compatibility instructions that nobody understands.

The practical result of good packaging

The four TrendsWhat labs can be inspected at several levels. A reader can understand a chart, examine a per-case table, open the raw result, or run the script. Each level supports a different amount of effort while preserving a route back to the underlying observation.

That is the standard to aim for when turning an AI experiment into a reusable skill. Preserve the question and conditions, keep failures visible, record the decisions, and provide a clear entry point. The final prompt is one artifact in that package, not the whole lesson.

The most useful knowledge is often the boundary: when the method applies, what it cannot establish, and what evidence is needed before going further. Save that boundary as carefully as you save the successful output.

Sources, materials, and limits

• Original executable lab package and recorded results.
• CSV, redaction, and retrieval articles for the observed examples discussed here.
• Outbox case study: source-inspected action-state distinctions.
• Original package manifest and procedure guidance. Proposed model evaluations and changed-condition experiments have not been run; no unobserved reuse success is claimed.