A citation is not evidence: testing an AI research checklist

A genuine quotation can support a false sentence

The most instructive result in this lab was not an invented source. It was a real source attached to the wrong claim. Our synthetic source says that a pilot included twelve tickets. A test claim says it included 120. The quotation is copied correctly, the source identifier exists, and a simple provenance check accepts the record. The claim is still wrong.

That distinction matters whenever AI helps write a research brief. A paragraph with references looks more trustworthy than one without them. A working link or matching quotation can give a reviewer a sense that verification has already happened. In reality, those checks answer only part of the question. They establish where a piece of text came from, not whether the text justifies the sentence beside it.

We created eight claim records against three original, fictional source statements and ran a local checker. Five records passed the source-and-quote check. Only three were supported according to the explicit editorial labels in the fixture. Two unsupported claims therefore passed the mechanical gate. This experiment measures that checker on these records; it does not estimate a language model's hallucination rate.

Original TrendsWhat visual based on the recorded experiment. “Supported” is a separately authored editorial label, not an inference produced by the checker.

Why the source ledger needs two decisions

A source ledger is a table connecting claims to evidence. It can be extremely useful because it exposes missing references. But its design determines what reviewers notice. If it contains only “claim” and “URL,” then a filled row may look complete even when the linked page is irrelevant. If it adds a quotation, a second failure remains possible: the quotation is real, but the claim overstates it.

For this lab, we separated provenance from support. Provenance asks whether the claimed source exists and contains the quoted passage. Support asks whether the passage warrants the actual assertion. In a real research project, support also depends on scope, date, population, methods, and conflicting evidence. We kept those dimensions small enough to inspect directly.

This is a useful skill for nontechnical readers as well. Before forwarding an AI-written brief, choose one important sentence and read its source. Ask what would have to be true for the sentence to follow. If the source reports an observation while the draft asserts a cause, the reference may be genuine and the reasoning still incomplete.

Our deliberately small source collection

The first synthetic source states that the pilot included twelve tickets. The second states that median draft time was four minutes. The third states that customer satisfaction was not measured. The pilot itself is fictional. These are controlled test statements, not hidden results from a customer study or from TrendsWhat operations.

We wrote eight records around those sources. Three preserve a supported count, time, or limitation. The others introduce a nonexistent source, an invented quotation, a changed count, unsupported causality, or an empty quotation. The expected support label is specified for each record before the checker evaluates it.

The checker accepts a record when its source identifier exists, its quotation contains non-whitespace text, and that quotation appears in the corresponding source. It does not ask a model to grade itself. It does not use semantic similarity. Those choices make the false accepts easy to explain: the program never evaluates the relationship between the claim and the quotation.

The observed outcomes

The two false accepts illustrate different reasoning problems. The wrong-count case is a direct mismatch that a narrowly designed numeric check might catch. The causality case is a larger leap. A source reporting draft duration does not by itself establish an improvement, a baseline, or a causal mechanism.

Do not turn five accepted records into a statement that the checker is “mostly reliable.” Acceptance rate measures how often a gate opens, not how often it should open. In this fixture, the important observation is that two of its five approvals are inappropriate under the declared support labels. A gate can process every row consistently while enforcing an insufficient rule.

Reproduce the test

Download the lab script and run python3 run-labs.py --output results.json. Read the evidence section in the output. It includes the full source collection, every claim, source identifier, quotation, provenance result, and expected support label. The original execution record preserves the run timestamp and interpreter version.

To understand the failure, edit the wrong-count claim while leaving its quotation unchanged. The provenance result remains the same because the checker does not inspect the claim. This is a property of the code, not an unpredictable model response. It is a quick way to see exactly what your validation step does and does not promise.

For background, the W3C provenance overview describes recording the origins and production of information. We use that distinction as context, not as a claim that this small table implements the full W3C model. Our proposed workflow adds an explicit editorial support decision beside origin tracking.

A better review sequence

Begin with claims that matter to the reader's decision. A decorative sentence about how quickly technology changes does not deserve the same review priority as a claim that a workflow reduced errors by half. Break compound statements into smaller claims before checking them. A single citation can support one clause while leaving another unsupported.

For each claim, record the source title, direct location, access date, and relevant passage. Then assign a support status: supported, partially supported, contradicted, or unresolved. Add a short explanation in ordinary language. “Reports duration but has no comparison group” is more useful than a bare confidence score because another reviewer can inspect the same reasoning.

When support is partial, narrow the wording rather than burying the limitation in a footnote. In our fictional pilot, “median draft time was four minutes” is defensible. “The tool made the team faster” is not. The first sentence tells readers what was measured. The second adds a conclusion that the available material cannot carry.

Copy this claim ledger

You can use this table in a document or spreadsheet. The key is to retain the exact claim under review. If the wording changes after review, the old approval may no longer apply. A source supporting “some” does not automatically support a later edit to “most.” Versioning the claim text prevents that subtle drift.

How AI can help without becoming the final witness

AI can extract candidate claims, suggest passages to inspect, and identify wording that appears stronger than the evidence. Those are useful assistance tasks. Treat the results as a review queue. Keep the underlying documents available so a person can resolve disputed support rather than asking the assistant to repeat its assurance.

A practical prompt is: “List factual claims in this draft. For each, provide the exact sentence, the supplied source passage, and any gap between them. Use unresolved when the supplied sources do not establish the claim.” This is a reusable instruction, not a prompt we benchmarked in this lab. Its performance depends on the model, inputs, and review process you actually use.

Avoid asking for a confidence percentage unless you have a meaningful way to evaluate that percentage. A model's self-reported confidence can look precise while adding little evidence. A quoted passage, a clear support explanation, and an unresolved label create a more inspectable handoff than an unexplained number.

What the experiment leaves unresolved

The fixtures are short and intentionally obvious. Real documents contain qualifications across paragraphs, tables with footnotes, conflicting revisions, and evidence that requires domain knowledge. Our literal quote check can also reject a legitimate paraphrase, since paraphrases need not appear word for word. That is acceptable for this gate because it explicitly requires quotations, but it would be a design error if we described it as a general truth detector.

The support labels are authored judgments for fictional statements. We did not measure agreement between independent reviewers or evaluate a model-based entailment system. A larger study would need a documented labeling procedure, difficult examples, and disagreement handling. It would also need to preserve the distinction between a source making a claim and that claim being true in the world.

That last boundary is easy to miss. Even a faithful summary of a real source can inherit the source's error. Important decisions may require multiple independent sources, primary records, or expert interpretation. The ledger gives those decisions somewhere to live; it does not eliminate the need to make them.

The habit to carry into your next brief

Treat a citation as a route to evidence. Follow that route, read the relevant passage, and compare it with the exact sentence you plan to publish. Keep provenance and support as separate columns so that neither can masquerade as the other.

Our tiny checker did its assigned job consistently and still allowed two unsupported claims through. That is the value of the experiment: it reveals an incomplete definition of “verified.” Improving AI research often begins by repairing that definition, before changing tools or adding a longer instruction to be careful.

Sources, materials, and limits

• W3C PROV overview: background on information provenance.
• Original executable fixtures and recorded results.
• Eight synthetic claim records; three synthetic sources; literal provenance checker; manually specified support labels; no language-model benchmark or real pilot performance claim.