Synthience Institute

Make the AI Show You the Source

This guide expands practice #4 of PG-000: 10 Things Every AI User Should Do.

A practitioner guide for catching misrepresented citations before they enter your work

Why this guide exists

Most users have heard that AI systems sometimes make up sources that do not exist. Fabricated citations are real, and they do happen. But they are not the most common citation failure, and they are not the most dangerous one.

The more common failure is subtler and harder to catch: the AI cites a real source, by a real author, in a real venue -- and then attributes a claim to that source that the source never actually made.

This guide is a short procedure for catching that failure before it enters your work.

The core failure mode: misrepresented citation

A misrepresented citation occurs when:

• The cited work exists
• The author, title, year, and venue are correct
• The DOI or URL resolves to the real paper
• And the claim being attributed to the work is not actually supported by the work

Every surface signal that readers use to spot fake citations passes. The only thing that does not pass is the one thing most readers never check: whether the source text actually says what the AI claims it says.

This failure is structurally invisible. The reader cannot detect it from the prose. The AI cannot detect it from its own output. The only way to catch it is to make the source text itself part of the conversation.

When you must use this procedure

Use this procedure whenever:

• The AI attributes a claim to a specific paper, report, person, organization, or news source
• You plan to repeat that attribution in your own work
• The claim would carry weight because of who is being cited
• Being wrong about the citation would damage your credibility, mislead a reader, or affect a real decision

If any of those are true, the citation needs to be verified before it leaves your hands.

The verification procedure

The principle is simple: do not accept that a source supports a claim until the source has been quoted alongside the claim.

Three steps.

Step 1 -- Ask for the exact passage

For every claim the AI attributes to a source, ask the AI to produce the exact passage from the source that supports it.

The two important parts of that instruction are verbatim and do not paraphrase as if quoting. Without those, the AI will frequently produce a fluent reconstruction of what the source could plausibly say, presented as if it were a direct quote. That is the failure mode this step is designed to catch.

Step 2 -- Read what came back

Look at the passage the AI produced and ask three questions:

• Does the passage actually support the claim, or does it only mention the same topic?
• Is the passage a direct quote, or is it a paraphrase being presented as a quote?
• Does the passage support the claim as written, or only a weaker, narrower, or different version of it?

"Mentions the topic" is not the same as "supports the claim." A paper that discusses memory in language models does not automatically support a claim that language models have human-like memory. The link between topic and claim is exactly where misrepresentation hides.

Step 3 -- Verify against the source itself

For any claim that will leave your hands -- public writing, professional communication, a decision affecting other people -- open the source and find the passage yourself.

If the source is publicly accessible (a paper on arXiv or PubMed, a press release, a public report), this usually takes thirty seconds and resolves the question for the cases that matter most.

If the source is not publicly accessible -- paywalled, behind authentication, or only available as an abstract -- you cannot verify the citation. In that case, either remove the citation, soften the claim, or replace the source with one you can read.

What good AI responses look like

An AI that is being honest about a citation will produce one of these:

• A verbatim passage from the source, with a clear pointer, that directly supports the claim
• A statement that it cannot produce the exact passage and an offer to revise the claim
• A statement that the source supports a narrower or different version of the claim, with the narrower claim spelled out

An AI that is failing the verification will produce one of these instead:

• A paraphrase of the source presented in quotation marks
• A general statement that the source "discusses" or "addresses" the topic, without a passage
• A passage that turns out, on inspection, to be about a related but different claim
• Confident assurance that the citation is correct without supplying the evidence requested

If the response falls into the second group, the citation has not been verified. Repeat the request, or remove the citation.

Key rules

• Never accept "the source discusses this" as evidence the source supports a claim
• Never accept a paraphrase presented as a quote
• Never cite a source you cannot read
• Never assume the AI checked the source before answering -- ask

The cost of asking for a quote is one extra turn in a conversation. The cost of repeating a misrepresented citation in public is much higher and lasts much longer.

What this procedure protects

Following this method protects against repeating fabricated quotes, repeating real quotes attached to false claims, citing sources that say the opposite of what is claimed, and citing sources that only tangentially mention the topic. It also reduces the cumulative reputational risk of building work on top of unverified citations.

What this procedure does not do

This method does not verify that the source itself is correct -- a real paper making a real claim can still be wrong. It does not catch every form of citation drift. And it does not relieve the writer of responsibility: the final check is always the human reading the source.

When in doubt

If you are uncertain whether a citation is safe to repeat, ask for the passage. If the passage does not arrive in a form you can verify, do not use the citation. The default is removal, not optimism.