How to Fact-Check and Verify Anything an AI Tells You

In October 2025, Deloitte’s Australian arm had to hand back part of its fee on a A$440,000 government contract to review the IT system behind the country’s welfare compliance framework. The report it delivered cited academic papers that don’t exist, and quoted a real court case, Deanna Amato v Commonwealth, with an invented four-line passage the judgment never contains, even getting the judge’s name wrong in the process. A University of Sydney law researcher, Christopher Rudge, spotted it and told the media. Deloitte confirmed the report had used generative AI and refunded the government A$97,000 (about US $63,000), roughly a fifth of the total contract.

This wasn’t a junior analyst copy-pasting from a chatbot into a term paper. It was one of the world’s largest professional services firms, delivering a paid report to a national government, and nobody who touched it before publication checked whether the citations were real. That’s the part worth sitting with: these were people whose entire job is diligence, and the fabricated content still made it all the way to a published government report.

If that can happen to Deloitte, it can happen to you too, not because you’re careless but because a hallucinated citation is built to look exactly like a real one. Fluency isn’t evidence. This article is the practical companion to our piece on why AI hallucinates in the first place: a repeatable process for checking whether what an AI just told you is true, plus specific playbooks for the situations where getting it wrong costs the most.

Quick answer: Fact-checking an AI answer means separating its checkable claims from its general explanation, weighing how much each claim would cost you if wrong, and then confirming the risky ones against a source that has nothing to do with the AI that gave them to you. The exact method changes by domain: checking a citation isn’t the same as checking a price or a line of code. But the underlying discipline is always the same. Fluency is not proof, and a specific claim is unconfirmed until you’ve checked it somewhere else.

Here’s what you’ll walk away knowing:

• A simple four-step framework for checking any AI answer, from a casual question to something you’re about to publish or act on.
• Why a cited, source-backed answer still needs checking: citing a source and being supported by it are two different things.
• Which tools genuinely help, and which give you false confidence, including reverse image search, citation databases, and browsing-enabled assistants.
• Domain-specific playbooks for code, medical information, legal claims, money and statistics, current events, and quotes.
• Two worked examples applying the framework to realistic AI answers, start to finish.

Why “it sounds right” was never the test

A language model’s job is to produce the most statistically plausible next words, not to verify that those words are true. For most everyday questions the two line up, because the internet is full of correct information stated in familiar patterns. They come apart on anything obscure, recent, or simply invented, and when they do, the model has no built-in signal telling it that’s happened. It keeps writing in the same confident voice regardless. What matters here is the practical consequence: you can’t tell a true AI sentence from a false one by how it sounds. You have to check.

That doesn’t mean distrusting everything an AI tells you. It means building a habit of checking the specific things that are cheap to verify and expensive to get wrong, and knowing which situations call for that extra step. The rest of this guide is that habit, broken into pieces you can use.

Why smart, careful people still skip this step

Knowing you should verify an AI’s claims and actually doing it are two different things, and the gap between them is wider than most people assume. This isn’t a discipline problem. It’s a predictable side effect of how fluent text makes us feel.

Psychologists call it the fluency heuristic: text that’s easy to read and confidently phrased gets processed by our brains as more likely to be true, independent of whether it actually is. An AI answer is, by design, optimized for exactly that kind of fluency: clear sentence structure, no hedging unless prompted, a tone indistinguishable between a claim it’s certain of and one it just invented. Deloitte’s reviewers weren’t unusually careless. They were reading text engineered, structurally, to read as trustworthy.

There’s a behavioral pattern underneath this worth naming directly: even when people are shown a citation, they usually don’t open it. The Reuters Institute’s 2025 research into how people use AI for news found that only about a third of people who see source links in an AI answer click through to check them. The citation looks like accountability, doing real psychological work, even though most readers never cash it in.

None of this means you need to interrogate every sentence an AI produces. It means noticing the specific moment your guard drops: right after an answer that sounds unusually complete, unusually specific, or unusually final. That’s precisely when the fluency heuristic is working hardest on you, and precisely when the four-step framework below is worth running rather than skipping.

The four-step verification framework

Every fact-check, whether it takes ten seconds or ten minutes, follows the same shape. Skipping a step is usually where the mistake gets through.

Step 1: Isolate the claims

Most AI answers are a mix of general explanation and specific, checkable facts sitting side by side, and only the second kind can be individually wrong. Before you evaluate anything, pull the specifics out: names, dates, numbers, quotes, statistics, citations, prices, package names, URLs. A paragraph explaining what inflation is doesn’t need fact-checking the same way a sentence claiming “inflation hit 4.2% in March” does.

This habit alone catches a lot, because it stops you from either distrusting an entire answer over one bad detail or trusting an entire answer because most of it happened to be right.

Step 2: Weigh the risk

Not every claim deserves the same scrutiny. A useful way to sort them:

Claim type	Example	Typical risk
General, widely known	”Paris is the capital of France”	Low: near-zero chance of hallucination
Specific but low-stakes	A rough estimate, a common definition	Low-medium: worth a glance, not a deep dive
Specific and checkable	A statistic, a date, a quote, a citation	High: verify before repeating or acting on it
Specific and consequential	Medical, legal, or financial guidance; anything you’ll publish	Critical: verify and get qualified human confirmation

Hallucination rates also climb in predictable places: anything obscure, anything after the model’s knowledge cutoff, anything in a niche field with easily-confused terminology. Treat those as automatically higher-risk, regardless of how the claim is worded.

Step 3: Verify independently

This is the step people skip, usually by asking the same AI “are you sure?” That doesn’t help. A model that generated a wrong answer with full confidence will often defend it with the same confidence, a pattern researchers call sycophancy when it instead caves and reverses a correct answer just because you pushed back. Neither response is evidence of anything.

Independent verification means leaving the AI chat window and checking the claim somewhere that has no relationship to it: a primary source, an official database, a different search, a person who actually knows. Librarians call this lateral reading: instead of scrutinizing the page in front of you, you open a new tab and see what independent sources say. The same principle applies to AI output. If a claim matters, it needs a source that didn’t come from the AI that made the claim.

Step 4: Calibrate your trust

Once you’ve verified, or tried to and failed, decide what that means for how you use the answer:

• Confirmed independently: use it, and cite the primary source rather than the AI if you’re passing it on.
• Plausible but unconfirmed: flag it as unverified, or ask someone who’d actually know before you repeat it.
• Couldn’t confirm it exists: treat it as false until proven otherwise. This is the right default for a citation, a case, or a package name you can’t find independently. The burden of proof sits with the claim, not with your search.

Six techniques worth building into habit

A few concrete moves make most of the framework above nearly automatic.

Ask for sources, then check the sources, not the answer. “What’s your source for that?” is a weak question, because a model that fabricated the original claim will often fabricate a plausible-looking source to back it up, complete with a real-sounding author and publication. The useful move is to take whatever source it names and go verify that source exists independently, the same way you’d verify the original claim. If the source doesn’t turn up anywhere outside the AI conversation, that’s your answer.

Reverse-search exact quotes. If an AI attributes a quote to someone, search the exact wording in quotation marks rather than paraphrasing it. A real quote usually surfaces the original context (an interview, an article, a transcript) within seconds. A fabricated one often returns nothing at all, or returns only other AI-generated pages repeating the same invented line back at you, which can feel like confirmation while really being an echo of the same mistake.

Search to disconfirm, not just to confirm. It’s tempting to search a claim in a way that’s likely to agree with it: searching “is X true” tends to surface pages arguing yes, because that’s what gets written. Try the inverse too. Search the claim alongside a word like “false,” “myth,” or “debunked.” If a widely believed but wrong version of the claim exists, this is usually how you’ll find the correction.

Run a calibration test on new tools. Before trusting an unfamiliar AI product with a question that matters, ask it something you already know the answer to first. How confidently and accurately it handles a question you can check tells you a great deal about how much to trust it on one you can’t, and it takes thirty seconds to do once, before you need the tool for something real.

Use a second model as a sanity check, not a verdict. Comparing answers across two AI models can surface disagreement worth investigating. It can’t confirm agreement is correct, because models are often trained on overlapping data and can share the exact same blind spot. Treat a matching answer from two models as mildly reassuring, never as independent proof.

Ask the AI to audit its own answer’s claims, then verify what it flags. A prompt like the one below won’t catch everything, since a model can be just as confidently wrong about its own confidence as it was about the original claim. But it’s a fast way to generate your checklist for step 1, especially on a long or dense answer where the specific claims are easy to lose track of.

List every specific factual claim in your previous answer as
a separate bullet point: names, dates, numbers, quotes, and
citations. For each one, note whether it's something you're
highly confident about or something I should verify independently.

Why “it cited a source” isn’t the same as “it’s true”

Tools that browse the web and cite sources as they answer, including Perplexity, ChatGPT with search enabled, Claude’s web search, and Google’s AI Overviews, are a genuine improvement over a model answering purely from memory. Grounding an answer in a real, retrieved document measurably lowers the odds of a fabricated claim. It doesn’t make the answer reliable by default.

Columbia Journalism Review’s Tow Center for Digital Journalism tested this directly: 1,600 real queries run through eight AI search tools that cite sources as part of their answers. The tools gave an incorrect answer more than 60% of the time overall, despite the live citations attached to nearly every response. Perplexity performed best at 37% incorrect. Grok-3 performed worst at 94% incorrect. Several tools routinely cited the wrong publication as the source of a story, or linked to syndicated or aggregator versions instead of the original reporting. Premium, paid tiers of some products actually answered incorrectly more often than the free versions, while sounding just as confident.

Stanford’s RegLab found the same pattern in a completely different domain. Testing leading AI legal research tools built specifically to ground their answers in real case law, they found Lexis+ AI, Westlaw AI-Assisted Research, and Ask Practical Law AI each hallucinating somewhere between 17% and 33% of the time. That’s a real improvement over a general-purpose chatbot answering the same legal queries with no legal-specific grounding at all, and still far from zero. The researchers coined a specific term for the most common failure: misgrounding, where a tool cites a real, existing source that simply doesn’t say what the AI claims it says. The citation checks out. The claim attached to it doesn’t.

The lesson isn’t to distrust source-citing tools. They’re a meaningfully better starting point than one that doesn’t cite anything. It’s that “it cited a source” answers the question “did it make this up out of nothing,” not the question “is this actually correct.” Those are different questions, and only opening the cited source and reading it yourself answers the second one.

The tools that actually help you check

None of these tools replace the framework above; they make step 3, verifying independently, faster. Which one is worth reaching for depends on what kind of claim you’re checking:

You’re checking	Reach for
A general claim already being discussed online	A search-grounded assistant or an independent fact-checking site
An academic citation or study	A DOI lookup, Google Scholar, or PubMed
A software package an AI recommended	The real package registry (PyPI, npm) directly
An image or video’s authenticity	Reverse image search plus provenance metadata
A government statistic or company financial figure	The primary source itself (the agency or filing), not a summary of it

Browsing and search-grounded assistants

Perplexity, ChatGPT with search enabled (rolled out to all users in February 2025), and Claude’s web search tool all attach live citations to their answers, which makes them checkable in a way a purely memory-based answer isn’t. Anthropic’s own documentation notes that Claude’s web search results always include a citation with a link and the specific text it drew on, rather than a bare, unlinked claim. Google’s own AI Overviews carries a built-in caution worth taking literally: the feature’s help documentation tells users its “AI responses may include mistakes” and to “always check important info in more than one place.” Treat that disclaimer as accurate, not boilerplate.

The practical benefit of a citation-producing tool isn’t that its answer is guaranteed correct. The Columbia Journalism Review numbers above rule that out. It’s that the citation gives you something concrete to check in step 3 of the framework. A tool that states a fact with nothing attached to it gives you nowhere to start.

Independent fact-checking sites

For claims that are already circulating publicly, such as viral statistics, news events, or quotes attributed to public figures, a human-run fact-checking site is often faster and more reliable than trying to verify it yourself from scratch. Snopes, PolitiFact, Reuters Fact Check, and AFP Fact Check all maintain searchable archives of claims they’ve already investigated, and a quick search of the claim’s key phrase on one of these sites will often tell you in seconds whether it’s a known fabrication, a distorted half-truth, or accurate.

The advantage these sites have over an AI answer isn’t just accuracy. It’s a documented trail. A fact-checking write-up shows its work: what was claimed, what was found, and where. That’s the same standard worth holding an AI’s answer to before you repeat it.

Citation and paper verification

For an academic citation, resolve the DOI directly at doi.org, or search the exact title on Google Scholar or PubMed. If the AI names a specific journal, volume, and page number, check that the article at that citation actually says what’s being attributed to it. A 2023 study in Nature Scientific Reports found that 55% of citations ChatGPT (GPT-3.5) generated for academic claims were fabricated outright, versus 18% for GPT-4, a large gap between models but a nonzero rate for both.

Image and video verification

For AI-generated or AI-suspected images, a reverse image search through Google Images or TinEye, both of which index a continuously growing library of billions of images, shows you where else an image has appeared online, which is often enough to confirm or debunk it on its own. Some platforms also embed content-provenance data: Google’s SynthID watermarks AI-generated images invisibly, and the C2PA standard attaches tamper-evident metadata about an image’s origin. Neither is universal yet, so their absence doesn’t prove an image is real. Their presence, though, is a genuine signal.

A domain-by-domain playbook

The same four-step framework applies everywhere, but what “verify independently” actually means changes a lot by field. These are the categories where getting it wrong costs the most.

Coding

Run it. A snippet from an AI coding assistant either compiles and does what you asked or it doesn’t, which makes this one of the easier domains to verify. The trap isn’t the code itself; it’s what the code imports. Researchers who generated over 2.2 million AI code samples found that close to 1 in 5 referenced a software package that doesn’t exist, a problem serious enough that attackers now register those exact fake package names and load them with malware, waiting for a coding assistant to recommend one. Check any unfamiliar package name on the real registry (PyPI, npm) before you install it. We cover this “slopsquatting” risk in more depth in our hallucinations piece.

“Runs without errors” and “does the right thing” also aren’t the same test. Code can execute cleanly while calling a deprecated method, mishandling an edge case, or using an API parameter that doesn’t exist in the version you’re actually running. For anything beyond a throwaway script, check unfamiliar function calls against the library’s own current documentation, not just against whether the code happened to run once.

Medical and health

Use AI to understand a condition or prepare questions for an appointment, not to replace one. Cross-check anything specific, such as a dosage, a drug interaction, or a symptom pattern, against an established medical reference like MedlinePlus, Mayo Clinic, or a PubMed-indexed study, and bring anything that actually matters to a licensed clinician before acting on it. No AI chatbot is currently cleared by regulators as a medical device for diagnosis or treatment decisions, so treat its health information the same way you’d treat a knowledgeable friend’s guess: a reasonable starting point, never a final answer.

Be especially careful with anything involving a specific number: a dosage, a weight-based calculation, a “safe” interaction between two medications. These are exactly the kind of narrow, specific claims most likely to be wrong, and the domain where being wrong carries the highest real-world cost on this entire list.

Legal

The American Bar Association’s Formal Opinion 512, issued in July 2024, states plainly that a lawyer’s uncritical reliance on generative AI output is “almost certainly malpractice.” That standard is a reasonable one for anyone, not just lawyers. Every case citation, statute reference, or legal claim an AI gives you needs to be checked against an actual court record or statute database, such as Google Scholar’s case law search, Justia, or your jurisdiction’s official court records, before you rely on it or repeat it to anyone. A database tracking court decisions worldwide involving hallucinated AI-generated legal citations has logged well over a thousand cases since 2023, including the widely reported Mata v. Avianca, where a New York lawyer was sanctioned $5,000 in 2023 for filing a brief built on six fabricated court cases ChatGPT had generated.

Statutes and regulations also change, and a model’s training data reflects the law as it stood at some point in the past, not necessarily today. When an AI cites a specific statute or regulation, check that you’re looking at the currently in-force version on an official government source, not just that the citation format looks plausible.

Money, statistics, and data

Go to the primary source, not the AI’s paraphrase of it. For US economic data, that’s the Bureau of Labor Statistics, the Census Bureau, or the Federal Reserve’s own FRED database. For a public company’s financials, that’s its SEC filings or investor relations page, not a summary of them. AI is genuinely useful for explaining what a statistic means or helping you find where to look. It’s the wrong tool for being the statistic’s source of record.

Check the “as of” date on whatever you land on, too, not just whether the number itself looks right. A correct figure from eighteen months ago, stated with the same confidence as a current one, is a different kind of error than an invented number, but it costs you the same way if you act on it.

Current events and anything recent

Every AI model has a knowledge cutoff: a date after which it simply has no training data, and therefore no reliable memory of what happened. For anything that occurred after that point, a model working purely from memory isn’t hallucinating exactly, but it also isn’t informed, and it may not volunteer that distinction unprompted. Ask directly what the model’s knowledge cutoff is, and treat anything after it as something only a browsing-enabled, source-citing answer can speak to. Even then, per the citation-accuracy numbers above, check the source.

Quotes and citations

Search the exact wording of any quote in quotation marks before repeating it. This single habit catches most fabricated attributions, because a real quote almost always surfaces its original context, and an invented one usually doesn’t surface anything beyond other AI-generated pages that made the same thing up.

Watch for the quieter version of this problem too: a real person said something in the neighborhood of what’s quoted, but the AI has smoothed or tightened the wording into something crisper than what they actually said, then presented it inside quotation marks as if it were verbatim. That’s not the same failure as a fully invented quote, but it’s just as much a misquote, and the only way to catch it is to find the original source and compare the exact words.

Two worked examples, step by step

Abstract advice is easy to nod along to and hard to apply. Here’s the same four-step framework run against two realistic, different kinds of AI answers.

Example one: a number that changes over time. Suppose you ask an AI assistant what interest rate the US Federal Reserve currently has set, and it gives you a confident, specific answer with a rate and a meeting date attached.

• Isolate the claims. There are two checkable specifics here: the rate itself, and the date of the meeting that set it. Everything else in the answer, a general explanation of what the federal funds rate is, counts as low-risk background.
• Weigh the risk. A specific current interest rate is exactly the kind of claim that’s easy to state confidently and easy to get wrong, especially since the rate changes on a schedule that may fall after the model’s knowledge cutoff. This lands squarely in the “specific and checkable” row of the risk table above.
• Verify independently. Go to the source with actual authority over this number: the Federal Reserve’s own website or its FRED economic database, not a search result summarizing what the AI said, and not the AI itself confirming its own answer.
• Calibrate your trust. If the primary source matches, you now have a confirmed number. Cite the Fed, not the chatbot, if you’re passing it along. If it doesn’t match, or if the AI’s meeting date is after its own knowledge cutoff, treat the AI’s number as stale and use the primary source’s figure instead.

Example two: a study cited to support a claim. Suppose you ask an AI to explain a health or productivity trend, and it backs up its answer with “a 2023 study published in [Journal Name] found that…” followed by a specific statistic.

• Isolate the claims. Two things need checking separately: whether the study exists at all, and whether it actually found the specific statistic being attributed to it. These can fail independently: a real study can still be misquoted.
• Weigh the risk. High. A specific citation attached to a specific number is exactly the pattern most likely to be fabricated wholesale, per the Nature Scientific Reports findings cited above, and exactly the kind of claim people repeat confidently once it’s been “sourced.”
• Verify independently. Search the journal name plus a distinctive phrase from the claim, or resolve it through Google Scholar or PubMed directly. If a DOI is given, resolve it at doi.org rather than trusting that a plausible-looking DOI format means the link is real.
• Calibrate your trust. If the study exists and the abstract supports the statistic, you’re clear to use it. Cite the study directly. If the study doesn’t turn up, or exists but the actual finding doesn’t match what was claimed, that’s misgrounding: discard the claim, not just the citation.

Both examples take longer to explain than to do. Once the framework is a habit, most checks like these take under a minute.

Can AI fact-check AI?

It’s tempting to close the loop by asking a second AI to check the first one’s work, and dedicated AI fact-checking tools are a real, growing category. The honest answer is that they help, unevenly, and shouldn’t replace checking the underlying source yourself.

A December 2024 study published in PNAS tested exactly this: researchers had ChatGPT fact-check a set of news headlines and compared its judgments to human fact-checkers. ChatGPT correctly flagged 90% of false headlines, a strong result. But it correctly identified only 15% of true headlines as true, frequently flagging accurate information as false. That mattered beyond the raw numbers: when the AI incorrectly labeled a true headline as false, people’s belief in that true headline measurably dropped. Human fact-checkers outperformed the AI across the board.

The practical takeaway is that an AI fact-checker is a reasonable first pass for catching obviously false claims, and a poor final word on anything it labels true, including a claim about its own previous answer. Use it as one more input, not the verdict.

Common misconceptions

“If it gives a source, that’s proof.” A citation shows the AI found or generated something that looks like a source. It doesn’t show that source says what’s being claimed. Stanford’s “misgrounding” research is the clearest evidence this gap is common even in tools built specifically to avoid it.

“Two AI models agreeing means it’s probably true.” Agreement between models trained on overlapping data isn’t independent confirmation. It can just as easily mean two models sharing the same blind spot. Treat it as a mild signal, not a verdict.

“This only matters for big, obviously important claims.” The quiet, single wrong detail buried inside an otherwise accurate paragraph is usually the most damaging kind, precisely because nothing about the surrounding accurate text flags it. Apply the framework to specific claims regardless of how minor the surrounding context seems.

“A more expensive or ‘premium’ AI tool is automatically more accurate.” The Columbia Journalism Review study found some paid tiers answering incorrectly more often than free versions of the same product, while sounding no less confident. Check a tool’s actual track record rather than assuming price signals accuracy; Vectara’s independently maintained hallucination leaderboard is a reasonable place to look.

“Verifying an AI answer takes too much time to be worth it.” Most of the verification described here (a reverse-quote search, a DOI lookup, a package-name check) takes under a minute once it’s habitual. The alternative, per Deloitte’s A$97,000 lesson, can cost considerably more.

“I’d notice if an AI got something this wrong.” The fluency heuristic described earlier works precisely because it doesn’t feel like anything from the inside. A fabricated claim reads exactly as smoothly as a true one. Deloitte’s own reviewers didn’t notice either, and catching a hallucination isn’t a matter of paying closer attention to how an answer sounds; it’s a matter of checking the specific claims, every time the stakes call for it.

The bottom line

An AI answer earns its fluency for free. It hasn’t earned your trust yet. That part still takes an extra step, and which step depends on what’s actually at stake in the specific claim in front of you. The four-part habit is simple enough to become automatic: isolate what’s actually checkable, weigh how much it would cost you to be wrong, verify it against a source that has nothing to do with the AI that told you, and calibrate how you use the answer based on what you find.

None of this is a reason to stop using AI tools. It’s a reason to stop treating confidence and fluency as a stand-in for accuracy, the same discipline you’d already apply to an unverified tip from a stranger, just aimed at a tool that happens to sound like an expert every single time it answers.

Frequently asked questions

If an AI shows its sources, do I still need to check them?

Yes. A cited answer is safer than an uncited one, but citing a source and being supported by it aren't the same thing. Stanford researchers call the gap 'misgrounding': a real source that doesn't actually say what the AI claims it says. Open the source and confirm it directly before you rely on the claim.

Is it safe to use AI for medical or legal advice?

Use it to understand a topic, not to replace a professional. The American Bar Association has warned that uncritical reliance on AI output is 'almost certainly malpractice' for lawyers, and the same caution applies to your own health and legal decisions. Treat AI as a starting point for questions to bring to a real doctor or lawyer, not a final answer.

Does asking a second AI model confirm whether the first one was right?

Only partly. Different models are often trained on overlapping data and can share the same blind spots, so agreement between two AI models isn't independent confirmation. It's a useful sanity check, but treat a genuinely independent, non-AI source, such as a primary document, an official database, or a person who knows, as the real verification step.

How do I check if an AI-cited study, court case, or article actually exists?

Search the exact title or a distinctive quoted phrase in quotation marks, and look the source up directly on its home turf: a DOI at doi.org, a paper on Google Scholar or PubMed, a case on a court database. If you can't find it independently within a couple of minutes, treat it as unconfirmed rather than assuming your search just missed it.

Do AI fact-checking tools actually work?

They help, but unevenly. A December 2024 study in PNAS found ChatGPT correctly flagged 90% of false headlines but only 15% of true ones, and mislabeling a true headline as false measurably reduced people's belief in it. AI fact-checkers are a useful first pass, not a replacement for checking a claim against its original source yourself.

How can I tell if an AI-generated image or video is real?

Don't trust how it looks. Modern AI images can look completely convincing. Run it through a reverse image search (Google Images, TinEye) to see where else it appears online, and check for content-provenance metadata like C2PA or Google's SynthID watermark where the platform supports it. Treat an image with no traceable origin as unverified.

• #AI Fact-Checking
• #AI Hallucinations
• #AI Accuracy
• #Large Language Models
• #AI for Beginners
• #AI Literacy