Can You Trust the AI's Answer? The Problem of Sources and Citations

In the previous article, we talked about a shift that looks small but runs deep: AI no longer hands you a list of links to choose from. It hands you a single, finished answer. But one question we haven't asked yet remains: can you trust the sources behind that answer?

The answer arrives polished. Its tone is confident. Beneath it sit numbers, sources, and links. Everything about it suggests it has been checked and verified. And that is exactly the problem: the answer looks more trustworthy than it actually is.

Confidence is one thing; accuracy is another. An answer written with confidence can still be wrong. A source listed at the bottom may not say what was attributed to it. The source itself may not even exist.

Why answers look more trustworthy than they are

Why do we trust these answers so easily? Because they carry every outward sign of reliability. The wording is clean. The tone is calm and self-assured. And there are attached sources we can click. All of this creates the impression that someone verified the information before it reached us.

But no one did. The model does not "know" the truth. It assembles words in a way that sounds convincing, then adds sources that look fitting. To the model, a source is not always evidence for a claim; sometimes it is just decoration that completes the shape of the answer.

Researchers at Stanford University audited the answers of four AI search engines. They found that roughly half of all sentences were not backed by any real source. And even when a source was cited, about a quarter of the time it did not actually say what was attributed to it. In other words, the answer looks well supported, while half of it hangs in the air.

The researchers themselves called this a "facade of trustworthiness": an outward appearance that invites trust without anything behind it to justify that trust.

What the numbers say

You might assume this is an old problem, solved as the tools matured. The newest figures say otherwise.

In early 2025, the Tow Center for Digital Journalism at Columbia University ran a test on eight AI search engines. It fed them passages from real articles and asked each to identify the source of every passage. The result: the engines were wrong more than sixty percent of the time. Some were wrong close to ninety percent of the time.

More telling than the error rate was how the engines behaved when they were wrong. They rarely said "I don't know." Instead, they gave a confident, incorrect answer. One engine returned wrong attributions in more than one hundred and thirty cases out of two hundred, and expressed doubt in only a handful of them.

And there is a twist worth noting: the paid versions were not more accurate. They were more confident in their wrong answers than the free versions. You pay more and get a firmer tone, not higher accuracy.

Nor is this an English-only problem. In an international study led by the BBC and the European Broadcasting Union, covering twenty-two news organizations and fourteen languages, roughly half of the answers had a significant issue, and about a third carried an error in the source or in the attribution of the information. The problem, then, crosses languages, and it concerns the Arabic reader as much as anyone else.

Where the citation breaks

The path from your question to your trust in the answer, and the three points where the citation breaks.

When we say a source is "wrong," we are not describing a single kind of error, but three different ones. It helps to know them, because once you start looking, you will spot them yourself.

Type one: a missing source. The answer states a piece of information but attaches no source to back it. The claim may be true or false, but you have no way to check. It hangs there without support.

Type two: a fabricated link. The answer gives you a link that looks entirely real, with a known site name and a convincing address. But when you open it, you find an error page, or a page with nothing to do with the topic. The link never existed; the model assembled it to complete the picture.

Type three: an altered quote. The answer attributes words to a real source, but the source never said them, or said them differently, or said them in the opposite context. In the BBC study, a portion of the quotes attributed to real sources had been altered from the original. The name is right; the words are wrong.

An example from the real world

These errors can look like a technical detail, until you collide with them in a real situation.

In 2023, a lawyer in New York prepared a legal brief for a federal court. He used an AI tool for his research, and it gave him six prior cases supporting his client's position, complete with names, numbers, and quotations. The cases looked entirely real. The trouble was that they did not exist. The model had invented them wholesale.

And here is the sharpest detail in the story: when doubt crept in, the lawyer asked the tool itself whether the cases were real. It assured him they were. That was a second lie on top of the first. The affair ended with a fine on the lawyer and his firm, and with a case now taught as a lesson in the dangers of blind trust in a confident answer.

You might say this was one careless user who failed to check. But even tools built specifically for this field are not immune. Stanford examined professional legal-research tools, some marketed explicitly as "hallucination-free." The result was that these tools fabricated information somewhere between one-seventh and one-third of the time. The promise was bigger than the reality.

The lesson here is simple: the higher the stakes, the greater the need to verify for yourself.

How the engines differ in handling sources

Not every engine treats sources the same way. The table below summarizes, based on the studies cited, how each one presents its sources, where its documented weakness lies, and what that means for you in practice.

Engine	How it shows sources	Documented weakness	What it means for you
ChatGPT (search)	Summarizes and attaches links	Wrong on the attribution of about two-thirds of passages, and rarely admitted doubt	Don't rely on the attribution before opening the link
Google (AI Overviews)	A snippet above the search results	Leans on the search ranking itself, and weakens on lightly covered topics	Verify the original source, not the summary
Perplexity	Shows numbered sources beside the answer	Altered a quote it attributed to a real source	Numbered sources are no guarantee of accuracy
Gemini	Summarizes while citing sources	Had the highest rate of source errors among the tested engines	Double-check before relying on it
Grok	Live search with citation	Had the highest rate of citation errors among the tested engines	Currently the weakest on source accuracy

The rule shared across every row is the same: the presence of a source does not mean it is correct. The numbered source, the blue link, the familiar site name — these are all surface signals. The only proof is to open the source and read it.

How to verify for yourself

You don't need technical skill to verify. You need a simple habit made of four steps:

Open the link before you believe it. Don't settle for its existence. A source you don't open has no value.

Compare the quote against the original. If the answer attributes words to some party, look for those words in their source. Make sure they exist, and that they were said in their correct context.

Prefer the primary source. The official site, the original study, or the party the information belongs to is better than a site that reports it secondhand.

Ask for sources explicitly. When you query the AI, ask it to state its sources and their links. Then open them. Simply asking for sources makes the answer more open to inspection.

These steps take no more than minutes. But they are the difference between building your decision on information, or on the illusion of information.

What this means for you as a business owner

So far we've spoken to you as a reader. But if you are a business owner or a brand, the matter has another side that concerns you directly.

If AI gets attribution wrong half the time, that means it can get the description of your company wrong too. It may attribute to you a service you don't offer, or a wrong price, or outdated information. Worse still, it may mention you confidently and incorrectly. Being mentioned wrongly can be worse than not being mentioned at all.

What to do? Make yourself a source that is easy for AI to cite accurately. That comes down to a few clear things:

Visible, explicit information. Write what you want said about you clearly on your site, in readable text, not hidden in an image or a file. Models read the visible text.

Accurate, up-to-date data. Your prices, services, numbers — keep them current. Old information on your site can come back to you in a wrong answer.

A clear reference about yourself. Make your site the first source for any information about you, in an easy-to-cite form: a question and answer, a table, or a direct definition.

And here is an honest note, because it is a common mistake in the marketing market: the structured data placed behind the scenes in a site's code (schema) is useful for other reasons, but on its own it does not buy you a citation from AI. A recent study showed that adding it did not raise the citation rate. What raises it is visible, trustworthy, inspectable content, and a reputation that makes your site worthy of trust. Don't spend your effort on technical tricks; spend it on genuinely being a source worth citing.

In closing

So we return to the same point we started from. AI is a powerful tool, but it is not infallible. A confident answer is not necessarily a correct one. And a cited source is not necessarily a real one.

The difference between those who benefit from these tools and those they mislead is not in how smart the tool is, but in one simple habit: opening the source before believing it. Trust is built on verification, not on a good appearance.

Sources

• Liu, Zhang & Liang (2023). "Evaluating Verifiability in Generative Search Engines." ACL Findings (EMNLP 2023). https://aclanthology.org/2023.findings-emnlp.467/
• Tow Center for Digital Journalism / Columbia Journalism Review (2025). "AI Search Has a Citation Problem." https://www.cjr.org/tow_center/we-compared-eight-ai-search-engines-theyre-all-bad-at-citing-news.php
• BBC (2025). "AI chatbots unable to accurately summarise news." https://www.bbc.com/news/articles/c0m17d8827ko
• European Broadcasting Union / BBC (2025). International news-accuracy study across 22 organizations and 14 languages. https://www.aljazeera.com/news/2025/10/22/ai-models-misrepresent-news-events-nearly-half-the-time-study-says
• Magesh, Surani, Dahl, Suzgun, Manning & Ho, Stanford University (2025). "Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools." Journal of Empirical Legal Studies. https://onlinelibrary.wiley.com/doi/full/10.1111/jels.12413
• Mata v. Avianca, Inc., S.D.N.Y. (2023). https://en.wikipedia.org/wiki/Mata_v._Avianca,_Inc.
• Ahrefs (2026). Study on structured data (schema) and AI citation rates. https://ahrefs.com/blog/schema-ai-citations