How conflicting internal documents cause AI hallucinations
Why RAG agents answer wrongly when company documents disagree, what research shows about knowledge conflicts, and how to fix the data before the API call.
The short answer
When a retrieval-augmented (RAG) agent pulls two internal documents that disagree, the model doesn't know which one the business follows. It generates each token from a probability distribution conditioned on everything in its context, so conflicting passages split that probability between answers, and which one wins depends on things that have nothing to do with truth: how many copies of each version were retrieved, where they sit in the prompt, how confidently they're worded, and what the model learned in training. The result is the stale answer, a blend of both that no one ever wrote, or a confident answer citing the wrong source. Research on knowledge conflicts shows models readily adopt wrong retrieved text and rarely flag disagreement on their own. The fix is upstream: deduplicate, resolve or explicitly flag conflicts, and attach dates and status before anything reaches the API. Ingesto does this: it keeps company knowledge synced, deduplicated and conflict-checked, and gives agents one current answer with its sources and any open conflicts, over MCP or an API.
The example we'll use
An engineering team builds an internal support copilot. It indexes Google Drive, the Notion wiki and the #support Slack channel, retrieves the top five chunks for each question, and asks the model to answer from them with citations.
An agent asks: "What's the refund window on annual plans?" Retrieval returns five chunks. One comes from Refund Policy v3 in Drive: 30 days from renewal, full refund. Two come from the Sales FAQ in Notion, which still says 14 days. One is an old PDF export of the same FAQ, also 14 days. The fifth is a Slack thread where the support lead confirms 30 days from renewal.
The copilot answers "14 days", citing the Sales FAQ. Or it answers "14 days, or 30 days for annual renewals", a policy nobody ever wrote. Either way, the model did what it was asked. It answered from its context, and its context disagreed with itself.
What happens inside the model when sources conflict
It helps to be precise about the mechanism, because it tells you where the fix has to go. Retrieved documents don't change a model's weights. The weights are fixed at inference. What the documents do is condition the model: every token it produces is sampled from a probability distribution that depends on all the text in its context window.
When the context contains one clear answer, most of the probability goes to it. When it contains "14 days" three times and "30 days" twice, the distribution over the next tokens after "The refund window is" is split between them. The model has no field that says which document is current, so whatever tips the balance is incidental:
- Repetition. Three stale copies are three pieces of evidence for the stale answer. Duplication works like voting, and old versions usually have more copies because they've been around longer.
- Position. Models use information at the start and end of a long context more reliably than information in the middle, so the order your retriever returns chunks in shapes the answer.
- Wording. A flat statement ("Refunds: 14 days") reads as more authoritative than a thread reply ("I think it's 30 from renewal now?"), whichever is actually current.
- Prior knowledge. What the model learned in training about typical refund windows, or anything else, pulls toward whichever retrieved version agrees with it.
None of these is a bug in the model. It's doing exactly what a language model does: producing the most likely continuation of a context that contains two answers.
What the research shows
Knowledge conflicts have been studied closely since RAG became the default architecture. A few findings matter most for anyone building on internal documents.
- Models are receptive to retrieved evidence, even when it's wrong. Xie et al. (ICLR 2024) found models highly receptive to external evidence that contradicts what they learned in training, as long as it's coherent and convincing, but showing strong confirmation bias when the evidence also contains information that agrees with their training.
- Wrong retrieved content often wins. In ClashEval (Wu, Wu and Zou, NeurIPS 2024), models including GPT-4o adopted incorrect retrieved content, overriding their own correct knowledge, more than 60% of the time. The less confident the model was in its own answer, the more likely it was to adopt the retrieved one.
- Models struggle to report conflicts. WikiContradict (Hou et al., NeurIPS 2024) tested models on pairs of passages that contradict each other and found they struggle to give answers that reflect the conflict, especially when the contradiction is implicit.
- Position matters. Liu et al. ("Lost in the Middle", TACL 2024) found performance is highest when relevant information is at the beginning or end of the context and degrades significantly when it's in the middle.
- Distractors hurt. Shi et al. (ICML 2023) showed model accuracy on math word problems dropped sharply when irrelevant information was added. Retrieved chunks that are near matches but not the answer act the same way.
Xu et al.'s survey (EMNLP 2024) sorts these into three kinds of conflict: between the context and the model's training, within the context itself, and within the model's own knowledge. Conflicting internal documents are the second kind, and it's the one that's entirely within your control.
Four ways conflicting documents turn into wrong answers
| Failure | What the model sees | What it says |
|---|---|---|
| Stale answer wins | Three copies of the old policy, one of the new | "14 days", citing the old FAQ |
| Blended answer | Two versions with different conditions | "14 days, or 30 for annual renewals", a rule nobody wrote |
| Invented reconciliation | Two answers and no way to choose | "It depends on your plan", with no basis in either source |
| Citation mismatch | The answer in one chunk, a similar title in another | The right answer attributed to the wrong document, or the reverse |
The blended answer is the most dangerous, because it looks careful. It mentions both numbers, cites real documents, and describes a policy that exists nowhere.
Why prompts and better retrieval don't fix it
The first thing most teams try is an instruction: "If sources conflict, prefer the most recent one." That needs a reliable date on every chunk, and file dates lie. A stale FAQ that someone reformatted last week has a newer modified date than the current policy. A Slack message has a timestamp, but not an effective date. The model can only apply the rule to dates it's given, and it can't tell a policy change from a typo fix.
The second is better retrieval: reranking, hybrid search, larger k. These improve relevance, and conflicting versions of the same policy are, by definition, all highly relevant. Better retrieval finds all of them faster. Embeddings capture what a passage is about, not whether it's still true, so "refund window, 14 days" and "refund window, 30 days from renewal" sit right next to each other in vector space.
Both are worth doing. Neither addresses the cause, which is that the source material disagrees and nothing in the pipeline knows which version the business follows. Only a person who owns the topic knows that, and they have to be asked before the agent is.
Fix the data before the API layer
The reliable fix is to settle, or at least label, conflicts before any chunk reaches the model. In practice that's six changes to the pipeline.
- Deduplicate by fact, not by file. Collapse every copy of the same policy into one current version, and keep the copies as sources rather than separate chunks. Three copies of the old FAQ stop outvoting the current policy.
- Detect conflicts at ingestion. When a new or changed source states a different rule from an existing one on the same topic, record it as a conflict instead of indexing both silently.
- Send conflicts to an owner. A named person for each topic decides which version is right, once. That's the step no model can do for you, because the answer isn't in the documents.
- Attach status and dates. Every unit of knowledge carries whether it's current, superseded or a temporary exception, its effective date, when it was last verified and who owns it.
- Take superseded content out of retrieval, or mark it clearly as history, so it can't be retrieved as the answer.
- Return open conflicts explicitly. If a conflict hasn't been settled yet, the agent receives it as a labeled field ("the Sales FAQ says 14 days; flagged to the owner") rather than as two unlabeled chunks, and can tell the user the answer is disputed.
Then test for it. Build an evaluation set of real questions where the topic owners have confirmed the answer, and include questions where you know old versions exist. Measure whether the agent gives the current answer, whether its citations support it, and whether it reports an open conflict when there is one.
Where Ingesto fitsIngesto does steps 1 to 6 before your agent calls
Building that pipeline means connectors for every source, change detection, conflict detection, an owner workflow and a serving layer. Ingesto is all of it. It gathers company knowledge from Slack, Microsoft Teams, Gmail, Outlook, Google Drive, Notion, Zoom and Google Meet, keeps it in sync as those sources change, collapses duplicates, and flags conflicts to the owner of each topic.
Your agent then calls one MCP server or API and gets one current answer, with its sources, dates and any open conflicts, the same answer your employees see.
- Deduplicated, current and conflict-checked knowledge, with a source on every line.
- Open conflicts come back as conflicts, so agents report disagreement instead of picking a side.
- Questions with no documented answer come back as gaps, not guesses, and are flagged to the owner.
- No re-indexing job to maintain: the knowledge updates when the sources do.
What clean inputs can and can't do
Clean, deduplicated, conflict-checked knowledge removes the largest cause of wrong answers from internal agents, and the one no amount of prompt engineering reaches. It doesn't make a model infallible. Models can still misread a clear source, and questions with no documented answer still need to be caught as gaps rather than answered with a guess. But the failure changes from "the agent confidently repeated a policy we retired last year" to occasional errors you can trace to a source and fix.
Sources
- Xie et al., Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts (ICLR 2024)
- Wu, Wu and Zou, ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidence (NeurIPS 2024)
- Hou et al., WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia (NeurIPS 2024)
- Liu et al., Lost in the Middle: How Language Models Use Long Contexts (TACL 2024)
- Shi et al., Large Language Models Can Be Easily Distracted by Irrelevant Context (ICML 2023)
- Xu et al., Knowledge Conflicts for LLMs: A Survey (EMNLP 2024)
