Signum
Learn

Why AI hallucinates, and how to reduce it

By Elliot Marsh, Editor · · 4 min read

A language model can write a fluent, confident paragraph that contains a made-up statistic, a court case that never existed or a quote nobody said. These errors are called hallucinations. They are not a bug that will be patched next week; they come from how the models work. But they can be made much rarer, and much easier to catch.

The short version

A hallucination is output that sounds plausible but is false or unsupported by the sources the model was given. It happens because language models are trained to produce likely-sounding text, not to check facts, and because training and testing have often rewarded a confident guess over admitting uncertainty.

Two kinds of hallucination

Researchers often split them in two (see the survey by Huang et al., 2023):

  • Factuality errors: the output contradicts real-world facts, such as a wrong date or invented citation.
  • Faithfulness errors: the output contradicts or goes beyond the material the model was given, such as a summary that adds a claim not in the document.

The second kind matters most in business use, where models summarise, extract or answer from your documents.

Why it happens

The model predicts text, it does not look things up

A model generates one token at a time, each chosen because it is likely given what came before. Facts the model saw many times in training are usually reproduced well. Rare facts, such as a minor paper's authors or a small company's founding year, are stored weakly, if at all. When asked, the model still produces something that fits the pattern of a correct answer.

Training rewards guessing

In a 2025 paper and blog post, OpenAI researchers argued that standard training and evaluation encourage hallucination: when benchmarks score only accuracy, a model that guesses gets more points on average than one that says "I don't know". Their proposed fix is to penalise confident errors more than abstentions, so honest uncertainty is rewarded.

The question pushes it

Leading questions ("Which 2019 study proved X?") presuppose an answer. Models tend to go along with the premise rather than challenge it.

Long and messy inputs

When the relevant fact is buried in a long context or mixed with contradictory material, the model may combine pieces wrongly.

Where the risk is highest

  • Citations, references and quotes. Exact strings are easy to fabricate in a plausible form.
  • Numbers, dates and names of less famous people, companies or places.
  • Niche or recent topics the model saw little of, or that happened after its training data ends.
  • Legal, medical and financial details, where small errors have big consequences.
  • Long agent tasks, where an early invented "fact" is built upon by later steps.

How labs are reducing it

Model makers now measure and report hallucination directly. For example, Google DeepMind said in October that Gemini 4 Carbon cut unsupported claims by about half compared with its predecessor, and that it flags caveats or missing evidence instead of filling gaps. Such figures are self-reported and measured on internal tests, so treat them as direction rather than guarantees. Common techniques include training on examples where the right answer is "I don't know", rewarding citations that check out, and giving models tools such as search so they can look things up.

How you can reduce it

  1. Give the model the source. Paste the document or use retrieval-augmented generation (RAG) so it answers from text, not memory. This is the single most effective step.
  2. Tell it what to do when it does not know. "If the answer is not in the sources, say so" measurably helps.
  3. Ask for citations to the provided material, then spot-check them.
  4. Avoid leading questions. Ask "Is there evidence that X?" rather than "What proves X?".
  5. Break big tasks down. Shorter, focused requests leave less room for drift.
  6. Use tools for exact work. Let the model run code for arithmetic, or search for current facts.
  7. Ask it to check itself, or have a second model or pass verify claims against sources. This catches some errors, not all.

How to catch it

  • Verify anything you will publish, sign or act on, especially quotes, numbers and references. Follow citations to the original.
  • Be suspicious of perfect specificity in areas where the model had no source: exact percentages, page numbers and DOIs.
  • Watch for confident tone without sources. Fluency is not evidence.
  • Keep humans in the loop for high-stakes outputs.

Will hallucinations go away?

Rates are falling as models improve and as they are trained to say when they are unsure. But a system that generates text by predicting likely continuations can always produce a likely-sounding falsehood. The realistic goal is a model that knows when it does not know, combined with workflows that check what matters.

The bottom line

AI hallucinations happen because models produce probable text, not verified facts, and because they have been rewarded for guessing. Ground them in sources, give them permission to say "I don't know", and check anything important. Used that way, the risk becomes manageable.

Sources