When AI Hallucinates: Why Language Models Invent Facts – and How RAG Solves the Problem

Why a language model invents facts

A language model does not store a dictionary. During training, it learned from vast amounts of text patterns and formed a so-called parameterized knowledge – similar to a memory that remembers the meaning of what was read but not the exact wording. When you ask a question, the model does not retrieve a stored fact. It predicts the most likely next word, word for word.

This works surprisingly well as long as the model can rely on well-covered knowledge. However, when a gap arises – because the topic is too niche, too new, or simply not encountered during training – the model fills that gap autonomously with a plausibly sounding invention. And it does not realize that it is doing so.

What a hallucination does in practice

A classic example: If you ask a language model for literature recommendations on a niche topic, you receive a flawlessly formatted list – with authors, titles, and even ISBN numbers. Only the books do not exist. Everything is invented but convincingly structured.

In the website context, this quickly becomes tricky: an AI that answers customer queries and freely invents prices, deadlines, or legal information does more harm than good. Follow-up questions like "Are you sure?" do not help, by the way – the model often just confirms itself then. The solution lies not in better questioning but in access to real facts.

The solution: Knowledge instead of guessing

This is where Retrieval Augmented Generation comes into play, or RAG for short. The idea is as simple as it is effective: Instead of letting the model respond from its unreliable memory, you present it with the relevant information directly – and instruct it to answer exclusively based on that.

The process in four steps:

• Receive the question. The user question does not go directly to the model but first to a search.
• Search knowledge base. The system finds the content-relevant passages in your own documents – website texts, manuals, FAQs.
• Provide facts. These passages are loaded into the model's context window along with the question, essentially into its working memory.
• Respond grounded. The system prompt effectively says: "Answer the question solely based on the provided documents." If the answer is missing there, the model should say so openly – instead of inventing.

To help the search understand what belongs together in content, embeddings are used: texts are translated into lists of numbers (vectors) that represent their meaning. Content that is similar lies close to each other in this numerical space – so a vector database can find the appropriate location even if the question uses completely different words than the original text.

What this means for your website

The real insight behind RAG is more fundamental than the technology: AI becomes reliable when it is grounded – when it works with real, verified data instead of probabilities. According to this principle, we are developing our own AI tools for TYPO3.

nt-ai, our AI extension for accessibility, does not guess – it verifies. The accessibility audit operates on a rule-based system according to the WCAG-2.2 criteria; where AI is used, such as for alt texts for images or PDF documents, it relies on the actual content of the image or document. Even the draft of an accessibility statement is generated solely from the measured audit data – not from what a model considers plausible.

nt-lingua, our translation solution, follows the same logic: Translations run through established specialized services, linked to your terminology in a glossary, and cached. This keeps technical terms consistent, and the result reproducible – instead of being newly and possibly differently invented with each request.

And the knowledge base for a RAG system is something most companies already have: their own website. Well-structured, up-to-date content is valuable not just for visitors and search engines – it is the foundation on which an AI can reliably provide information.

Conclusion

Language models are powerful tools, but they are not knowledge databases. They invent where they lack facts – politely, fluently, and unfortunately convincingly. To use AI sensibly, one must ensure that it accesses genuine sources rather than freely formulating. RAG is the technical path to that; the principle behind it – grounding instead of guessing – is the actual core.

Do you want to responsibly integrate AI into your TYPO3 website – accessible, multilingual, and fact-based? Talk to us about nt-ai and nt-lingua.