Why Chatbots Make Things Up

Large language models, the technology behind today's AI chatbots, can write essays, summarize reports, and answer questions in fluent, confident prose. Yet they sometimes produce statements that are simply false: invented statistics, nonexistent court cases, or quotations that no one ever said. Researchers call these errors hallucinations.

The problem stems from how the models are built. A language model is trained to predict which words are likely to come next, based on patterns in vast amounts of text. It does not look facts up in a database the way a search engine does. When the patterns point toward a plausible-sounding answer, the model may generate it even if it is untrue.

Developers are attacking the problem from several directions. One approach, known as retrieval-augmented generation, allows the model to consult trusted documents before it answers and to cite its sources. Others train models to express uncertainty or to decline to answer when they do not know, rather than guess.

Such techniques have reduced hallucinations considerably, but they have not eliminated them. For now, experts advise users to treat AI output as a helpful first draft rather than a final authority, and to verify any important claim, especially in fields like law, medicine, and finance, where a confident mistake can be costly.