# Illusions Unraveled: The Magic and Madness of Hallucinations in LLMs Published on Dec 6, 2023 **Research done by Guruprasad Raghavan, Jon Gill, Victor I. Afolabi** **TL;DR** We have benchmarked several popular open source LLMs (including the latest _Llama-v2–7b-chat_) to estimate both, the frequency and degree of hallucinations. Overall, we find that on average, popular, **_open-source models hallucinate close to 55% of the time on a context-aware Q&A task_**, when tested without any tuning. We also make our [**_evaluation pipeline (code-base) open-source_**](https://github.com/YurtsAI/llm-hallucination-eval), so anyone can do this! In a future post, we will explain our strategy on how to reduce hallucinations and propose model intervention methods that achieve the same. ## Hallucinations are Inherent to LLMs Hallucinations are observed across a wide array of generative AI systems, > Hallucinations are loosely defined as confident, factually incorrect outputs produced by an AI system not justified by its training data. For a deeper dive on understanding the mathematical basis for why hallucinations are inherent to all generative AI systems, and specifically to Large language models, check out the section titled **“Mathematically speaking, why do hallucinations occur?”** ## Metrics to Measure LLM Hallucination One of the biggest challenges in attempting to resolve the issue of LLM hallucination is the abstract, subjective nature of the problem. Answers to questions such as “ **How do we know when an LLM hallucinates?”** or “ **What is considered ground-truth?”** are critical to solving the problem; however, these questions do not necessarily have universally accepted answers. To address these two specific types of hallucinations, we introduce quantitative metrics to measure two types of hallucinations, Type 1 and Type 2 hallucinations, whose definitions are as follows: **Type 1 (T1) hallucination: LLM creates new entities** T1 hallucination refers to the scenario when an LLM is fed a prompt that contains a query and context (knowledge from the enterprise document) relevant to the query, and its response contains entities (names of persons, places, organizations, dates) that are not present in the context or query. **Example for Type 1 hallucination** **Article:** [https://techcrunch.com/2023/06/13/striveworks-snaps-up-first-funding-of-33m-to-build-tools-for-machine-learning-operations/](https://techcrunch.com/2023/06/13/striveworks-snaps-up-first-funding-of-33m-to-build-tools-for-machine-learning-operations/) **Type 1 Hallucination:** Apart from the entity `Chariot`, none of the other entities are present in the original article or in the question. Although the response might seem correct, it isn’t factually precise, given that all information about Chariot is ONLY present in the article. **Type 2 hallucination: LLM falsifies relation between entities** T2 hallucination refers to the scenario when an LLM is fed a prompt that contains a query and context (from the enterprise document) relevant to the query, and its response contains the right entities (previously mentioned in the prompt), but the relationship between the entities is not synchronized with the knowledge mentioned in the prompt. In this blog, we will focus on Type 1 hallucination and will share our updates on Type 2 hallucination in a future edition. ## Benchmarking Type 1 Hallucinations of Instruction-Tuned Open-Source Models ### **Open-source evaluation pipeline** At Legion, we have designed a pipeline to reliably — and quantitatively — measure the relative degree of hallucination for any LLM generation. Effectively, the pipeline accepts an instruction-tuned model and a dataset, feeds the dataset through the model, and finally, calculates the model’s T1 hallucination score. This approach enables us to: - Rapidly filter open-source models (available on HuggingFace) that have desirable features (like, very low hallucination, low toxicity, etc). - Establish quantitative baselines for measuring progress on reducing LLM hallucination across open-source models Here’s a schematic of the pipeline that we built for rapid evaluation of open-source models on instruction-based tasks, specifically question-answer with relevant background context. Our pipeline has been open-sourced for the community and is available [here.](https://github.com/YurtsAI/llm-hallucination-eval) ### **Synthetic context-aware question answering dataset generation** In our pipeline described above, we generated a dataset using TechCrunch (TC) articles published over the last three years for the context-aware question-answer task. Our evaluation dataset comprises of 300 randomly sampled TC articles. The workflow for generating the dataset is described in detail in the section titled **“Q _&_ A dataset curation”** ## Evaluation of Instruction-Tuned Open-Source Models We evaluate several open-source models to assess both, their frequency and degree of hallucination (specifically Type 1 Hallucination) on the context-aware question-answer task. | Model Names | Parameter sizes | GPU Memory | Context length | License | | --- | --- | --- | --- | --- | | [Llama-v2-7b-chat](https://huggingface.co/meta-llama/Llama-2-7b-hf) | 7B | ~15GB | 4096 | Commercial | | [Falcon-7b-instruct](https://huggingface.co/tiiuae/falcon-7b-instruct) | 7B | ~15GB | 2048 | Apache 2.0 | | [Falcon-40b-instruct](https://huggingface.co/tiiuae/falcon-40b-instruct) | 40B | ~90GB | 2048 | Apache 2.0 | | [Open-Assistant-12B](https://huggingface.co/OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5) | 12B | ~48GB | 2048 | Apache 2.0 | | [Xgen-7b-instruct](https://huggingface.co/Salesforce/xgen-7b-8k-inst) | 7B | ~28GB | 8192 | Non-commercial (But, the base models are Apache 2.0) | | [Dolly-v2-12b](https://huggingface.co/databricks/dolly-v2-12b) | 12B | ~48GB | 2048 | MIT | > Result — 1: About 55% of instruct-tuned open-source model responses hallucinate entities in their responses. > Result — 2: On average, instruct-tuned open-source model responses have about ‘3’ made-up entities in each answer. > Result — 3: Of the models we’ve evaluated on the Context-aware Q&A task, ‘Llama-v2–7b-chat’ model is the least susceptible to Type 1 hallucination > Result — 4: The subjective performance of the instruct-tuned open source model responses on the Context-aware Q&A task is comparable. ## How do we reduce Type 1 Hallucinations in open-source models? We believe a robust strategy to reduce Type 1 hallucinations in instruct-tuned open-source models is to navigate the LLM parameter space in such a way that it alters the functional map from input text strings to the simplex output vector to ensure model responses are biased to only have entities mentioned in the context and question. At Legion, we have been exploring a variant of Reinforcement learning with a novel engineered reward function to traverse the LLM parameter space for reducing Type 1 hallucination. We will present those results in an upcoming blog post on the topic. ## Appendix ### Mathematically speaking, why do hallucinations occur? To get to the root of why generative AI systems are susceptible to hallucinations, it is essential to briefly describe the inner workings of generative AI systems (like large language models). 1. From the lens of mathematical functions 2. From the lens of probability distributions