Illusions Unraveled: The Magic and Madness of Hallucinations in LLMs

Published on

Dec 6, 2023

Research done by Guruprasad Raghavan, Jon Gill, Victor I. Afolabi
TL;DR We have benchmarked several popular open source LLMs (including the latest Llama-v2–7b-chat) to estimate both, the frequency and degree of hallucinations. Overall, we find that on average, popular, open-source models hallucinate close to 55% of the time on a context-aware Q&A task, when tested without any tuning. We also make our evaluation pipeline (code-base) open-source, so anyone can do this! In a future post, we will explain our strategy on how to reduce hallucinations and propose model intervention methods that achieve the same.

Hallucinations are Inherent to LLMs

Hallucinations are observed across a wide array of generative AI systems,

Hallucinations are loosely defined as confident, factually incorrect outputs produced by an AI system not justified by its training data.

For a deeper dive on understanding the mathematical basis for why hallucinations are inherent to all generative AI systems, and specifically to Large language models, check out the section titled “Mathematically speaking, why do hallucinations occur?”

Metrics to Measure LLM Hallucination

One of the biggest challenges in attempting to resolve the issue of LLM hallucination is the abstract, subjective nature of the problem. Answers to questions such as “ How do we know when an LLM hallucinates?” or “ What is considered ground-truth?” are critical to solving the problem; however, these questions do not necessarily have universally accepted answers.

To address these two specific types of hallucinations, we introduce quantitative metrics to measure two types of hallucinations, Type 1 and Type 2 hallucinations, whose definitions are as follows:

Type 1 (T1) hallucination: LLM creates new entities T1 hallucination refers to the scenario when an LLM is fed a prompt that contains a query and context (knowledge from the enterprise document) relevant to the query, and its response contains entities (names of persons, places, organizations, dates) that are not present in the context or query.

Example for Type 1 hallucination

Article: https://techcrunch.com/2023/06/13/striveworks-snaps-up-first-funding-of-33m-to-build-tools-for-machine-learning-operations/

Type 1 Hallucination: Apart from the entity Chariot, none of the other entities are present in the original article or in the question. Although the response might seem correct, it isn’t factually precise, given that all information about Chariot is ONLY present in the article.

Type 2 hallucination: LLM falsifies relation between entities T2 hallucination refers to the scenario when an LLM is fed a prompt that contains a query and context (from the enterprise document) relevant to the query, and its response contains the right entities (previously mentioned in the prompt), but the relationship between the entities is not synchronized with the knowledge mentioned in the prompt.

In this blog, we will focus on Type 1 hallucination and will share our updates on Type 2 hallucination in a future edition.

Benchmarking Type 1 Hallucinations of Instruction-Tuned Open-Source Models

Open-source evaluation pipeline

At Legion, we have designed a pipeline to reliably — and quantitatively — measure the relative degree of hallucination for any LLM generation. Effectively, the pipeline accepts an instruction-tuned model and a dataset, feeds the dataset through the model, and finally, calculates the model’s T1 hallucination score.

This approach enables us to:

• Rapidly filter open-source models (available on HuggingFace) that have desirable features (like, very low hallucination, low toxicity, etc).
• Establish quantitative baselines for measuring progress on reducing LLM hallucination across open-source models

Here’s a schematic of the pipeline that we built for rapid evaluation of open-source models on instruction-based tasks, specifically question-answer with relevant background context. Our pipeline has been open-sourced for the community and is available here.

Synthetic context-aware question answering dataset generation

In our pipeline described above, we generated a dataset using TechCrunch (TC) articles published over the last three years for the context-aware question-answer task. Our evaluation dataset comprises of 300 randomly sampled TC articles. The workflow for generating the dataset is described in detail in the section titled “Q & A dataset curation”

Evaluation of Instruction-Tuned Open-Source Models

We evaluate several open-source models to assess both, their frequency and degree of hallucination (specifically Type 1 Hallucination) on the context-aware question-answer task.

Result — 1: About 55% of instruct-tuned open-source model responses hallucinate entities in their responses.

Result — 2: On average, instruct-tuned open-source model responses have about ‘3’ made-up entities in each answer.

Result — 3: Of the models we’ve evaluated on the Context-aware Q&A task, ‘Llama-v2–7b-chat’ model is the least susceptible to Type 1 hallucination

Result — 4: The subjective performance of the instruct-tuned open source model responses on the Context-aware Q&A task is comparable.

How do we reduce Type 1 Hallucinations in open-source models?

We believe a robust strategy to reduce Type 1 hallucinations in instruct-tuned open-source models is to navigate the LLM parameter space in such a way that it alters the functional map from input text strings to the simplex output vector to ensure model responses are biased to only have entities mentioned in the context and question.

At Legion, we have been exploring a variant of Reinforcement learning with a novel engineered reward function to traverse the LLM parameter space for reducing Type 1 hallucination. We will present those results in an upcoming blog post on the topic.

Appendix

Mathematically speaking, why do hallucinations occur?

To get to the root of why generative AI systems are susceptible to hallucinations, it is essential to briefly describe the inner workings of generative AI systems (like large language models).

1. From the lens of mathematical functions
2. From the lens of probability distributions