Unveiling the Reliability of Language Models: Fact or Fiction

Table of Contents

1. Introduction
2. Language Models and Factual Information
3. Hallucinations in Language Models
4. Predictors of Factual Knowledge
   • 4.1 Knowledge Memorization
   • 4.2 PopQA Dataset
   • 4.3 Popularity and Factual Accuracy
5. The Role of Model Scaling
   • 5.1 Improvement in Memorization
   • 5.2 Scaling and Popularity Threshold
   • 5.3 Retrieval-Based Prompting for Tail Performance
6. Combining Parametric and Non-parametric Memories
   • 6.1 Complementary Nature of Memories
   • 6.2 Adaptive Retrieval for Tail Entities
7. Conclusion

When Not to Trust Language Models: Investigating the Effectiveness of Parametric and Non-parametric Memories

Language models have demonstrated their ability to encode a vast amount of factual information in their parameters. However, they are also prone to producing hallucinations or factual errors. This raises the question of when language models are likely to hallucinate and the factors that contribute to their reliability. In this article, we explore the effectiveness of parametric and non-parametric memories in language models and investigate potential predictors of factual knowledge.

Introduction

Language models have proven to be valuable tools in generating text and answering questions. They often provide accurate and factual information based on their training data. However, there are instances where language models produce convincing but incorrect information, leading to hallucinations. Despite the recognition of hallucinations as a significant issue, we still lack a comprehensive understanding of when language models are likely to hallucinate.

Language Models and Factual Information

Language models contain a wealth of factual information in their parameters. For example, they can accurately provide the number of Beetle species in the world, which can be verified through external sources. However, language models are also capable of generating hallucinations, wherein their outputs include false factual assertions. This poses a challenge in determining when to trust the information provided by language models.

Hallucinations in Language Models

Even though hallucinations have been widely recognized as a problem in language models, there is a lack of actionable understanding regarding their occurrence. Fine-tuning language models on specific tasks to predict factual correctness is one approach, but it requires access to model activations, which are not always available. To gain insights into the predictors of factual knowledge, we introduce a new dataset called PopQA.

Predictors of Factual Knowledge

To investigate the factors that determine the accuracy of information generated by language models, we Create the PopQA dataset. This dataset focuses on Atomic forms of factual knowledge grounded to Wikidata. Each question in the dataset is tied to the popularity of the subject entity. We hypothesize that factual knowledge about popular entities is more likely to be memorized by language models than knowledge about less popular entities.

Knowledge Memorization

We define knowledge memorization as the ability of a language model to correctly generate the target object given the subject and relationship of a knowledge triple. By converting these knowledge triples into natural language questions, we can assess the language model's memorization abilities.

PopQA Dataset

The PopQA dataset consists of knowledge triples in a subject-relationship-object format, with each question tied to the popularity of the subject entity. This popularity is determined by the monthly page views of the entity's Wikipedia page. We analyze the correlation between factual accuracy and the log popularity of the subject entity using various models and relationship types.

Popularity and Factual Accuracy

Our analysis reveals a positive correlation between factual accuracy and the popularity of the subject entity. Popular entities are more likely to be memorized accurately by language models compared to less popular entities. This understanding helps us in determining the reliability of language models based on the subject entity's popularity.

The Role of Model Scaling

Model scaling has shown significant improvements in various tasks, indicating progress in bridging the gap between parametric abilities and non-parametric retrieval. However, when it comes to factual knowledge, scaling primarily improves the memorization of popular entities. The impact on memorization in the tails of popularity, where less popular entities reside, is minimal.

Improvement in Memorization

For questions about the least popular entities, scaling does not lead to a noticeable improvement in memorization. The graph demonstrates that as the model Scale increases, the improvement in factual accuracy remains near zero.

Scaling and Popularity Threshold

We observe that model scaling acts as a soft threshold on popularity above which the language model is likely to have memorized the fact accurately. However, for questions about less popular entities, scaling does not contribute significantly to improved memorization.

Retrieval-based Prompting for Tail Performance

While scaling has limited impact on tail performance, retrieval-based prompting shows promising results. By retrieving documents from the web and incorporating them into the language model's Context, we can enhance performance, particularly for smaller language models. This retrieval-based prompting approach outperforms vanilla models and confirms trends observed in recent papers on knowledge-intensive tasks.

Combining Parametric and Non-parametric Memories

Our investigation not only focuses on parametric and non-parametric memories separately but also explores their complementarity. One of our key findings is that retrieval-based augmentation complements the parametric knowledge of language models, especially for questions about tail entities.

Complementary Nature of Memories

Parametric memories represent the knowledge encoded in the language model's parameters, while non-parametric memories incorporate external knowledge from retrieved documents. By combining both types of memories, language models can fill the gaps in their parametric knowledge, resulting in improved performance for less popular entities.

Adaptive Retrieval for Tail Entities

Considering the harmful effect of retrieved documents on questions about the most popular entities, we propose adaptive retrieval. This approach involves setting a threshold popularity level, below which retrieval is used and above which it is avoided. Adaptive retrieval offers substantial improvements, particularly for stronger language models that have already memorized a significant amount of information.

Conclusion

In this article, we have examined the factors that impact the reliability of language models in generating factual information. We introduced the PopQA dataset and showed that the popularity of the subject entity is a strong predictor of factual accuracy. We also explored the role of model scaling and the benefit of combining parametric and non-parametric memories. By understanding the limitations and strengths of language models, we can enhance their factual reliability and improve their application in various domains.

Highlights

• Language models encode a wealth of factual information but are prone to hallucinations.
• The PopQA dataset reveals a positive correlation between factual accuracy and subject entity popularity.
• Model scaling primarily improves memorization of popular entities, while retrieval-based prompting enhances performance for less popular entities.
• Combining parametric and non-parametric memories leads to improved performance, particularly for tail entities.
• Adaptive retrieval helps avoid unnecessary knowledge retrieval and reduces inference time and API costs.

FAQ

Q: Can language models be trusted to provide factual information? A: Language models can provide accurate factual information but are also prone to generating hallucinations. It's important to consider factors such as the popularity of the subject entity and the use of retrieval-based prompting to improve reliability.

Q: How does model scaling impact the factual accuracy of language models? A: Model scaling primarily improves the memorization of popular entities but has limited impact on less popular entities. Retrieval-based prompting shows more promising results in enhancing performance, especially for smaller language models.

Q: What is the benefit of combining parametric and non-parametric memories? A: Combining parametric and non-parametric memories allows language models to fill the gaps in their knowledge. Non-parametric retrieval-based augmentation helps improve performance, particularly for tail entities, complementing the parametric knowledge encoded in the models' parameters.

Q: How does adaptive retrieval improve factual knowledge in language models? A: By setting a popularity threshold, adaptive retrieval avoids unnecessary knowledge retrieval for popular entities and focuses on retrieving information for less popular entities. This approach enhances the factual reliability of language models while reducing inference time and API costs.