Optimizing Large Language Models: Achieving Accuracy through Fine-Tuning

Table of Contents

1. Introduction
2. The Importance of Fine-Tuning Large Language Models
3. The Role of Subject Matter Experts in Data Development
4. Case Study: Improving a Retrieval-Augmented Generation System
   • Understanding the Problem
   • Challenges with Existing Solutions
   • The Approach to Fine-Tuning
   • Fine-Tuning the Document Chunking and Extraction
   • Fine-Tuning the Embedding Space
   • Fine-Tuning the Chunk Retrieval
5. Results and Analysis
   • The Benefits of Fine-Tuning All Components
   • The Impact of Subject Matter Expert Feedback
6. Conclusion

Introduction

In today's talk, titled "Data Development for Generative AI: A Systems-Level View", we will explore the world of large language models and their potential to solve real-world business problems. However, we will emphasize the need for these models to interact with other components and data sources in a larger ecosystem. Our discussion will be led by Chris Glaze, a Staff Research Scientist at Snorkel AI.

The Importance of Fine-Tuning Large Language Models

Large language models hold great promise in addressing various business challenges. However, their accuracy is highly dependent on the ecosystem in which they operate. It goes beyond just the model itself; the entire environment influences the model's performance. This includes Upstream and downstream components, such as data preprocessing and post-processing stages. Fine-tuning these components, along with the large language model, can significantly enhance accuracy and performance.

The Role of Subject Matter Experts in Data Development

Subject matter experts play a crucial role in data development for generative AI. They possess domain-specific knowledge and understand what "good" looks like in the context of the problem being addressed. Their expertise is invaluable in fine-tuning large language models and other system components. However, relying solely on subject matter experts for individual data point labeling is neither feasible nor cost-effective. We must find scalable ways to involve them in the data development process.

Case Study: Improving a Retrieval-Augmented Generation System

In this case study, we will delve into a project undertaken for a top global bank. The bank had employed an out-of-the-box retrieval-augmented generation system, but it failed to provide accurate answers to queries. We identified the need for fine-tuning not only the large language model but also the upstream components, such as document chunking, embedding space, and chunk retrieval.

Understanding the Problem

The retrieval-augmented generation system was designed to interact with unstructured financial documents, which were often lengthy and complex. It required trained experts to read, interpret, and extract Relevant information from these documents. The existing system struggled due to the generic nature of the embeddings used to encode the documents. Additionally, it lacked the ability to identify fine-grained distinctions, such as dates and legal definitions, necessary for accurate retrieval and generation.

Challenges with Existing Solutions

The baseline system, using default parameters and embeddings, consistently delivered inaccurate responses to queries. The lack of contextualized understanding and fine-grained distinctions hindered the system's performance. We needed to find a way to fine-tune the entire ecosystem to address these challenges.

The Approach to Fine-Tuning

Our approach involved efficiently incorporating subject matter expert feedback into the fine-tuning process. We aimed to keep subject matter experts in the loop while leveraging scalable tooling to multiply their expertise. By involving them in the creation of training sets and injecting their logic and intuitions, we could improve the large language model's ability to learn from the data.

Fine-Tuning the Document Chunking and Extraction

To address the challenges with document chunking and information extraction, we developed custom extractors. These extractors, created with minimal development time, were designed to identify subtle signals such as dates and legal definitions within the financial documents. Through subject matter expert input and programmatic annotations, we achieved highly accurate results.

Fine-Tuning the Embedding Space

The embeddings used in the retrieval-augmented generation system inherently struggled to distinguish relevant and irrelevant chunks within the documents. By fine-tuning the embedding space in consultation with subject matter experts, we were able to train an embedding model that made fine-grained distinctions and significantly improved the system's accuracy.

Fine-Tuning the Chunk Retrieval

The process of selecting and retrieving relevant document chunks for the large language model posed its own challenges. The existing system employed a fixed number of chunks, which proved insufficient for complex queries. We developed an adaptive algorithm that dynamically selected relevant chunks based on the question's nature and distribution of relevance scores. This adaptive approach ensured that the large language model received the necessary information to generate accurate responses.

Results and Analysis

The fine-tuning efforts yielded impressive results, with a 54-point increase in question-answer accuracy achieved within a three-week development period. However, it is important to note that the improvements were not solely attributed to fine-tuning the embedding space. Fine-tuning all components, including the document chunking, extraction, and retrieval, contributed to the significant increase in accuracy. This underscores the importance of considering the entire ecosystem and involving subject matter experts throughout the data development process.

Conclusion

In conclusion, large language models hold immense potential for solving real-world business problems. However, their success hinges on the fine-tuning of the entire ecosystem. By incorporating subject matter expert feedback and using scalable tools and techniques, we can optimize the performance of large language models and the associated components. This case study exemplifies the effectiveness of such an approach, leading to substantial improvements in accuracy and performance.

FAQ:

Q: What is the role of subject matter experts in data development for generative AI?
A: Subject matter experts play a crucial role in fine-tuning large language models and other system components. They provide domain-specific knowledge and understand what constitutes accurate outputs in their field of expertise.

Q: How did fine-tuning the embedding space contribute to the improvement in accuracy?
A: Fine-tuning the embedding space allowed for the recognition of fine-grained distinctions within the financial documents, enabling the system to accurately retrieve relevant information. This led to a significant increase in accuracy and improved the overall performance of the system.

Q: What challenges did the existing retrieval-augmented generation system face?
A: The system struggled to distinguish relevant and irrelevant chunks within the documents, resulting in inaccurate responses. Additionally, it lacked the ability to identify important details, such as dates and legal definitions, which were crucial for accurate retrieval and generation.

Q: How can subject matter experts be involved in the data development process without incurring excessive costs?
A: Scalable tooling and programmatic techniques can be employed to efficiently utilize subject matter expert feedback. By involving them in the creation of training sets and leveraging their expertise, we can optimize the performance of large language models without incurring excessive costs.

Q: What were the main components fine-tuned in the retrieval-augmented generation system?
A: The document chunking and extraction, embedding space, and chunk retrieval stages were all fine-tuned to improve the system's accuracy. Fine-tuning all these components in conjunction with the large language model yielded significant improvements in performance.

Resources: Snorkel AI (www.snorkel.ai)