Can I read Reinforcement Learning for Optimizing RAG for Domain Chatbots on EtoBox?

Reinforcement Learning for Optimizing RAG for Domain Chatbots by Kulkarni, Mandar; Tangarajan, Praveen; Kim, Kyung; Trivedi, Anusua is a scholarly article available to read on EtoBox.

What is Reinforcement Learning for Optimizing RAG for Domain Chatbots about?

With the advent of Large Language Models (LLM), conversational assistants have become prevalent for domain use cases. LLMs acquire the ability to contextual question answering through training, and Retrieval Augmented Generation (RAG) further enables the bot to answer domain-specific questions. This paper describes a RAG-based approach for building a chatbot that answers user's queries using Frequently Asked Questions (FAQ) data. We train an in-house retrieval embedding model using infoNCE loss, and experimental results demonstrate that the in-house model works significantly better than the well-known general-purpose public embedding model, both in terms of retrieval accuracy and Out-of-Domain (OOD) query detection. As an LLM, we use an open API-based paid ChatGPT model. We noticed that a previously retrieved-context could be used to generate an answer for specific patterns/sequences of queries (e.g., follow-up queries). Hence, there is a scope to optimize the number of LLM tokens and cost. Assuming a fixed retrieval model and an LLM, we optimize the number of LLM tokens using Reinforcement Learning (RL). Specifically, we propose a policy-based model external to the RAG, which inte

Author

Kulkarni, Mandar; Tangarajan, Praveen; Kim, Kyung; Trivedi, Anusua

Published

2024

Language

EN