Product Insights RAG Chatbot with lakeFS
Get valuable insight into the success of your product through the use of a RAG solution that leverages data versioning from lakeFS by Treeverse.
The success of a product is achieved through delighting customers through its feature set, service, support, cost and overall value. Measuring these factors can be a challenge as information on each area is often siloed with no easy way to aggregate and summarize it into an easily consumable format. Having summarized product insights from customer reviews, sales metrics, support cases, brand recognition, social media likes and follows and more will be critical to creators and sellers of the product. For example, product reviews can be found in myriad places. They might be in an email, the product review page of your company website, a post on social media, in an online shopping destination or even a product review blog. Gathering, storing and parsing through all those possible places for reviews is a massive and constant undertaking, especially since many of those content sources are composed of unstructured data. Sales and support data will also be in their respective customer relationship management (CRM) tools.
Implementing a RAG (Retrieval Augmented Generation) solution that can ingest, transform and make data from these sources searchable, for better prompts and more accurate responses, is an important part of getting valuable insights about your product from users.
Using lakeFS by Treeverse as the data versioning component allows data engineers to isolate versions of the data in the form of branches, similar to code branches. lakeFS offers a git-like user interface and workflow allowing the use of branches that can be used for each stage from dev, to testing, to production without duplicating data. Testing with new data is as easy as creating a new branch on the data repository, adding additional files, and committing the changes.
- Data Gathering
User Role(s): Data Engineer
The data engineer is responsible for gathering data from multiple sources and uploading it to the data store. The data in this use-case may come from multiple sources, including HTML, JSON, PDFs, CSV, etc. Data gathering is performed outside of the OpenShift cluster. The data engineer will interface directly with lakeFS to manage the lifecycle of the data and models used in the use-case.
- Data Versioning with lakeFS by Treeverse
User Role(s): Data Engineer, Data Scientist, AI/MLOps
lakeFS by Treeverse is a Partner Validated, containerized application that is supported on OpenShift and offers the data engineer a familiar git-like workflow to upload and manage the data. Similar to how developers use git for source code versioning, the data engineer can log into lakeFS, create a repository, then upload their data to the repository. Each data repository configured in lakeFS will have a connection to a backend object storage using an AWS S3 RESTful interface. The data engineer may also leverage lakeFS’s S3 Gateway functionality, and associated S3 API, to interface with the data repository in lakeFS. One of the major values that lakeFS brings, is the versioning of object data, without data duplication.
In addition to being the interface used by the data engineer to manage the lifecycle of data and models, lakeFS will also serve as an integration point for the chatbot application, through the use of LangChain, to pull data to chunk and store as embeddings in the RAG store. OpenShift AI will also be able to pull models for serving from lakeFS.
- Data Storage in MinIO
User Role(s): Data Engineer, AI/MLOps
AI/ML data is commonly stored in object storage. In this case, we use an on-premises instance of MinIO running in the local OpenShift cluster, however a cloud-based MinIO service could also be used. MinIO will enable the data engineer to securely store the object data in storage buckets. These storage buckets will be mapped to one or more data repositories in lakeFS. MinIO may also store the LLM (Large Language Model), in which case OpenShift AI would have a Data Connection configured to pull the model from lakeFS, which would pull it from MinIO on the backend, as shown in the diagram.
- RAG Store and Search from FAISS
FAISS provides the RAG store and vector database search functions. This is where the proprietary or private data will reside and be searched for context of the user’s question to the LLM. It will accept search requests from the chatbot application and return context that can then be used by the chatbot in the prompt to the LLM. FAISS may utilize Nvidia AI accelerators for resource intensive tasks.
- Product Insights RAG Chatbot
The chatbot provides the following:
- Document processing, chunking, and storage of embeddings in the RAG store
- API for user interface to ask questions of the LLM.
- Search requests to the RAG store to obtain proprietary or private information, which represents context about the question
- Combines the context obtained from the RAG store with the question from the user, to create a prompt to be sent to the LLM’s endpoint URL
- Sends the prompt to the LLM
- Receive the response from the LLM and forward it back to the user
The chatbot may also run as a containerized workload on the local OpenShift cluster.
