Skip to content

An implementation of Multimodal RAG chatbot for Research paper QnA using Langchain

Notifications You must be signed in to change notification settings

AggarwalShourya/MultiModal-RAG

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 

Repository files navigation

MultiModal-RAG

Overview:

With the growing number of research papers being published it is becoming more and more important to continue reading and understnading them to understand the AI landscape. The following project aims to reduce the gap between the research paper and an engineering undergrad. The following project outlines two approaches to build a chatbot to have a conversation with a chatbot about a research paper(Attention is all you need in this case). Since research paper consists of unstructured data in the form of tables,images and texts it becomes increasing difficult to extract each bit of information.

Approach 1: in MM-RAG

image image

Approach 2: in Multimodal-RAG

the following approach utillisees CLIP and an ensemble retriever using Similarity search and MMR(rereanker)

ChatGPT Image Aug 17, 2025, 07_25_28 PM

About

An implementation of Multimodal RAG chatbot for Research paper QnA using Langchain

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published