Merge pull request #210 from sergiopaniego/multimodal-rag-colpali

Added Cookbook for Multimodal Retrieval-Augmented Generation (RAG) with Document Retrieval (ColPali) and Vision Language Models (VLMs)
huggingface · Oct 7, 2024 · 6db545d · 6db545d
2 parents b5d4ed9 + dcc741c
commit 6db545d
Show file tree

Hide file tree

Showing 3 changed files with 10,753 additions and 0 deletions.
diff --git a/notebooks/en/_toctree.yml b/notebooks/en/_toctree.yml
@@ -86,6 +86,8 @@
           title: Analyzing Artistic Styles with Multimodal Embeddings
         - local: faiss_with_hf_datasets_and_clip
           title: Embedding multimodal data for similarity search
+        - local: multimodal_rag_using_document_retrieval_and_vlms
+          title: Multimodal Retrieval-Augmented Generation (RAG) with Document Retrieval (ColPali) and Vision Language Models (VLMs)
 
     - title: Search Recipes
       isExpanded: false

diff --git a/notebooks/en/index.md b/notebooks/en/index.md
@@ -23,6 +23,7 @@ Check out the recently added notebooks:
 - [Enhancing RAG Reasoning with Knowledge Graphs](rag_with_knowledge_graphs_neo4j)
 - [Fine-Tuning Object Detection on a Custom Dataset 🖼, Deployment in Spaces, and Gradio API Integration](fine_tuning_detr_custom_dataset)
 - [Fine-Tuning a Semantic Segmentation Model on a Custom Dataset and Usage via the Inference API](semantic_segmentation_fine_tuning_inference)
+- [Multimodal Retrieval-Augmented Generation (RAG) with Document Retrieval (ColPali) and Vision Language Models (VLMs)](multimodal_rag_using_document_retrieval_and_vlms)
 
 
 

diff --git a/notebooks/en/multimodal_rag_using_document_retrieval_and_vlms.ipynb b/notebooks/en/multimodal_rag_using_document_retrieval_and_vlms.ipynb