Retrieval and RAG
Multimodal RAG: Search and Recommendations
Explore multimodal RAG through AI search and recommender systems using GPT-4, CLIP and ChromaDB.
A small sample of 30,000+ courses curated from around the world, all available to DoJo members.
A free 30-minute conversation with Ted about your goals.
Already a member? Open this course (opens in a new tab)
Course topics
Retrieval and RAG
- Multimodal search
- RAG search system
- Recommender system
You check the sources
Find your fit
Is this a useful starting point?
For learners with Python and embedding/vector knowledge who can install the listed libraries.
- Starting point
- Intermediate
- Course length
- 1.5 total hours
- Lessons
- 24 lectures
Course by Paulo Dichone | Software Engineer, AWS Cloud Practitioner & Instructor
Course last updated: 11/2024
Requirements
- Basic understanding of Python programming.
- Familiarity with machine learning concepts (embeddings, vectors).
- No prior experience with multimodal systems is needed, but knowledge of AI tools like GPT or CLIP will be helpful.
- A computer with internet access and the ability to install Python libraries like Streamlit, OpenAI, and ChromaDB.
What is included
- 1.5 hours on-demand video
- 2 articles
- Access on mobile
- Certificate of completion
DoJo membership: A$250 per month per seat, excluding GST. Includes fortnightly group support and a private member community.
Explore the lessons
Course curriculum.
8 sections • 24 lectures • 1h 35m total length
Introduction3 lectures • 6min
- Introduction & Prerequisites02:37
- Course Structure00:32
- WATCH THIS DEMO - What You'll Build in This Course02:37
Download Code and Resources2 lectures • 2min
- PLEASE Watch this - How To Get the Source Code02:16
- Download source code00:01
Development environment Setup1 lecture • 1min
- Development Environment Setup - Overview01:04
RAG (Retrieval Augmented Generation) and Multimodal Systems Deep Dive3 lectures • 17min
- RAG Systems - Deep Dive Crush Course04:17
- RAG Benefits and Practical Application04:14
- Multimodal RAG - Overview & Motivation and Benefits - How it Works08:54
Search in a Multimodal RAG System3 lectures • 11min
- How Search is Integrated into a Multimodal RAG System - Full Workflow03:19
- Why Multimodal Search is so Powerful02:53
- Visual Explanation Why Multimodal Search is so Powerful04:47
Hands-on: Multimodal Search RAG System2 lectures • 17min
- Multimodal Search System Setup - Create Embeddings of Images07:57
- Finish the Multimodal Search System09:17
Hands-On - Multimodal Recommender System8 lectures • 36min
- Multimodal Recommender System - Overview03:28
- Getting our Dataset from Hugging Face & Showing Number of Rows03:27
- Saving all Images Locally03:00
- Saving Image Embeddings to Vector Database04:51
- Testing our Multimodal Recommender System - Fetching the Correct Images03:52
- Setting up the RAG Flow - Part 108:12
- Putting it all Together and Testing the Multimodal Recommender RAG System06:29
- Adding a UI to the Multimodal Recommender System - Streamlit02:58
Next Steps2 lectures • 5min
- Next steps02:53
- Bonus Lecture01:55
From learning to practice
Try this next.
Compare search and recommendation results from a small multimodal learning example.
A useful next step
Choose where to start.
Talk through your goals in a free 30-minute discovery call with Ted. Bring a task you want to improve and the tools you already use.
Book a discovery callExplore DoJo membership