(English)RAG with Local LLMs: Ollama, LangChain & ChromaDB

About this course

This course includes the use of AI.English narration and English captions are included. This course uses AI-generated narration and AI-assisted translation. The instructor is responsible for the course content. Turn on captions to follow technical terms and code more easily.

Build a document question-answering system that runs on your own computer, then learn how to test whether it actually answers well.

A working RAG demo is only the beginning. The more useful question is: when it gives the wrong answer, can you find out why? Was the evidence missing from retrieval? Did chunking separate a condition from its exception? Did the prompt make the model refuse a question it could answer? Or did a more expensive retrieval technique improve a ranking metric without improving the final answer?

In this practical course, you will build the complete workflow with Python, Ollama, Qwen2.5, bge-m3, ChromaDB and LangChain. After downloading the required models and packages, document retrieval and answer generation use local models. No paid cloud LLM API key is required for the course application. Initial installation and optional model downloads require an internet connection.

The course is organized into eight sections and six implementation phases. Start with setup, then explore embeddings and cosine similarity with NumPy. Extract and clean text, design chunks and metadata, store vectors in ChromaDB, and connect retrieval to generation using LangChain and LCEL. Finally, build a local Gradio application with source display, incremental indexing and follow-up questions.

Evaluation is central. You will work with a 30-question English evaluation set covering direct questions, identifiers, negation, questions spanning documents and paraphrases. Measure Recall@k, MRR and nDCG, inspect answer checks and refusals, and compare prompt variants, BM25, hybrid retrieval, Reciprocal Rank Fusion and reranking.

Read the full description on Udemy on Udemy (opens in a new tab)

Prices, discounts and availability are set by Udemy. We may earn a commission when you purchase through links on this site.