After this lesson, you will be able to: Design a Retrieval-Augmented Generation pipeline end-to-end: chunking, embedding model, vector store choice, retrieval, and quality measurement.
RAG is the most common agent pattern in production. It also has the most subtle quality failures. This lesson covers the design decisions that separate a RAG that works from one that confidently hallucinates.
RAG = Retrieval-Augmented Generation. Given a user query, retrieve relevant chunks from a document store, stuff them into the LLM prompt, generate an answer grounded in the retrieved content. Use RAG when: your knowledge base is too large for the context window, your knowledge changes frequently (no time to re-train), or you need citations to source documents. Skip RAG when: the whole corpus fits in a single context window (Claude 4 / Gemini 2 Pro 1M+ tokens), or the task is reasoning/style not retrieval.
Chunk size: 200-800 tokens is the typical range. Smaller = more precise retrieval but loses context; larger = more context but dilutes relevance. Start at 500. Chunk overlap: 10-20% (50-100 tokens) preserves continuity across boundaries. Chunking method: by paragraph (markdown), by sentence + token cap (LangChain's RecursiveCharacterTextSplitter), or semantic (split where embedding similarity drops). Always test chunking against a real query set; the right strategy depends on document structure.
OpenAI text-embedding-3-small (1536 dim, ~$0.02/M tokens). Default for most production work; cheap, fast, good. OpenAI text-embedding-3-large (3072 dim, ~$0.13/M tokens). Marginally better at the high end; rarely worth the cost. Voyage AI voyage-3 (Anthropic's preferred partner). Strong for technical content. Cohere embed-v3. Strong multilingual. Open-source: BAAI/bge-large-en-v1.5, intfloat/e5-large-v2. Self-host via Hugging Face. Rule: use the same embedding model for indexing and querying. Mixing them silently destroys recall.
Pinecone: hosted, fast, free tier for small projects ($0 up to 1M vectors). Easiest start. Chroma: open-source, run locally or self-hosted. The dev favourite. pgvector (Postgres extension): your existing Postgres becomes a vector DB. Single-DB simplicity wins in real production. Weaviate, Qdrant: open-source alternatives with hybrid (vector + keyword) search built in. Picking rule: pgvector if you already use Postgres; Pinecone if you want zero ops; Chroma for prototyping; Weaviate / Qdrant for advanced hybrid search.
Same pattern, whether you use Flowise, LangFlow, LangChain code, or LlamaIndex.
Indexing chatflow (run once): Document Loader (PDF / web / text) → Text Splitter (Recursive, chunk=500, overlap=100) → Embedding (text-embedding-3-small) → Vector Store (Pinecone or Chroma)
Query chatflow (runs per user message): Chat Input → Vector Store Retriever (k=5) → Prompt Template ('Answer using only the following context: {context}. Question: {question}') → LLM (claude-sonnet-4-6) → Chat Output
Add 'Cite sources' by including chunk metadata in the prompt and instructing the model to reference them
Test with 5-10 real questions; check whether the retrieved chunks are relevant before judging the answer quality
Trusting cosine similarity as ground truth. Always look at the actual retrieved chunks for a few queries; numbers lie. Indexing the entire corpus once, then never updating. Real corpora change; build an incremental update path from day one. Letting users ask anything and assuming the retriever will find a relevant chunk. If the question is off-topic, the retriever returns near-misses and the LLM still confidently answers. Add an 'is this in scope?' check. Skipping hybrid (keyword + vector) search. Pure semantic retrieval misses exact-match technical queries. Not citing sources. Without citations, users have no recourse when the model hallucinates a chunk into existence.
Pick the highest-impact diagnostic.
Sign in and purchase access to unlock this lesson.