RAG - 101

LLM

rag

• RAG - 101
• RAG (Retrieval-Augmented Generation)
  • Schema
  • RAG Pipeline
  • What RAG system do:
  • RAG type
  • Techical
  • LLM Aided Retrieval - Optimizing RAG
  • RAG Optimization

RAG (Retrieval-Augmented Generation)

• Combines information retrieval with a language model to generate accurate, relevant, less hallucination, private-secure and up-to-date answers.
• RAG = Retrieval + Augmented + Generation
• Instead of relying only on what the model was trained on, RAG lets the model look up information from trusted sources before generating a response.
• Retrieves more accurate, contextually appropriate, relevant data from a knowledge base
• Uses an LLM to generate a response based on that context
• RAG systems augment LLM model’s generative capabilities with real-time retrieval of information, ensuring responses are fluent, factually grounded, and relevant.
• Implementing RAG involves considerations such as the size of the context window and the need for data retrieval and storage in appropriate formats

• Core components:
  • Embedding generator
  • Retriever module
  • Prompt Constructor
  • Generator (LLM model)

Schema

RAG Pipeline

• Tech side
  • Raw docs
  • Document Processing
    • Chunking
    • add metadata
  • Embedding generation each chunk
  • Vectors and original text stored at each DB
• User side
  • Query Processing
  • Query Embedding
  • Retrieval module
    • Hybrid search (Similarity)
    • Maximum Marginal Relevance (MMR)
      • Top-k chunks
        • BM25 Search (Top-30)
        • Vector Search (Top-10)
        • RRF Fusion (up to Top-40)
      • Reranker
        • Jina Cross-Encoder Scoring and top-10 Results
        • Most Diverse
  • Context Formatting
    • Organize with Metadata
    • Apply instructions (format, tone, constraints)
• LLM model

What RAG system do:

• Chunk documents
• Turns documents into searchable vectors (embed chunks)
• Finds information using semantic search (retrive top-k chunks)
• Sends relevant context to the LLM
• Generates accurate answeres from the data

RAG type

Techical

LLM Aided Retrieval - Optimizing RAG

• Chunking Strategy
• Hybrid Search: (combination of traditional keyword retrieval (BM25) + dense (vector-based) retrieval)
• Metadata Filtering: use tag, source, type, date, etc.
• Prompt Engineering
• Caching Results: speeds up repeated queries
• Multi-hop RAG: breaks complex queries into sub-questions and retrieves and reasons across RAG.
• Query rewriting: Improve vague user queries
• Self-RAG: Model checks its own output
• SelfQuery: where we use an LLM to convert the user question into a query.
  • Core component: information » Query Parser » Search term + Filter
• Compression: Using LLM we could increase the number of results we can put in the context by shrinking the responses to only the relevant information.

RAG Optimization

• Everything starts with data quality.
• Poor chunking leads to:
  • Too large → noisy, low precision
  • Too small → missing context
• Hybrid search (semantic + keyword) becomes essential
• Re-ranking is critical to prioritize true relevance—not just similarity
• Enforce context. Too much information leads to the “lost in the middle” effect.
• LLMs may ignore retrieved context unless prompts enforce:
  • Grounding
  • Citations
  • Context adherence
• Continuous feedback loops
• Performance monitoring
• Adaptive tuning to evolving data