There are three types of language models (LLM’s) for custom applications.

  • Prompt Engineering
  • Retrieval Augmented Generation
    • Combining language models with your own data
  • Fine-Tuning
    • Language models are retrained with your own data

How Does RAG Work?

The following diagram illustrates a RAG pipeline:

Overview of RAG

  • Indexing Stage
    • Conversion of private data into searchable vector indices
    • Processing various data types such as text documents, database entries, and knowledge graphs
  • Storing
    • Typically, indexed data is stored in memory
    • Also possible to store it in a directory
  • Vector Stores
    • Vector stores are useful for storing embeddings created during the indexing process
    • Embeddings
      • The vector store index converts entire text into embeddings using an LLM
      • Output of k most similar embeddings as text snippets
  • Query
    • Original query and relevant information snippets from the vector index are passed to the language model
  • Retrieval
    • The system retrieves the most relevant information from stored indices and forwards it to the LLM, which responds with current and context-specific information.
  • Postprocessing
  • Response Synthesis
    • The response synthesis is the final phase where the query, the most relevant data, and the initial prompt are combined and sent to the LLM to generate a response.

Application Example: Searching for Suitable Lamp

from llama_index.core import VectorStoreIndex, get_response_synthesizer, Settings, Document
from llama_index.core.retrievers import VectorIndexRetriever
from llama_index.core.query_engine import RetrieverQueryEngine
from llama_index.core.postprocessor import SimilarityPostprocessor
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.llms.ollama import Ollama
import os

documents = [
     Document(text='name": "Leuchte 1", "typ": "Pendelleuchte", "leistung": "12W LED'),
     Document(text='name": "Leuchte 2", "typ": "Wandlampe", "leistung": "8W LED'),
     Document(text='name": "Leuchte 3", "typ": "Stehlampe", "leistung": "15W LED"}'),
     Document(text='name": "Leuchte 4", "typ": "Deckenleuchte", "leistung": "20W LED"}'),
     Document(text='name": "Leuchte 5", "typ": "Tischlampe", "leistung": "10W LED"}'),
     Document(text='name": "Leuchte 6", "typ": "Strahler", "leistung": "30W Halogen"}'),
     Document(text='name": "Powerglare", "typ": "Flutlicht", "leistung": "100W LED"}'),
     Document(text='name": "Leuchte 8", "typ": "Außenlampe", "leistung": "18W LED"}'),
     Document(text='name": "Leuchte 9", "typ": "Notbeleuchtung", "leistung": "5W LED"}'),
     Document(text='name": "Leuchte 10", "typ": "Schienenstrahler", "leistung": "25W LED"}'),
]

# Model Loading
os.environ["HF_HOME"] = "./model_cache"
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-base-en-v1.5")
Settings.llm = Ollama(model="llama3.1:8b", request_timeout=360.0)

# build index
index = VectorStoreIndex.from_documents(documents)

# configure retriever
retriever = VectorIndexRetriever(
    index=index,
    similarity_top_k=10,
)

# configure response synthesizer
response_synthesizer = get_response_synthesizer(response_mode="compact")

# assemble query engine
query_engine = RetrieverQueryEngine(
    retriever=retriever,
    response_synthesizer=response_synthesizer,
    node_postprocessors=[SimilarityPostprocessor(similarity_cutoff=0.2)],
)

# Query
response = query_engine.query("What is the name of our only Flutlicht type lamp")
print(response)

Learnings

  • Operation of RAG Pipeline
  • Optimization of search speed with caching, configuration of synthesizer, and use of “smaller” models (see Link to Repo)
  • Configuration of search quantity, factors influencing search results

References