Basic RAG
Here we show an example of building a basic RAG application with Indexify. We are going to upload content about Kevin Durant from Wikipedia and ask questions about KD's career.
Install the Indexify Extractor SDK, Indexify Langchain and the Indexify Client
Start the Indexify Server
Download an Embedding Extractor
On another terminal start the embedding extractor which we will use to index text from the Wikipedia page.
Initialize Client
We will use the langchain wikipedia loader to download content from wikipedia and upload to Indexify. We will also use langchain to prompt OpenAI for the RAG application.
Instantiate the Indexify Client
Create an Extraction Graph
extraction_graph_spec = """
name: 'wikipediaknowledgebase'
extraction_policies:
- extractor: 'tensorlake/minilm-l6'
name: 'wiki-embedding'
"""
extraction_graph = ExtractionGraph.from_yaml(extraction_graph_spec)
client.create_extraction_graph(extraction_graph)
# display indexes
print(client.indexes())
Add Docs
Now download some pages from Wikipedia and upload them to Indexify
from langchain_community.document_loaders import WikipediaLoader
docs = WikipediaLoader(query="Kevin Durant", load_max_docs=10).load()
Perform RAG
Create a retriever to feed in data from Indexify.
from indexify_langchain import IndexifyRetriever
params = {"name": "wikipediaknowledgebase.wiki-embedding.embedding", "top_k": 20}
retriever = IndexifyRetriever(client=client, params=params)
Initialize the Langchain Retriever, create a chain to prompt OpenAI with data retrieved from Indexify to create a simple Q and A bot
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_openai import ChatOpenAI