[Python] Why is my RAG LLM not retrieving my custom data?

I am very new to RAG and LLM so forgive some of my technical skills.

I'm trying to create a chatbot using LLMs that teaches DSA topics, and I learnt that using RAG technique you can supposedly feed new data into the LLM so it can give you the most updated answer.

Basically this was the block diagram I saw

(https://i.sstatic.net/x8M1fMiI.png)

So any new information, you can add it into the vectorstore/data store and the embedding models will supposedly fetch it into the LLM and give you said updated answer.

I tried to input some custom data into my csv full of DSA terms and descriptions (which i scrapped off wikipedia). At the last one, I input a custom term and custom description, hoping that the chatbot will reply the same description.

However, the chatbot replied it did not know the answer my question relating the custom term. Im not sure what did I do wrong, I also tried to use a supposedly better model and it still couldnt answer my question relating to the custom DSA term.

this was the first iteration of my chatbot

from langchain.document_loaders import CSVLoader
from langchain_text_splitters import CharacterTextSplitter
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_chroma import Chroma
from langchain.chains import RetrievalQA
from langchain_community.llms import Ollama

# Initialize the CSVLoader with your CSV file path
loader = CSVLoader(file_path="dsa1.csv")

# Load the documents
docs = loader.load()

# Initialize the CharacterTextSplitter
text_splitter = CharacterTextSplitter(
separator="\n",
chunk_size=2000,
chunk_overlap=200
)

# Split the documents into smaller chunks
texts = text_splitter.split_documents(docs)

embeddings = HuggingFaceEmbeddings()

db = Chroma.from_documents(texts,embeddings)

llm = Ollama(model="llama3.1")

chain = RetrievalQA.from_chain_type(
llm,
retriever = db.as_retriever()
)

question = "What is SIT Ernest CK Travis Junxian tree"
result = chain({"query":question})

print(result)


this was the reply

{'query': 'What is SIT Ernest CK Travis Junxian tree', 'result': 'I don\'t know what a "SIT Ernest CK Travis Junxian tree" is, as the provided context only mentions it without providing any additional information or explanation. The descriptions seem to be cut off or incomplete. If you have more context or clarify what this term refers to, I\'d be happy to try and help!'}


this was the second iteration using a stronger embedding model

from langchain.document_loaders import CSVLoader
from langchain_text_splitters import CharacterTextSplitter
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_chroma import Chroma
from langchain.chains import RetrievalQA
from langchain_community.llms import Ollama

# Enable detailed logging for debugging
# langchain_logging.set_verbosity_debug()

loader = CSVLoader(file_path="dsa1.csv")
docs = loader.load()

text_splitter = CharacterTextSplitter(
separator="\n",
chunk_size=2000,
chunk_overlap=200
)

texts = text_splitter.split_documents(docs)

embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")

db = Chroma.from_documents(texts, embeddings, persist_directory="chroma_db_new")

retriever = db.as_retriever(
search_type="similarity", # Use similarity search
search_kwargs={"k": 5} # Retrieve top 5 documents
)

llm = Ollama(model="llama3.1")

chain = RetrievalQA.from_chain_type(
llm=llm,
retriever=retriever
)

question = "What is SIT Ernest CK Travis Junxian tree"

result = chain.run(question)

print(result)



this was the reply

I don't know.



I did a for loop to print all the chunks of data my vectorstore (chroma) extracted

for i, text in enumerate(texts):
print(f"Chunk {i+1}:\n{text.page_content}\n{'-'*40}")


and my custom term was inside

Chunk 992:
Term: SIT Ernest CK Travis Junxian tree
Description: Possibly the best tree there is. Uses some kind of special LLM called Performance group LLM that will give O(log) complexity speed

Continue reading...