[Python] File searching with query using Ollama and langchain, but it get files which is...

I need to implement a file search system using Ollama, LangChain, and ChromaDB. However, when I search using my current code, it retrieves some irrelevant files from the database.

For example, if I search for "Invoice under 15,000", the system should return only invoice files where the price is below 14,000, but it's not working as expected.

I need a solution to refine the search results and ensure accurate filtering based on numerical conditions.

First Script: Document Processing and Indexing

• Loads various document types (TXT, PDF, DOCX, CSV, etc.) and extracts text.
• Splits large documents into smaller text chunks for efficient processing.
• Converts text chunks into vector embeddings using HuggingFace models.
• Stores embeddings along with metadata in ChromaDB for fast retrieval.
• Enables semantic search to find relevant documents based on user queries.

Second Script: Querying and File Retrieval via Flask API

• Sets up a Flask web app with endpoints for querying and file retrieval.
• Uses a RetrievalQA chain with ChromaDB and Ollama for intelligent search.
• Retrieves and summarizes relevant document content based on queries.
• Filters search results to improve accuracy, especially for numerical queries.
• Provides a download link for source files containing relevant information.

Ollama file search - source file

Continue reading...