[Python] How to make a fast and accurate relevant data extraction for a chatbot? [closed]

I'm working on a property management assistant that needs to search through a database containing information about apartments, amenities, services, contact info, etc., but I'm struggling with performance and accuracy issues. Currently, I have a large JSON file with nested property information and I'm using tree navigation with multiple Claude API calls to traverse and find relevant data when users ask natural language questions like "What's the gym schedule?" or "How do I report a maintenance issue?"

The problem is that this approach is really slow because of the multiple API calls needed to navigate the JSON tree, often inaccurate because it sometimes misses relevant information or gets lost in the tree structure. I'm looking for a fast search solution that can understand natural language queries, ideally with a one-time setup where I can dump all the text data and have an API or service handle the search logic automatically to get more accurate context retrieval for user questions. An ideal scenario would be if I could just put all that data in the context for the LLM, but this would make the costs too high and could exceed the input token window as the database has quite a lot of text.

I'm considering ChromaDB for local vector search but after testing it for a bit it shows itself even less accurate than my approach. Any advice on how to handle it or experience with similar projects would be hugely appreciated! My current tech stack is Python with Claude API for processing and responses.

Continue reading...