[Python] Planning a Multi-Model SQL Chatbot for Pharma , Would Love Feedback [closed]

I’m the sole AI engineer at a pharma company (yep, just me!), and I’m currently designing an internal AI assistant that lets non-technical teams query our SQL databases using plain English. I’m a fresher, working alone on a semi-enterprise level project, and to be honest, it’s a little overwhelming. I'm handling everything myself, including architecture, model routing, cost planning, and overall design. I want to make sure the approach I'm considering is solid before I commit to building the full system.

Here’s what I’ve planned so far using LangChain: User asks a natural language query. LangChain introspects the SQL schema and feeds that structure to the LLM. The LLM generates a SQL query. Query is executed, and the results are passed back to the LLM. The LLM explains the output in plain English. All this is wrapped in a chatbot interface. My Cost-Control Strategy (Open to Input) Since we’re planning to use cloud LLMs, cost is a big concern. I’m testing two routing approaches: Plan A: Use GPT-4o (or similar heavy model) for generating SQL. Use GPT-3.5 (or Gemini Flash) for summarizing the SQL output. Plan B (My current Strategy): User query is first sent to the light model. It tries to handle the SQL gen directly if simple enough. If not, it outputs a No, and I route the query to GPT-4o for SQL generation. The summarization always stays with the light model to keep costs low. Is the approach I’ve come up with truly viable? What other methods or strategies can I implement to improve it further?

Continue reading...