[Python] What are alternative open-source SLM models for conversational retrieval in LangChain...

I'm building a hotel front desk assistant using LangChain in Python. The assistant uses a conversational retrieval system where users can ask questions about hotel services, and the bot replies based on a set of FAQs stored in a local .txt file.

Currently, I’m using Google’s Gemini 1.5 Flash model for the LLM (ChatGoogleGenerativeAI) and GoogleGenerativeAIEmbeddings for embedding generation.

Here’s a simplified version of my current setup:

from langchain_community.document_loaders import TextLoader
from langchain.indexes import VectorstoreIndexCreator
from langchain.text_splitter import CharacterTextSplitter
from langchain_google_genai import ChatGoogleGenerativeAI,
GoogleGenerativeAIEmbeddings
from langchain.memory import ConversationBufferWindowMemory
from langchain.chains import ConversationalRetrievalChain
from langchain.prompts import PromptTemplate

# Prompt setup
custom_prompt = PromptTemplate.from_template("""
You are a friendly hotel assistant...
Context: {context}
Question: {question}
""")

llm = ChatGoogleGenerativeAI(model="gemini-1.5-flash", google_api_key="...")

embeddings = GoogleGenerativeAIEmbeddings(model="models/embedding-001")
# Text loading and retrieval omitted for brevity


I'd like to replace the Gemini model with an open-source Small Language Model (SLM):

Conversational retrieval

FAQ-style interactions

Low-latency use cases

Which SLM models I can use. Any help, guidance, or example code for integrating a local SLM with LangChain's conversational retrieval chain would be really appreciated!

Continue reading...