[Python] How to implement context-aware tool routing with local models like Ollama?

I'm using a locally hosted model(llama3.2) with Ollama and trying to replicate functionality similar to bind_tools(to create and run the tools with LLM ) for tool calling. Since bind_tools is not available in the Ollama model llama3.2, I created a custom router like this:

def router_node(state):
input_text = state['messages'][-1]['content']
if 'account' in input_text.lower():
return 'chatbot_node'
else:
return 'other_node'


This approach works for simple keyword matching, but it doesn’t really understand the full context or intent behind a user’s input. I’m looking for a better, more context-aware way to route requests to different tools or nodes.

What I'm looking for:

• A method to semantically analyze the input and decide which tool to invoke.
• A way to achieve this using Ollama (or any local model) since bind_tools is not available.

Has anyone faced this issue or found an effective way to do context-based tool routing with local LLMs?

Continue reading...