[Python] Granite 3.3:8b with Ollama — LLM never invokes provided tools during chat interaction

I’ve been struggling for several days trying to understand an issue I’m facing, without success...

I’m using Granite 3.3:8b via Ollama (latest version) and have implemented a function to call my LLM with some arguments like format_schema and tools.

def run_prompt(messages, model: str, format_schema = None, tools: dict[str, Callable] = None) -> str:
conversation = messages.copy()

while True:
try:
# Send the request to the Ollama server
response: ChatResponse = client.chat(
model=model,
messages=conversation,
format=format_schema.model_json_schema() if format_schema else None,
tools=list(tools.values()) if tools else None,
stream=False,
)

message = response.message

# Check if the message contains tool calls
if message.tool_calls:
for tool_call in message.tool_calls:
logger.info(f"Tool call detected: {tool_call.function.name}")
if function_to_call := tools.get(tool_call.function.name):
logger.info(f"Calling tool: {tool_call.function.name}")
output = function_to_call(**tool_call.function.arguments)
conversation.append({
'role': 'tool',
'name': tool_call.function.name,
'content': output
})
else:
logger.warning(f"Tool not found: {tool_call.function.name}")

# Continue loop to get next message after tool call result
continue

# Final response received
return message.get('content')

except requests.RequestException as e:
logger.error(f"Error running prompt: {e}")
raise


I pass the function this way:

available_tools = {
'rag_search': rag_search,
}


where rag_search is a Callable.

I call run_prompt like this:

# Compose LLM prompt
user_input = {
"input_json": json.loads(query),
}

messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": json.dumps(user_input, indent=2, ensure_ascii=False)}
]

response = ""

for _ in range(MAX_RETRIES):
response = run_prompt(
messages=messages,
model=OLLAMA_MODEL,
format_schema=Agent4Output,
tools=available_tools
)


However, no matter what I try, the LLM never triggers any tool call (i.e., message.tool_calls is always empty).

Has anyone encountered this issue before or knows what could be missing in my setup or prompt to make the model actually use the tools?

Thank you for your help!

Continue reading...