[Python] Granite 3.3:8b with Ollama — LLM never invokes provided tools during chat interaction

I’m using Granite 3.3:8b via Ollama (latest version) and have implemented a function to call my LLM with some arguments like format_schema and tools.

ollama version is 0.9.6

def run_prompt(messages, model: str, format_schema = None, tools: dict[str, Callable] = None) -> str:
conversation = messages.copy()

while True:
try:
# Send the request to the Ollama server
response: ChatResponse = client.chat(
model=model,
messages=conversation,
format=format_schema.model_json_schema() if format_schema else None,
tools=list(tools.values()) if tools else None,
stream=False,
)

message = response.message

# Check if the message contains tool calls
if message.tool_calls:
for tool_call in message.tool_calls:
logger.info(f"Tool call detected: {tool_call.function.name}")
if function_to_call := tools.get(tool_call.function.name):
logger.info(f"Calling tool: {tool_call.function.name}")
output = function_to_call(**tool_call.function.arguments)
conversation.append({
'role': 'tool',
'name': tool_call.function.name,
'content': output
})
else:
logger.warning(f"Tool not found: {tool_call.function.name}")

# Continue loop to get next message after tool call result
continue

# Final response received
return message.get('content')

except requests.RequestException as e:
logger.error(f"Error running prompt: {e}")
raise


I pass the function this way:

available_tools = {
'rag_search': rag_search,
}


where rag_search is a Callable. This is what the function looks like :

def rag_search(query: str, top_k: int = 5) -> list:
"""
Perform a semantic search over indexed documentation or CSV descriptions.

This tool retrieves the most relevant textual documents or file descriptions
based on a natural language query. It is used to ground reasoning steps in
up-to-date or domain-specific context.

Args:
query (str): Natural language question or goal.
top_k (int, optional): Maximum number of relevant documents to return.

Returns:
list: A list of documents, each with its source, content, and metadata.
"""
results = retriever.semantic_search(query, top_k)
return [
{
"source": r.metadata.get("source", "Unknown"),
"content": r.page_content,
"metadata": r.metadata,
} for r in results
]


I call run_prompt like this:

# Compose LLM prompt
user_input = {
"input_json": json.loads(query),
}

messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": json.dumps(user_input, indent=2, ensure_ascii=False)}
]

response = ""

for _ in range(MAX_RETRIES):
response = run_prompt(
messages=messages,
model=OLLAMA_MODEL,
format_schema=Agent4Output,
tools=available_tools
)


However, no matter what I try, the LLM never triggers any tool call (i.e., message.tool_calls is always empty).

Here is an example of what I send as user input:

You are a data visualization assistant.

You MUST use the tool rag_search before returning ANY JSON.
TOOLS AVAILABLE:
- rag_search(query: str): use this to get info about chart types.

STRICT WORKFLOW:
1. For each possible chart (bar, pie, line, etc.), you MUST call rag_search with a query like "bar chart" or "pie chart".
2. Wait for the tool result.
3. Then build the JSON "visualisations" using that info.

[...]


Here is a response from the LLM with this prompt:

model='granite3.3:8b' created_at='2025-07-23T08:22:36.686825Z' done=True done_reason='stop' total_duration=24844374458 load_duration=47660708 prompt_eval_count=808 prompt_eval_duration=8809000292 eval_count=164 eval_duration=15971681708 message=Message(role='assistant', content='{[...]}', thinking=None, images=None, tool_calls=None)


Has anyone encountered this issue before or knows what could be missing in my setup or prompt to make the model actually use the tools?

Thank you for your help!

Continue reading...