[Python] Claude Sonnet can call tool only once at a time

I'm testing a function calling capability of Claude Sonnet 3.7, but it can only call the tool once before it responds to the client.

Here is my code:

from llama_index.core.program.function_program import FunctionCallingProgram
from llama_index.llms.anthropic import Anthropic
from llama_index.core import PromptTemplate

class ObjectInstance(BaseModel):
type: str = Field(
...,
description="The category or class of the object extracted from the text. Must exactly match one of the predefined object types provided.",
)
name: str = Field(
...,
description="The exact name of the object as it appears in the text in upper case.",
)
description: str = Field(
...,
description="Information about the object from the text it is extracted from",
)


test_prompt = PromptTemplate(
"""
Task Description:

From a given text, extract one or more objects that match the object type provided.
For each object, call the provided tool to format the output.

OBject Types and Descriptions:

Entity Type 1: PERSON
Description: A name of a person
Entity Type 2: CITY NAME
Description: A name of a city

Text:

{content}
"""
)

llm = Anthropic(model = "claude-3-7-sonnet-20250219")

program = FunctionCallingProgram.from_defaults(
output_cls=ObjectInstance,
prompt=test_prompt,
llm=llm,
allow_parallel_tool_calls=True,
)

content = "John Doe is a person who lives in San Francisco."
output = program(content=content)

# It only extracts
# Name: JOHN DOE
# Type: PERSON

# It should extract this as well
# Name: San Francisco
# Type: CITY NAME


I also tried using structured_predict and get_function_tool, but both failed, resulting in extracting only one object.

My attempt No. 2

response = llm.structured_predict(
output_cls, prompt, content=content, allow_parallel_tool_calls=True
)


My attempt No. 3

from llama_index.core.program.function_program import get_function_tool

tool = get_function_tool(output_cls)

resp = llm.chat_with_tools(
[tool],
user_msg=prompt.format(content=content),
allow_parallel_tool_calls=True,
)

tool_calls = llm.get_tool_calls_from_response(
resp, error_on_no_tool_calls=False
)


For the same code and prompt, both gpt-4o and mistral-large were able to call the tool multiple times. Only Anthropic Claude fails to achieve this. I wonder if I'm missing something.

Continue reading...