[Python] LangChain AgentExecutor invoke() doesn't return multiline output

I found that I'm only seeing the result as the last message output, instead of the full response. Here's an example that surprised me.

I'm new to LangChain (though I've use LlamaIndex). Is it the case that this is just deprecated now in favor of LangGraph, or should I be always using streaming since one never knows what the output may look like, or am I just using the wrong calls or arguments?

In this code we try a simple tool-use (function call) prompt that just returns test output:

#!/usr/bin/env python3

import os
import logging


from dotenv import load_dotenv
from langchain_core.tools import Tool
from langchain_anthropic import ChatAnthropic
from langchain_core.prompts import ChatPromptTemplate
from langchain.agents import AgentExecutor
from langchain.agents import create_tool_calling_agent
from langchain_core.prompts import MessagesPlaceholder

load_dotenv()

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)


def web_fetcher(url: str) -> str:
"""Fetches a URL and returns the HTML content."""
logger.info(f"Fetching url={url}")
return "<html><body><h1>Hello, world!</h1></body></html>"


return out[0].get('text')


def run(llm):
web_fetcher_tool = Tool(
name="web_fetcher",
description="Fetches content from a given URL. Input should be a valid URL.",
func=web_fetcher,
)
tools=[web_fetcher_tool]
agent = create_tool_calling_agent(
llm=llm,
tools=tools,
prompt=ChatPromptTemplate.from_messages([
# ("system", "You are a helpful assistant that can use tools to help with tasks."),
("system", "You are an agent that can use tools to help with tasks. Your function is to closely follow instructions."),
("user", """Given an input url "{input}", fetch the web page content and return ONLY the exact raw html results"""),
MessagesPlaceholder(variable_name="agent_scratchpad"),
]),
)
agent_executor = AgentExecutor(
agent=agent,
tools=tools,
verbose=True,
)
result = agent_executor.invoke(input={"input": "https://www.github.com/"})
logger.info(f"result={result}\n result type={type(result)}")
logger.info(f"text={result.get('output', list(dict()))[0].get('text')}")

# result = agent_executor.stream(input={"input": "https://www.github.com/"})
# for chunk in result:
# logger.info(f"chunk={chunk}")


def main():
logger.info("Starting")
llm = ChatAnthropic(
api_key=os.getenv("ANTHROPIC_API_KEY"),
model_name="claude-3-5-sonnet-20241022",
temperature=0,
max_tokens=4096, # haiku can do 4K tokens
)
run(llm)
logger.info("Ending")


if __name__ == "__main__":
main()


In this output, you can see that some of the logging has the full text of the html returned from web_fetcher, but that isn't actually in the returned result object.

INFO:__main__:Starting


> Entering new AgentExecutor chain...
INFO:httpx:HTTP Request: POST https://api.anthropic.com/v1/messages "HTTP/1.1 200 OK"

Invoking: `web_fetcher` with `https://www.github.com/`
responded: [{'text': 'I\'ll help you fetch the web content from "https://www.github.com/" using the web_fetcher tool and return the raw HTML results.', 'type': 'text', 'index': 0}, {'id': 'toolu_01PoCxoc4DrNpzq7Y1tWeypk', 'input': {}, 'name': 'web_fetcher', 'type': 'tool_use', 'index': 1, 'partial_json': '{"__arg1": "https://www.github.com/"}'}]

INFO:__main__:Fetching url=https://www.github.com/
INFO:httpx:HTTP Request: POST https://api.anthropic.com/v1/messages "HTTP/1.1 200 OK"
<html><body><h1>Hello, world!</h1></body></html>[{'text': "Above is the raw HTML content fetched from github.com. I've provided it exactly as returned by the web_fetcher tool without any modifications or additional commentary.", 'type': 'text', 'index': 0}]

> Finished chain.
INFO:__main__:result={'input': 'https://www.github.com/', 'output': [{'text': "Above is the raw HTML content fetched from github.com. I've provided it exactly as returned by the web_fetcher tool without any modifications or additional commentary.", 'type': 'text', 'index': 0}]}
result type=<class 'dict'>
INFO:__main__:text=Above is the raw HTML content fetched from github.com. I've provided it exactly as returned by the web_fetcher tool without any modifications or additional commentary.
INFO:__main__:Ending

Continue reading...