[Python] Is it possible to access the history of calls done by LangChain LLM object to...

When we create an Agent in LangChain we provide a Large Language Model object (LLM), so that the Agent can make calls to an API provided by OpenAI or any other provider. For example:

llm = OpenAI(temperature=0)

agent = initialize_agent(
[tool_1, tool_2, tool_3],
llm,
agent = 'zero-shot-react-description',
verbose=True
)


To address a single prompt of a user the agent might make several calls to the external API.

Is there a way to access all the calls made by LLM object to the API?

For example here is described a way to get number of tokens in the request and in the response. What I need, instead, is the requests and the responses themselves (and not just number of tokens in them).

Continue reading...