[Python] How to check total no. of tokens while using Langchain's...

I am using langchain's create_pandas_dataframe_agent agent to analyse a dataframe. The code looks like below:

from langchain_experimental.agents import create_pandas_dataframe_agent
import pandas as pd
from langchain_openai import AzureOpenAI

df = pd.read_csv("file_path")

llm = AzureOpenAI(
deployment_name=name, # I have a variable 'name'
temperature=0.0,
)
agent_executor = create_pandas_dataframe_agent(
llm,
df,
# Few other params
)

prompt = """ Some Text """
agent_executor.invoke(prompt)


Now as per my understanding when agent's invoke is called both the prompt and the df is passed to the LLM.

Note: I am using gpt 3.5 turbo instruct as my LLM.

Now I want to check how many tokens are consumed when I run this code. Any idea how to know that, preferably using some code.

I tried checking Azure Dashboard, but it's difficult to isolate tokens from a single request.

Continue reading...