I have tried to train PandasAI in two ways: instruction training & q/a training and I followed the tutorial here exactly：https://docs.getpanda.ai/v2/intro. I'm currently using ChromaDB locally that stores training data, but when I ask the query, it doesn't seem to fetch anything from vectorstore that is correctly configured and populated. By the way, I can't move to pandasai v3.0 cause having some problems with other dependecies inside the project, so I'm using v2.0. Relevant dependecies: _pandasai==2.4.2 chromadb==0.5.18 langchain_groq==0.3.2 pandas==1.5.3 numpy==1.26.4 protobuf==5.28.3 sentence_transformers==3.3.0 lancedb==0.15.0 pymilvus==2.4.9 onnxruntime==1.20.0 marshmallow==3.23.1 Here is my code (I use jupyter): import os import logging import pandas as pd from pandasai.agent import Agent from pandasai.ee.vectorstores import ChromaDB from langchain_groq import ChatGroq from pandasai.helpers.logger import Logger logger = Logger(save_logs=True, verbose=True) logger.log("Starting data processing", level=logging.INFO) os.environ["PANDASAI_API_KEY"] = "{value}" llm = ChatGroq( model_name="deepseek-r1-distill-llama-70b", api_key="{value}", timeout=30, max_retries=2, temperature=0, max_tokens=1500, ) train_file_path = "marvel_characters_dataset.csv" # https://www.kaggle.com/datasets/hamnakaleemds/marvel-universe-characters-dataset df = pd.read_csv(train_file_path) agent = Agent(df, vectorstore=ChromaDB(), config={"llm": llm}) query = "Show the count of each type of role" code = """ ## Given the query asked by user you need to generate the code in following format. import pandas as pd df = dfs[0] ## Filter the dataframe by relevant column ## Add additional filters by column based on the query ## Always ignore role with value: 'Antihero' all_roles = [] all_roles.extend(df['Role'].tolist()) counts = {} for role in all_roles: if role != 'Antihero': # Skip 'Antihero' roles: ignore this type of role if role in counts: counts[role] += 1 else: counts[role] = 1 ## Always provide the result in the following format do not add return statement result_df = pd.DataFrame({'Role': list(counts.keys()), 'Count': list(counts.values())}) result = {'type': 'dataframe', 'value': result_df} ## Strictly do not add any return statement in the end """ agent.train(queries=[query], codes= Código: , docs=["The role with value 'Antihero' is not a type of role, always ignore"]) agent.chat("Show the count of each type of role") The output doesn't consider the training information that I've configured (ignore role with value 'Antihero' and in the log doesn't show anything related to retriever in chromaDB). My log file: 2025-05-04 13:38:23 [INFO] Starting data processing 2025-05-04 13:38:54 [INFO] Persisting Agent Training data in c:\test-pandasai\chromadb 2025-05-04 13:38:54 [INFO] Successfully initialized collection pandasai 2025-05-04 13:39:04 [INFO] Agent successfully trained on the data 2025-05-04 13:39:21 [INFO] Question: Show the count of each type of role 2025-05-04 13:39:21 [INFO] Running PandasAI with langchain_groq-chat LLM... 2025-05-04 13:39:21 [INFO] Prompt ID: df8e38e9-7de1-4686-80fa-34abdb2afdf8 2025-05-04 13:39:21 [INFO] Executing Pipeline: GenerateChatPipeline 2025-05-04 13:39:21 [INFO] Executing Step 0: ValidatePipelineInput 2025-05-04 13:39:21 [INFO] Executing Step 1: CacheLookup 2025-05-04 13:39:21 [INFO] Executing Step 2: PromptGeneration 2025-05-04 13:39:21 [INFO] Using prompt: <dataframe> dfs[0]:45x6 Character,Real Name,Affiliation,Powers,Role,Power Level rhino,Norman Osborn,Guardians of the Galaxy,"Skilled combatant, Enhanced reflexes",Antihero,Low hela,James Howlett,Avengers,"Chi manipulation, Martial arts expert",Hero,Low hawkeye,Marc Spector,X-Men,Magical powers,Villain,Low </dataframe> Update this initial code: ```python # TODO: import the required dependencies import pandas as pd # Write code here # Declare result var: type (possible values "string", "number", "dataframe", "plot"). Examples: { "type": "string", "value": f"The highest salary is {highest_salary}." } or { "type": "number", "value": 125 } or { "type": "dataframe", "value": pd.DataFrame({...}) } or { "type": "plot", "value": "temp_chart.png" } ### QUERY Show the count of each type of role Variable `dfs: list[pd.DataFrame]` is already declared. At the end, declare "result" variable as a dictionary of type and value. If you are asked to plot a chart, use "matplotlib" for charts, save as png. Generate python code and return full updated code: 2025-05-04 13:39:21 [INFO] Executing Step 3: CodeGenerator 2025-05-04 13:39:23 [INFO] HTTP Request: POST https://api.groq.com/openai/v1/chat/completions "HTTP/1.1 200 OK" 2025-05-04 13:39:23 [INFO] Prompt used: <dataframe> dfs[0]:45x6 Character,Real Name,Affiliation,Powers,Role,Power Level rhino,Norman Osborn,Guardians of the Galaxy,"Skilled combatant, Enhanced reflexes",Antihero,Low hela,James Howlett,Avengers,"Chi manipulation, Martial arts expert",Hero,Low hawkeye,Marc Spector,X-Men,Magical powers,Villain,Low </dataframe> Update this initial code: ```python # TODO: import the required dependencies import pandas as pd # Write code here # Declare result var: type (possible values "string", "number", "dataframe", "plot"). Examples: { "type": "string", "value": f"The highest salary is {highest_salary}." } or { "type": "number", "value": 125 } or { "type": "dataframe", "value": pd.DataFrame({...}) } or { "type": "plot", "value": "temp_chart.png" } Generate python code and return full updated code: 2025-05-04 13:39:23 [INFO] Code generated: # TODO: import the required dependencies import pandas as pd # Count the number of each role type roles_count = dfs[0]['Role'].value_counts() # Declare result var result = { "type": "dataframe", "value": roles_count } ``` 2025-05-04 13:39:23 [INFO] Executing Step 4: CachePopulation 2025-05-04 13:39:23 [INFO] Executing Step 5: CodeCleaning 2025-05-04 13:39:23 [INFO] Code running: roles_count = dfs[0]['Role'].value_counts() result = {'type': 'dataframe', 'value': roles_count} ``` 2025-05-04 13:39:23 [INFO] Executing Step 6: CodeExecution 2025-05-04 13:39:23 [INFO] Executing Step 7: ResultValidation 2025-05-04 13:39:23 [INFO] Answer: {'type': 'dataframe', 'value': Role Hero 35 Villain 8 Antihero 2} 2025-05-04 13:39:23 [INFO] Executing Step 8: ResultParsing 2025-05-04 13:42:59 [INFO] HTTP Request: POST https://api.groq.com/openai/v1/chat/completions "HTTP/1.1 200 OK" I've followed all the instruction, but I'm stuck. My final result is that during the user query, the pandasai retrieves the docs or Q/A from local vectorstore and produce the desired output using the information retrieved. Thanks for any type of help! [url="https://stackoverflow.com/questions/79605632/pandasai-cant-use-training-data"]Continue reading...[/url]
