[Python] Snowflake - Error while creating Temp View from snowpark dataframe

Hope you are all doing well.

I am facing a weird issue in Snowpark (Python) while creating a temp view from Dataframe. I have searched online and while I have had hits, there is no proper solution.

Issue:

• I have a python list of lists containing date, directory name, file name. I am creating a dataframe from it. The dataframe is then used to create or replace a temp view.
• The code is being run on a python worksheet from within snowflake.
• I am randomly receiving error while creating the view.
  snowflake.snowpark.exceptions.SnowparkCreateViewException: (1203): Your dataframe may include DDL or DML operations. Creating a view from this DataFrame is currently not supported.
• The error occurs very randomly. But I think it occurs when the number of records increases. Just a guess since the same process works for me if I run with less than 200 records but with more records it fails. Maybe just a co-incidence.

Approaches Tried:

• Have removed the part where I create additional columns.
• Have tried creating a copy of the dataframe using df.select("*") and then using that to create the view.
• Have tried caching the dataframe before creating the view.

Actual Code:

# assume the list is being created in a different process.
cur_run_files_list.append((date, dir_name, file_name))
tst_vw_nm_fq = "DB_NAME.SCHEMA_NAME.TEST_VIEW"
# --------------------------------------------------------------------------------
df_cur_run_files_list = session.create_dataframe(
cur_run_files_list,
schema=["EXP_DATE", "DIR_NAME", "FILE_NAME"],
)
df_cur_run_files_list.create_or_replace_temp_view(tst_vw_nm_fq)



Test Code - As requested: Note: This code fails intermittently only - values hard coded for testing...

# assume the list is being created in a different process.
# cur_run_files_list.append((date, dir_name, file_name))
cur_run_files_list = [
("2024-12-01", "dir_name01", "file_name01.txt"),
("2024-12-01", "dir_name02", "file_name02.txt"),
("2024-12-01", "dir_name03", "file_name03.txt"),
("2024-12-01", "dir_name04", "file_name04.txt"),
("2024-12-01", "dir_name05", "file_name05.txt"),
("2024-12-01", "dir_name06", "file_name06.txt"),
("2024-12-01", "dir_name07", "file_name07.txt"),
("2024-12-01", "dir_name08", "file_name08.txt"),
("2024-12-01", "dir_name09", "file_name09.txt"),
("2024-12-01", "dir_name10", "file_name10.txt"),
]
tst_vw_nm_fq = "DB_NAME.SCHEMA_NAME.TEST_VIEW"
# --------------------------------------------------------------------------------
df_cur_run_files_list = session.create_dataframe(
cur_run_files_list,
schema=["EXP_DATE", "DIR_NAME", "FILE_NAME"],
)
df_cur_run_files_list.create_or_replace_temp_view(tst_vw_nm_fq)
return df_cur_run_files_list # Added since snowpark needs a return value

Continue reading...