[Python] Is this correct to run a CPU-bound operation using ProcessPoolExecutor in asyncio...

I'm trying to run CPU-bound operations in non-blocking style using ProcessPoolExecutor from concurrent.futures module and asyncio.get_running_loop() functions.

I have my function that runs CPU-bound operations with polars under the hood. Second function runs first function in a ProcessPool using asyncio loop. Will this result in a bad performance or functions implemented like this one will run in separate processes and everything is fine?

def process_zip_file(file: ZipFile):
try:
csv_files = unzip_file(file)
except Exception as e:
raise ZipfileProcessingError("Error unzipping file.")

try:
dataframes = create_dataframes(csv_files) # polars operations under the hood

concat_df = concat_dataframes(dataframes, file.filename)
return concat_df
except Exception as e:
module_logging.error(e)
raise DataframeProcessingErorr("Error processing csv files.")



async def process_zip_file_concurrent(zip_file: ZipFile):
loop = asyncio.get_running_loop()
with ProcessPoolExecutor(max_workers=4) as excecutor:
result = await loop.run_in_executor(excecutor, process_zip_file, zip_file)
return result

Continue reading...