[Python] How to profile code where the time is spent in imap_unordered?

This code (it will only work in Linux) makes a 100MB numpy array and then runs imap_unordered where it in fact does no computation.

It runs slowly and consistently. It outputs a . each time the square function is called and each takes roughly the same amount of time. How can I profile what it is doing?

from multiprocessing import Pool
import numpy as np
from pympler.asizeof import asizeof


class ParallelProcessor:
def __init__(self, num_processes=None):
self.vals = np.random.random((3536, 3536))
print("Size of array in bytes", asizeof(self.vals))

def _square(self, x):
print(".", end="", flush=True)
return x * x

def process(self, data):
"""
Processes the data in parallel using the square method.
aram data: An iterable of items to be squared.
:return: A list of squared results.
"""
with Pool(1) as pool:
for result in pool.imap_unordered(self._square, data):
# print(result)
pass

if __name__ == "__main__":
# Create an instance of the ParallelProcessor
processor = ParallelProcessor()

# Input data
data = range(1000)

# Run the processing in parallel
processor.process(data)

Continue reading...