[Python] How to avoid memory error for large dataarray

In Python I have a large xarray dataarray (da) of the shape

In [1]: da.shape
Out[1]: (744, 24, 30, 131, 215)


I need to perform the operation da = 10**da however, I'm running into a memory error

MemoryError: Unable to allocate 56.2 GiB for an array with shape (744, 24, 30, 131, 215) and data type float32
​

Is there a way to do the operation iteratively? I tried to manually loop but that gives the same error.

for ii in range(len(da[:,0,0,0,0])):
da[ii,:,:,:,:] = 10**da[ii,:,:,:,:]


It works fine with smaller arrays e.g., (24, 24, 30, 131, 215).

Continue reading...