[Python] numpy.load() seems to use double the memory of the array at peak

I thought numpy.load() writes directly into the array data memory.

However, the results of the profiling functions (I profiled with memray and mprof) seems a bit strange to me...

My array is 2GB big. I would have expected a peak memory of 2GB for the loading function. I could also imagine, that 4GB peak-wise used as the array might be loaded into a buffer and than written to the data memory.
Nevertheless, the timing in the profiling seems odd. After loading the program should wait for a second and print the time. Nevertheless, the memory size increases after this one second?

I sampled at 0.001s for mprof and 0.1ms for memray.

import time
from datetime import datetime as dt

import memray
import numpy as np

def main():
time.sleep(1)
print(dt.now())
m_ = np.load(
os.path.join(os.path.expanduser('~'), 'test_data/max_compression_chunk.npy'),
mmap_mode=None,
)
print(dt.now())
time.sleep(1)
print(dt.now())

if __name__ == "__main__":
with memray.Tracker("memray_test.bin", memory_interval_ms=0.1, follow_fork=True, native_traces=True):
main()


Attached are also the mprof and memray results:





Could anyone explain to me, what is going on? Or give me a hint what to check next?

Continue reading...