[Python] Memory usage accumulates in simple loop until PC crashes

I have recently experienced some major crashes on my computer after executing some Python code of mine. My PC would freeze and sometimes show a bluescreen after a few rounds of a seemingly unproblematic loop. I did some investigation and realised that while executing the loop, memory usage would accumulate for no obvious reason. I am using JupyterLab. Below you will find a MWE of what reproduces the problem in (my) JupyterLab.

import numpy as np
import gmpy2 as gp
import psutil, os # To keep track of memory usage.


# Calculate some matrix with gmpy2 for higher precision.
# It seems that the cosine here is essential for the memory accumuluation.
def get_matrix() :
n = 2**8-1
mat = np.zeros( (n, n), dtype=object )
for i in range(n) :
for j in range(n) :
mat[i,j] = gp.cos( i*j )
return mat


# I want to keep track of memory usage so I use this.
process = psutil.Process(os.getpid())


# Compute matrix but do not save and keep track of memory usage.
for _ in range(20) :
with gp.context(gp.get_context()) as context :
context.precision = 123
get_matrix()
memory_used_in_MB = process.memory_info().rss*1e-6
print(f'Memory used: {memory_used_in_MB}')


My output reads.

Memory used: 207.40096
Memory used: 209.77254399999998
Memory used: 211.861504
Memory used: 213.95046399999998
Memory used: 216.04761599999998
Memory used: 218.206208
Memory used: 220.295168
Memory used: 222.384128
Memory used: 224.477184
Memory used: 226.56614399999998
Memory used: 228.655104
Memory used: 230.752256
Memory used: 232.841216
Memory used: 234.930176
Memory used: 237.019136
Memory used: 240.18944
Memory used: 243.08121599999998
Memory used: 245.448704
Memory used: 247.894016
Memory used: 250.261504


This is odd as I do not save any of the matrices I generate. It seems that they are stored internally for some reason and at some place but I do not know why and how to avoid this. It is really annoying as I naturally want to avoid regular crashes of my PC when doing simple loops.

Continue reading...