[Python] Long Running Python Program - Memory Usage Increasing Indefinitely

I have a simple Python program designed to continuously plot data from a frequently-updated csv file, which is intended to run for months at a time. A simplified version of the program can be found below (there are additional pandas transformations done on the dataframe in my actual program than what I'm showing here, but this is the essence of the program):

import pandas
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation

fig, ax = plt.subplots()
x = np.arange(5) # x-axis: 5 bars
bars = ax.bar(x, np.zeros_like(x))

ax.set_ylim(0, 100)
ax.set_title("Real-Time Bar Chart")

def get_data():
df = pandas.read_csv('data/ztest_data.csv', header=None) # roughly 5.39 MB CSV file with 3 columns: string, integer, integer
df.columns = ['unit', 'value', 'selection']
df = df.loc[df['selection'] <= 5]
df = df.groupby(['selection']).agg(avg_value=('value','mean')).reset_index()
df['avg_value']/= 100000000
return df

def update(frame):
df = get_data()
new_heights = np.random.randint(0, 50, size=5) + np.array(df['avg_value'])
for bar, height in zip(bars, new_heights):
bar.set_height(height)
return bars

ani = FuncAnimation(fig, update, interval=1000*5, blit=False, cache_frame_data=False)

plt.show()


I noticed that the program's memory usage continues to increase indefinitely, but I don't know why. Using gc.collect() at the end of the update() function doesn't resolve the issue. Here's an example of the memory usage as reported by Windows task manager after running for a few days:



Considering I intend to leave this program running for months at a time, this is not ideal, although not immediately problematic. Am I doing anything wrong here? If not, do you have any recommendations for what I should do?

I'm on Python 3.11.1 with Pandas version 1.5.2.

Continue reading...