[Python] Why does my Python program not finish cleanly when more than one thread runs...

I have written a Python GUI for a media player (a shared library I created from ffplay in ffmpeg). It uses replay gain to adjust volume so that most music sounds about equally loud. I was not able to find any shared library that calculates replay gain so I run ffmpeg as a subprocess to return replay gain via a pipe through its stderr output.

I run an ffmpeg subprocess for each music file in a separate thread. I use threads because running the subprocess commands from the main GUI causes short gaps or hiccups in the playback of files (the player also runs in a thread). What I have discovered is that if more than one of the threads for ffmpeg runs concurrently, the Python GUI will not shutdown cleanly.

I should explain what I mean by not shutting down cleanly. I use Geany for my IDE; it runs the program in a terminal window. When everything terminates as it should, messages appear in the window "(program exited with code: 0) Press return to continue." By pressing Enter, the window closes immediately.

What I observe instead is that when more than one of the threads runs concurrently, the same messages appear but pressing Enter will not close the window. I must either close it by clicking on the X in the upper left corner or pressing ctrl+C.

I use a timer to create and start threads. On my system, most threads complete in about 2 sec. By setting the timer interval to 3 sec or more, everything shuts down properly. If I shorten the interval to about 2 sec or less, the problem I describe above occurs. I currently have the timer set to 100 msec. For a variety of reasons, I do not what to use an interval longer than that.

I have pared the GUI down to a minimal example. This example does not have the player. It is just a widget that starts a sequence of threads that each run ffmpeg in a subprocess.

The example code is shown below. I am running Python 3.12, PyQt 5.15 on Linux Mint 22.1. Just in case it matters, I am using ffmpeg 7.1.

import sys, subprocess, queue, threading
from PyQt5 import QtWidgets, QtGui, QtCore

class AudioPlayerDialog(QtWidgets.QDialog):
def __init__(self, mediaList):
super().__init__()
vbox = QtWidgets.QVBoxLayout()
vbox.addWidget(QtWidgets.QLabel('WIDGET'))
self.setLayout(vbox)
self.calcGainTimer = None
self.gdCalcThreads = None
self.finished.connect(self.closeEvent)

self.mediaList = mediaList
self.mediaListLen = len(mediaList)
self.normGainList = [-1. for x in mediaList]
self.buildNormGainList()

def closeEvent(self, event):
print('**** close event')
if self.calcGainTimer is not None:
print('**** stopping calcGainTimer')
self.calcGainTimer.stop()
if self.gdCalcThreads is not None:
print('**** checking gain calc threads')
for idx, t in self.gdCalcThreads.copy().items():
if t.is_alive():
print('**** waiting for thread {} to finish'.format(idx))
t.join()
print('**** closing ...')

def buildNormGainList(self):
self.calcGainQueue = queue.LifoQueue(maxsize=self.mediaListLen + 10)
for i in range(self.mediaListLen-1, -1, -1):
self.calcGainQueue.put_nowait(i)
# setup gain/dur calculation thread dictionary
self.gdCalcThreads = dict()
# For each expiration of the following timer, a thread will be
# started to calculate the replay gain for a particular file
# and a check will determine if all files have been processed.
print('@@@@ starting updt timer')
self.calcGainTimer = QtCore.QTimer()
self.calcGainTimer.timeout.connect(self.stopGainDurCalcWhenDone)
self.calcGainTimer.setInterval(100) # interval in msec.
self.calcGainTimer.start()

def stopGainDurCalcWhenDone(self):
# this is the target of the updt timer
try:
# start a thread if the queue is not empty
idx = self.calcGainQueue.get_nowait()
if idx not in self.gdCalcThreads:
print('@@@@ creating thread for', idx)
thread = threading.Thread(None, self.requestGain, args=(idx,))
self.gdCalcThreads[idx] = thread
thread.start()
except queue.Empty:
pass
except Exception as err:
print('**** ERROR: get request from calcGainQueue failed.\n\t'\
'{}'.format(err))
# check if any thread finished but updates not yet finished
for idx,t in self.gdCalcThreads.items():
if not t.is_alive() and self.normGainList[idx] < 0.:
# Thread finished; update not yet completed. Invert the
# sign of the gain, which indicates update completed.
self.normGainList[idx] = -self.normGainList[idx]
print('@@@@ update finished for', idx)

# check if all files have been processed; stop updt timer if so
done = True # assume all calc done
for i in range(self.mediaListLen):
if self.normGainList < 0.:
done = False
break
if done:
print('@@@@ all calc are done -----------------------')
# all files have been processed
print('@@@@ stopping updt timer')
self.calcGainTimer.stop()
self.calcGainTimer = None
self.gdCalcThreads = None

TEST = False # set True to pass dummy filename of ffmpeg

def requestGain(self, idx):
print('@@@@ start subprocess for', idx)
if self.TEST:
fileName = 'XXXX'
else:
fileName = self.mediaList[idx]
cmd = ['ffmpeg', '-i', '{}'.format(fileName), '-hide_banner',
'-af', 'replaygain', '-f', 'null', '-']
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
try:
proc.wait(timeout=30)
_,err = proc.communicate()
text = err.decode('utf8')
except:
proc.kill()
text = ''
# actual program parses gain from text
gain = 1.
self.normGainList[idx] = -gain # save negative gain


if __name__ == '__main__':

app = QtWidgets.QApplication(sys.argv)

# put full-qualified filenames of media in list
mediaList = [
]

apd = AudioPlayerDialog(mediaList)
apd.exec()

print('\nFINI')
sys.exit()


I tried running the subprocess without a PIPE. That did not make a difference.

A variable named TEST appears immediately above the requestGain() routine. By setting it True, a dummy filename is passed to ffmpeg. This causes the program to terminate quickly. Nothing useful is done but it prevents two threads from running concurrently and demonstrates the program will shutdown cleanly.

As you may see, the problem occurs even though all threads have completed.

It may be important to point out again that the the player is running in a threading.Thread instance. The player thread plus a single thread for the ffmpeg subprocess does not cause a problem. Two ffmpeg subprocess threads alone (without the player) cause the problem.

I must be doing something wrong but I cannot see what it is. Perhaps you can see it?

Continue reading...