[Python] Python threads killed when main thread times out with only one thread, but survive...

I have a Python function decorated with a 1-minute timeout. Inside it I spawn one or more worker threads that each run a 2-minute sleep via a dynamically executed script (using exec() in a custom executor). Here’s a little part of code:

def _execute_parallel(self, input_params, connector_init_params, code, task):
order = -1
threads = []
output_queue = []
while input_params:
ipparam = input_params.pop(0)
order = order + 1

th = threading.Thread(target=self._execute, args=(connector_init_params, code, task, ipparam, order, output_queue,))
th.start()
threads.append(th)

for th in threads:
th.join()

return output_queue


Actual behavior:

If num_workers = 1, the parent hits the 60-second timeout, and the single worker thread is also killed before its 2-minute sleep completes. If num_workers = 2, the parent still times out at 60 s, but both worker threads survive the parent’s timeout and complete their 120 s sleep, printing their “finished sleep” messages. Expected behavior: Child threads should continue running to completion regardless of how many are created, since none are daemon threads. Why does one child get killed when there’s only a single worker, but survive when there are two?

Environment:

Python 3.9 on macOS Using a @timeout decorator (raises an exception in the main thread after 60 s) Custom executor uses exec() under the hood, but the same behavior occurs in this simplified example. What’s causing this difference in thread lifetimes, and how can I ensure worker threads always outlive the timed-out parent?

Continue reading...