I am trying to write a Python script to run another potentially multithreaded script in a way to further parallelise them. Consider a (C++) executable which can be run as run_task <sample_number>, which spawns 10 threads. The sample numbers are in a Python list. What is the ideal way to execute them in a Python script, such that at any time, three executables are always running? P.S. Previously, I split the total number of samples into three bash scripts and used to run all three scripts simultaneously by using a tmux session with three panes. Hoping to do it in a better way. Continue reading...
