[Python] Python subprocess.popen deadlocking if manipulating stdin and stdout?

I wrote a script to manipulate stdin and stdout for a chess engine. Basically I open the program with subprocess.popen and manipulate input and output using threading.Thread for each. However, if several instances of my script are spawned, the program stops working after like 15 minutes. I presume that is the deadlock due to a full buffer described by the documentation. Unfortunately, I cannot use the recommended "communicate" function, because I need to modify both input and output continuously. Is there any way how to deal with that? Here's a minimal version of what I do:

import sys
import time
import subprocess
import threading


class ChessEngine:

def __init__(self):
self.lock = threading.Lock()

def write(self, command):
with self.lock:
self.engine.stdin.write(command + "\n")
self.engine.stdin.flush()

def read_stdout(self):
while True:
text = self.engine.stdout.readline()
text = text.replace("foo", "bar")
print(text, end="", flush=True)

def read_stdin(self):
while True:
command = input()
command = command.replace("foo", "bar")
self.write(command)

def run(self):
self.engine = subprocess.Popen(
"/usr/games/stockfish",
shell=True,
universal_newlines=True,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
bufsize=1,
cwd=self.folder,
)
threading.Thread(target=self.read_stdout, daemon=True).start()
threading.Thread(target=self.read_stdin, daemon=True).start()
while True:
time.sleep(1)
if not self.engine.poll() is None:
sys.exit(0)


if __name__ == "__main__":
chess_engine = ChessEngine()
chess_engine.run()



I looked for several ways how to make sure the script is not blocking - without success.

Continue reading...