[Python] subprocess and exchanging json: How can I use read() on stdin non-blockingly?

I have a main process, that sends json structured data to a subprocess. The subprocess is working with this data and giving back information on the progress in a percentage to the main process (which shall update a progress bar in the user interface).

The problem is, that the output of the subprocess is only received by the main process when the subprocess is already finished. It blocks on the read()-Statement I suppose. How can I get the main process to work with the response, as soon as child process posts a line to its stdout?

Here's the minimal working example:

parent.py

from json import dumps
import subprocess
from time import sleep

lines_to_exchange = ["this is line one", "this is line two", "this is line three", "this is line four", "this is line five"]
command = ["python", "-u", "./child.py"]
print("start process")


sub = subprocess.Popen(command, text=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE)

sub.stdin.write(dumps(lines_to_exchange))
sub.stdin.close()


while True:
sleep(0.1)
stdout = sub.stdout.read()
print(stdout)

if sub.poll() is not None:
print("process completed")
break


child.py

from time import sleep
from json import loads
import sys
lines = loads(input())

for line in lines:
sleep(1)
print(line)
sys.stdout.flush()


I am working on Windows with python 3.10 and pycharm IDE.

Continue reading...