[Python] subprocess.Popen: how to proper communicate in an interactive way?

I'm working on an app that requires a conversational interaction with an external process, simulated here by the terminal calculator bc: I send something to stdin, and I get the response via stdout, or stderr if an error occurs. However, the proc quits when a non-fatal error occurs.

Below is a snippet ztui.py to replicate the issue. Instead of moving on to the next stdin input, the proc just exits unexpectedly after printing to stderr, and the python program experiences the BrokenPipeError when it tries to send/flush the 3rd command as that is obviously closed.

from subprocess import Popen, PIPE

proc = Popen(["bc"], stdin=PIPE, stdout=PIPE, stderr=PIPE)

proc.stdin.write(b"2+2\n")
proc.stdin.flush()
print(proc.stdout.readline().decode())

proc.stdin.write(b"len()\n")
proc.stdin.flush()
print(proc.stdout.readline().decode())
print("stderr ==>", proc.stderr.readlines(2))

# and here the proc exits, checked using ps aux from another terminal window

proc.stdin.write(b"3+3\n")
proc.stdin.flush()
print(proc.stdout.readline().decode())


This is the output

fabio: ~/Downloads $ python3.11 --version
Python 3.11.3
fabio: ~/Downloads $ python3.11 ztui.py
4


stderr ==> [b'\n', b'Runtime error: undefined function: len()\n']
Traceback (most recent call last):
File "/Users/fabio/Downloads/ztui.py", line 15, in <module>
proc.stdin.flush()
BrokenPipeError: [Errno 32] Broken pipe


Needless to say, the actual bc program does not exit when you pass len(): it just shows the error and gets ready for the new input

fabio: ~/Downloads $ bc
>>> 2*2
4
>>> len()

Runtime error: undefined function: len()
0: (main)

>>> 3*3
9
>>> quit


How can I troubleshoot that? I had initially looked into the communicate() command, but that is for a one-off request, and I need to continuously send and receive commands and responses.

Continue reading...