I am calling an executable file, written in C, via the Popen constructor in Python to read what the executable normally outputs to the console/terminal. The printf in the C code terminate with \n regularly and it continously outputs to the terminal until it is interrupted by e.g. a keyboard interrupt. The script and the executable run on a Raspberry Pi. I am using the following Python code for this, following numerous tutorials on retrieving the subprocess's output/prints to console in real time: import subprocess import time proc = subprocess.Popen(["PATH TO EXECUTABLE"],stdout=subprocess.PIPE) while True: start_time = time.time() line = proc.stdout.readline() print(line) print(time.time()-start.time) I measured the time for debugging purposes. For some reason, the first iteration of the loop takes 20 to 30 seconds to complete due to line = proc.stdout.readline() (I know that because everytime I interrupt the script, it interrupts at this line). Subsequent iterations take a few microseconds, but every 40th or so iteration exhibits the same behavior as the first iteration again (20 to 30 seconds to do line = proc.stdout.readline()). It seems that reading stdout takes a long time, possibly because some buffer is being filled before being let out to stdout? I have tried setting the buffersize of the Popen-object, used read() etc., but to no avail. I do not think that the executable somehow buffers because a newline normally flushes the buffer in C. I do not know if that is a problem with the C code or the Python script. My knowledge especially on the inner works and the low-level functionalities of Python and C is rather limited, that I appreciate any suggestions! Continue reading...
