[Python] Combine multiprocessing with subprocess stdin

I am trying to populate input for my fzf sript from multiple parallel processes.

fzf = subprocess.Popen(
[
"fzf",
],
stdin=subprocess.PIPE,
text=True,
)


so I want to be able write to directly to this PIPE from other proccesses.

As multiprocessing does not support pickling IO objects I used file descriptor id. Like this:

python_process = multiprocessing.Process(
target=generate_input,
args=(fzf.stdin.fileno(),),
)


Inside my process I did write like this:

os.dup(write_fd) # I thought it could help to resolve error with fd.
os.write(write_fd, "Any string".encode("utf-8"))


This was leading to the OSError: [Errno 9] Bad file descriptor. I dont understand why I am gettings this, because all file descriptors are inherited from parent process. To ensure this, I tried to call this code inside my child process.

_fd_types = (
("REG", stat.S_ISREG),
("FIFO", stat.S_ISFIFO),
("DIR", stat.S_ISDIR),
("CHR", stat.S_ISCHR),
("BLK", stat.S_ISBLK),
("LNK", stat.S_ISLNK),
("SOCK", stat.S_ISSOCK),
)

for fd in range(100):
try:
s = os.fstat(fd)
except:
continue
for fd_type, func in _fd_types:
if func(s.st_mode):
break
else:
fd_type = str(s.st_mode)
print(f"fd: {fd}, type: {fd_type}")


This snippet printed that my file descriptor was available via os.fstat function.

I googled a lot a have not found something similar. Or at least some projects that utilize python in this way. I understand that I can approach this in other ways.
For example:

child = subprocess.Popen(
[
"my_python_program --child",
],
stdout=subprocess.PIPE,
)
fzf = subprocess.Popen(
[
"fzf",
],
stdin=child.stdout,
)


MacM1, Python3.12

But I really want to learn how to it in a more "basic" way. I will be very grateful for any info related, thank you.

Continue reading...