I have some code that cycles through some files and then makes subsequent calls to a "chain" of multiple APIs using contents from each file, where each API needs the output of the previous one to run. files = [file_1.png, file_2.png, file_3.png] for file in files: out_1 = call_api_1(file) out_2 = call_api_2(file, out_1) ... out_n = call_api_n(file, out_nminus1) save_final_output(out_n) How can I run this code in parallel by not only parallelizing the loop on the files which are read, but also so that each subsequent API call is run in parallel and only when the output from the previous one is ready for each different file? For example, I would like to start calling api_1 on each file in parallel, and while this is running, start calling api_2 only on the contents processed by api_1, and so on. Continue reading...
