[Python] Sourcing shell file from Python

I am working on a code which does multiprocessing. For this, I want to source different files accordingly. I want to have the sourcing inside my Python file as I am not very keen on bash. I tried all the solutions outlined in the following post, but nothing worked for me. Before running the Python file I source the file such that echo $DETECTOR_PATH returns /data/tomble/eic/epic/install/share/epic. Now, I run the Python file and I copy the directory which holds the sh file:

def copy_epic(self, curr_px_path):
# copy epic to respective px folder for parameter reference
os.system(f'cp -r {self.det_dir} {curr_px_path}')
return os.path.join(curr_px_path, "epic")


and then I run a loop changing curr_px_path:

# copy epic and compact folders (os.path.join(curr_px_path, "epic"))
curr_px_epic_path = self.copy_epic(curr_px_path)

print(f"curr_px_epic_path: {curr_px_epic_path}")

# source current detector
self.source_px_epic(curr_px_epic_path + "/install/bin/thisepic.sh")

print(f"os.environ['DETECTOR_PATH']: {os.environ['DETECTOR_PATH']}")


Using one of the solutions suggested in the post:

def source_px_epic(self, current_px_epic_sh_path):
command = shlex.split("bash -c f'source {current_px_epic_sh_path} && env'")
proc = subprocess.Popen(command, stdout = subprocess.PIPE)
for line in proc.stdout:
(key, _, value) = line.partition("=")
os.environ[key] = value
proc.communicate()

pprint.pprint(dict(os.environ))


But my prints suggest that this is not working:

curr_px_epic_path: /data/tomble/Analysis_epic_new/simulations/simEvents/2.0x0.1px/epic
os.environ['DETECTOR_PATH']: /data/tomble/eic/epic/install/share/epic


Since I copied the folder with the shell file, I wish to get os.environ['DETECTOR_PATH']: /data/tomble/Analysis_epic_new/simulations/simEvents/2.0x0.1px/epic after sourcing.

I tried all the Python sourcing functions in the post I linked above. I am running thew file while in a singularity environment, and am running Ubuntu 24.04.01. What am I doing wrong?

Continue reading...