[Python] Can't find local python package in parallel directory when using Prefect + Docker

I have setup a Prefect deployment on a Docker image. The problem is that my flow package depends on a local package which is in a parallel directory:

main_folder
|_ my_flow
|_ src
|_ my_flow
|_ source code the depends on flow_dep...
|_ flow_dep
|_ src
|_ flow_dep
|_ source code...


On my local machine, this works perfectly. The problem is that I can't find a way to have this setup work in my Docker container: when deploying, I download the code from S3 and I pip install flow_dep as editable. The thing is, the ONLY way to have this work is to manually add /flow_dep/src to sys.path (something that on my local machine is done automatically by pip). If I modify PYTHONPATH adding flow_dep/src it DOES NOT WORK. I cannot wrap my head around this. I have printed sys.path and it's identical in both cases, but if modified directly, the flow_dep is found; if modified through PYTHONPATH, flow_dep is NOT found. Again, the sys.path is identical in both cases when printed. What is going on?

Continue reading...