[Python] GCP Cloud Functions - How to import local scripts outside of cloud function? How to...

I have one giant repo folder which contains many more sub-repos, and each sub-repo contains multiple cloud functions, i.e.

|-repo_master
|-repo_a
|-cloud_function_a
|-.github/workflows/deploy.yml
|-main.py
|-requirements.txt
|-config.py
|-cloud_function_b
|-.github/workflows/deploy.yml
|-main.py
|-requirements.txt
|-config.py
|-repo_b
|-cloud_function_a
|-.github/workflows/deploy.yml
|-main.py
|-requirements.txt
|-config.py
|-cloud_function_b
|-.github/workflows/deploy.yml
|-main.py
|-requirements.txt
|-config.py
|-repo_c
|-cloud_function_a
|-.github/workflows/deploy.yml
|-main.py
|-requirements.txt
|-config.py
|-cloud_function_b
|-.github/workflows/deploy.yml
|-main.py
|-requirements.txt
|-config.py
etc...


each cloud function has a config.py file and I would like to move this config.py file into the repo_master directory and have all the cloud functions refer to it.

I am able to achieve this locally by adding the repo_master to sys.path, for instance, my python files (main.py) look like this.

# Import config file
import os
import sys
sys.path.append(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
from config import CONFIG_VARIABLE


essentially by adding the repo_master dir to the sys.path I can access CONFIG_VARIABLE in all of my main.py files.

The issue is when I deploy this to github's workflow and deploy the cloud function I get the error in GCP

ImportError: cannot import name 'CONFIG_VARIABLE' from 'config' (unknown location)


My cloud function deployment looks like this (with variables defined in deploy.yml env)

gcloud functions deploy ${{ matrix.function.name }} \
--gen2 \
--trigger-http \
--source=cloud_functions/${{ matrix.function.name }} \
--runtime=${{ env.RUNTIME }} \
--region=${{ env.REGION }} \
--memory=${{ env.MEMORY }} \
--service-account=${{ env.WORKFLOWS_SA }} \
--timeout=${{ env.TIMEOUT }} \
--no-allow-unauthenticated \
--set-secrets="GOOGLE_CLOUD_PROJECT=GOOGLE_CLOUD_PROJECT:latest,GOOGLE_CLOUD_PROJECT_NUMBER=GOOGLE_CLOUD_PROJECT_NUMBER:latest"


Potential Solutions


I have thought about having a custom git push bash script which copies the config into all the cloud function directories, however I would prefer to avoid this and I would like to know if there is a better way to access these global variables/scripts or if there is a better way to structure my entire project

Continue reading...