[Python] Cloud Run Fails to Import Custom Package from Artifact Registry (Terraform Deployment)

I'm deploying a Cloud Run function using Terraform, and the function depends on a custom Python package that we uploaded to Google Artifact Registry using Twine. However, the deployment fails with the following error:ModuleNotFoundError: No module named 'google.cloud.logging'

Traceback (most recent call last):
File "/layers/google.python.pip/pip/bin/functions-framework", line 8, in <module>
sys.exit(_cli())
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/click/core.py", line 1161, in __call__
return self.main(*args, **kwargs)
...
File "/workspace/main.py", line 1, in <module>
from ddsgcloud.utils import cloud_helpers as ch
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/ddsgcloud/utils/cloud_helpers.py", line 1, in <module>
import google.cloud.logging
ModuleNotFoundError: No module named 'google.cloud.logging'


Setup: The function runs Python 3.9 and is deployed using Terraform. The custom package is stored in Artifact Registry (ddsgbq is the repository, not the package name).

I uploaded the package using twine:

twine upload --repository-url https://us-central1-python.pkg.dev/dev-di-dashboards-data/ddsgbq/ dist/*


requirements.txt contains:

###### core dependencies
click
google-api-core
google-auth
google-cloud-appengine-logging
google-cloud-audit-log
google-cloud-bigquery
google-cloud-bigquery-storage
google-cloud-logging
google-cloud-core
pandas
pandas-gbq
ipython
jupyter
jupyterlab
jupyterlab-git
pyarrow
numpy
twine
keyrings.google-artifactregistry-auth
tabulate
google
itables
protobuf
google-cloud-logging
google-cloud-pubsub
jira
python-dateutil


Terraform deployment:

resource "google_cloudfunctions2_function" "cloud_function_v2" {
name = "dds-inovapp"
description = "Function for inovapp operations"
runtime = "python39"
service_config {
service_account_email = "dss-ops-1@dev-di-dashboards-data.iam.gserviceaccount.com"
}
}


What I Suspect:

• The custom package is not being installed correctly from Artifact Registry, causing dependencies like google.cloud.logging to be missing.
• There might be an authentication issue preventing Cloud Run from pulling private packages.

What I’ve Tried:

• Ensured requirements.txt correctly references the package from Artifact Registry.
• Verified service account has Artifact Registry Reader permissions.
• Deployed with gcloud builds submit --no-cache to force a fresh dependency install.
• Checked Cloud Build logs—it seems the package is not being installed.
• Added an explicit pip install command in the entry script.

Questions:

• How can I verify if the custom package is actually being installed during deployment?
• How do I ensure Cloud Run has the correct permissions to install private packages from Artifact Registry?
• Is there a way to debug why dependencies might be missing inside Cloud Run? Any insights would be greatly appreciated!

Continue reading...