[Python] Google Cloud and project ID from google.auth.default() in docker container

I'm playing Whac-A-Mole with Google Cloud authentication. Sorry for the lengthy question but since I have no idea where the problem is, I'm trying to give enough context.

With Python, calling google.auth.default(), I can retrieve the ID of my Google Cloud project outside of a docker container but not within.

I've set my project:

$ gcloud config set project my-project

Updated property [core/project].


I've set the "Quota project":

$ gcloud auth application-default set-quota-project my-project

Credentials saved to file: [/home/.../.config/gcloud/application_default_credentials.json]

These credentials will be used by any library that requests Application Default Credentials (ADC).

Quota project "my-project" was added to ADC which can be used by Google client libraries for billing and quota. Note that some services may still bill the project owning the resource.


I've set the "billing/quota_project":

$ gcloud config set billing/quota_project my-project

Updated property [billing/quota_project].


Then, in the docs for a containerized development environment, it says:

To test your containerized application on your local workstation, you can configure your container to authenticate with your local ADC file. For more information, see Configure ADC with your Google Account.
​

I'm testing my containerized application on my local workstation, so I've created local authentication credentials for my user account:

$ gcloud auth application-default login

Your browser has been opened to visit:
...

Credentials saved to file: [/home/.../.config/gcloud/application_default_credentials.json]

These credentials will be used by any library that requests Application Default Credentials (ADC).

Quota project "chatbot-smart" was added to ADC which can be used by Google client libraries for billing and quota. Note that some services may still bill the project owning the resource.


Also, based in this SO answer, since I'm using docker-compose, I added the config folder to the yaml:

volumes:
- ~/.config/gcloud:/root/.config/gcloud


Now, based on examples:

The simplest way to use credentials from a user account is via Application Default Credentials using gcloud auth application-default login (as mentioned above) and google.auth.default():
​

import google.auth

credentials, project = google.auth.default()


Outside of a container, I get both the credentials and project id, but inside a container this returns None for the project id, like documented here:

Project ID may be None, which indicates that the Project ID could not be ascertained from the environment.
​

The question is, how can I get the Project ID ascertained from the environment inside a docker container?

I found this Github issue with the same problem, but the solution leads eventually to the Google documentation, where I've already been wandering for some time:

On a second read, I'm don't think this particular flow is supported for getting a default project.

Please see https://hub.docker.com/r/google/cloud-sdk/ and https://github.com/GoogleCloudPlatform/cloud-sdk-docker for authenticating inside a docker image.

Thanks!
​

Continue reading...