[Python] Python, Poetry, cant publish to Google Artifact Registry repository | 'The request...

I have a proprietary Python project that I manage using Poetry. I stored the code in the Google Artifact Registry in the past, but now I switched to a different dev environment (Arch Linux in a VM -> Windows + Ubuntu WSL) and I can't upload it the AR, while getting this error (this is on my new WSL machine):

$ poetry publish --build --repository all-mpm
There are 2 files ready for publishing. Build anyway? (yes/no) [no] yes
Building mpm (0.1.54)
- Building sdist
- Built mpm-0.1.54.tar.gz
- Building wheel
- Built mpm-0.1.54-py3-none-any.whl

Publishing mpm (0.1.54) to all-mpm
- Uploading mpm-0.1.54-py3-none-any.whl FAILED

HTTP Error 401: Unauthorized | b'The request does not have valid authentication credentials.\n'


Here's the catch: This is only happens to this repository. If I create a new repository on the same machine using the same project, I don't get an error and am able to upload my package successfully.


I believe this is a keyring issue. Listing all my keyring backends:

$ keyring --list-backends
keyrings.gauth.GooglePythonAuth (priority: 9)
keyring.backends.chainer.ChainerBackend (priority: 10)
keyring.backends.SecretService.Keyring (priority: 5)
keyring.backends.fail.Keyring (priority: 0)


I used gsutil for its creation:

gcloud artifacts repositories create my-repo \
--repository-format=python \
--location=europe-west3


After that I set for my Python project which I manage using poetry.

poetry config repository.myrepo "https://europe-west3-python.pkg.dev/my-project/my-repo/


My usual build process:

poetry publish --build --repository my-repo


If I run this command with the verbose option (-vvv), I get the following details:

$ poetry publish --build --repository all-mpm -vvv
Loading configuration file /home/til/.config/pypoetry/config.toml
Loading configuration file /home/til/.config/pypoetry/auth.toml
There are 2 files ready for publishing. Build anyway? (yes/no) [no] yes
Using virtualenv: /home/til/.cache/pypoetry/virtualenvs/mpm-W0UszHAu-py3.11
Building mpm (0.1.54)
- Building sdist
Ignoring: tests/__pycache__/test_plytix.cpython-311-pytest-7.4.2.pyc
********redacted********
- Adding: /home/til/code/aplus-automation/src/mpm/__init__.py
********redacted********
- Adding: pyproject.toml
- Adding: README.md
- Built mpm-0.1.54.tar.gz
- Building wheel
Ignoring: htmlcov/d_a44f0ac069e85531_test_plytix_py.html
********redacted********
- Adding: /home/til/code/aplus-automation/src/mpm/__init__.py
********redacted********
Skipping: /home/til/code/aplus-automation/LICENSE
Skipping: /home/til/code/aplus-automation/COPYING
- Built mpm-0.1.54-py3-none-any.whl

[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
[keyring.backend] Loading Google Auth
Found authentication information for all-mpm.
Publishing mpm (0.1.54) to all-mpm
- Uploading mpm-0.1.54-py3-none-any.whl 0%[urllib3.connectionpool] Starting new HTTPS connection (1): europe-west3-python.pkg.dev:443
- Uploading mpm-0.1.54-py3-none-any.whl 100%[urllib3.connectionpool] https://europe-west3-python.pkg.dev:443 "POST /mpm99-398708/all-mpm HTTP/1.1" 401 60
- Uploading mpm-0.1.54-py3-none-any.whl FAILED

Stack trace:

1 ~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/poetry/publishing/uploader.py:265 in _upload_file
263│ bar.display()
264│ else:
→ 265│ resp.raise_for_status()
266│ except (requests.ConnectionError, requests.HTTPError) as e:
267│ if self._io.output.is_decorated():

HTTPError

401 Client Error: Unauthorized for url: https://europe-west3-python.pkg.dev/mpm99-398708/all-mpm

at ~/.local/lib/python3.11/site-packages/requests/models.py:1021 in raise_for_status
1017│ f"{self.status_code} Server Error: {reason} for url: {self.url}"
1018│ )
1019│
1020│ if http_error_msg:
→ 1021│ raise HTTPError(http_error_msg, response=self)
1022│
1023│ def close(self):
1024│ """Releases the connection back to the pool. Once this method has been
1025│ called the underlying ``raw`` object must not be accessed again.

The following error occurred when trying to handle this error:


Stack trace:

11 ~/.local/lib/python3.11/site-packages/cleo/application.py:327 in run
325│
326│ try:
→ 327│ exit_code = self._run(io)
328│ except BrokenPipeError:
329│ # If we are piped to another process, it may close early and send a

10 ~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/poetry/console/application.py:190 in _run
188│ self._load_plugins(io)
189│
→ 190│ exit_code: int = super()._run(io)
191│ return exit_code
192│

9 ~/.local/lib/python3.11/site-packages/cleo/application.py:431 in _run
429│ io.input.interactive(interactive)
430│
→ 431│ exit_code = self._run_command(command, io)
432│ self._running_command = None
433│

8 ~/.local/lib/python3.11/site-packages/cleo/application.py:473 in _run_command
471│
472│ if error is not None:
→ 473│ raise error
474│
475│ return terminate_event.exit_code

7 ~/.local/lib/python3.11/site-packages/cleo/application.py:457 in _run_command
455│
456│ if command_event.command_should_run():
→ 457│ exit_code = command.run(io)
458│ else:
459│ exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

6 ~/.local/lib/python3.11/site-packages/cleo/commands/base_command.py:119 in run
117│ io.input.validate()
118│
→ 119│ status_code = self.execute(io)
120│
121│ if status_code is None:

5 ~/.local/lib/python3.11/site-packages/cleo/commands/command.py:62 in execute
60│
61│ try:
→ 62│ return self.handle()
63│ except KeyboardInterrupt:
64│ return 1

4 ~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/poetry/console/commands/publish.py:82 in handle
80│ )
81│
→ 82│ publisher.publish(
83│ self.option("repository"),
84│ self.option("username"),

3 ~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/poetry/publishing/publisher.py:86 in publish
84│ )
85│
→ 86│ self._uploader.upload(
87│ url,
88│ cert=resolved_cert,

2 ~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/poetry/publishing/uploader.py:107 in upload
105│
106│ try:
→ 107│ self._upload(session, url, dry_run, skip_existing)
108│ finally:
109│ session.close()

1 ~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/poetry/publishing/uploader.py:191 in _upload
189│ ) -> None:
190│ for file in self.files:
→ 191│ self._upload_file(session, url, file, dry_run, skip_existing)
192│
193│ def _upload_file(

UploadError

HTTP Error 401: Unauthorized | b'The request does not have valid authentication credentials.\n'

at ~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/poetry/publishing/uploader.py:271 in _upload_file
267│ if self._io.output.is_decorated():
268│ self._io.overwrite(
269│ f" - Uploading {file.name} FAILED"
270│ )
→ 271│ raise UploadError(e)
272│ finally:
273│ self._io.write_line("")
274│
275│ def _register(self, session: requests.Session, url: str) -> requests.Response:


Other measures I took:

• reinstallation of gsutils
• consulted the AR docs, setting up using a) ADC, b) a service account, c) .pypirc file
• added the keyring to poetry

More debug info:

Poetry plugins:

$ poetry self show plugins

• poetry-plugin-export (1.5.0) Poetry plugin to export the dependencies to various formats
1 application plugin

Dependencies
- poetry (>=1.5.0,<2.0.0)
- poetry-core (>=1.6.0,<2.0.0)


After removing .config/gcloud/ and setting up using gcloud init again (still getting the bad credentials error)

$ gcloud init
Welcome! This command will take you through the configuration of gcloud.

Your current configuration has been set to: [default]

You can skip diagnostics next time by using the following flag:
gcloud init --skip-diagnostics

Network diagnostic detects and fixes local network connection issues.
Checking network connection...done.
Reachability Check passed.
Network diagnostic passed (1/1 checks passed).

You must log in to continue. Would you like to log in (Y/n)? y

Go to the following link in your browser:

****redacted****

Enter authorization code: ****redacted****


Updates are available for some Google Cloud CLI components. To install them,
please run:
$ gcloud components update

You are logged in as: [***redacted****].

Pick cloud project to use:
****redacted****
Please enter numeric choice or text value (must exactly match list item): 3

Your current project has been set to: [****redacted****].



There is a slight difference when uploading to any other registry:

[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
[keyring.backend] Loading Google Auth
### this is different! vvvvv
[google.auth._default] Checking None for explicit credentials as part of auth process...
[google.auth._default] Checking Cloud SDK credentials as part of auth process...
[google.auth.transport.requests] Making request: POST https://oauth2.googleapis.com/token
[urllib3.connectionpool] Starting new HTTPS connection (1): oauth2.googleapis.com:443
[urllib3.connectionpool] https://oauth2.googleapis.com:443 "POST /token HTTP/1.1" 200 None
Found authentication information for all-mpm2.


I'm at a loss at what to try next.

Continue reading...