[Python] Poetry looks for packages in global PYTHONPATH first instead of in poetry-managed...

On Windows, I have a poetry project which references Python 3.12, and includes dependencies to package A and package B. Poetry was added to my system using pipx in the context of Python 3.12.

I also have Python 3.8 on my machine with package A installed in the global environment via pip install. This is because I have a python add-in in another piece of software which requires Python 3.8. This add-in also requires that I have a system environment variable PYTHONPATH which references the location of my 3.8 packages (usr\AppData\Local\Programs\Python\Python38\Lib\site-packages) so that it can find where to search for packages.

Anyhow, when I try to run a script in my poetry project, I get an import error on "import package A" because it is evidently trying to import first from the Python38 global library rather than the poetry-managed VE, and since package A happened to be installed in the Python38 global library, it uses that version which is not compatible. It still manages to find package B without any trouble, even though package B is only installed in the poetry VE.

If I remove the PYTHONPATH system env variable, everything runs as expected - it uses only the packages it finds in the poetry VE.

Why does Poetry look first in the location of PYTHONPATH rather than its VE? How can I prevent this behavior? Or is there something else I am missing?

I searched around and have not found this specific problem mentioned anywhere.

Step-by-step to reproduce this:

1. Install Python 3.12, and install poetry using pipx per standard poetry installation approach.
2. Install Python 3.8, and add a system environment variable:

PYTHONPATH = C:\Users\username\AppData\Local\Programs\Python\Python38\Lib\site-packages

1. In context of Python 3.8 global environment, pip install numpy.
2. Create a poetry project in context of python 3.12 poetry new testProject.
3. In new project (cd testProject), add numpy and keyboard: poetry add numpy keyboard.
4. To testProject main folder, add file 'test.py' with contents:

import keyboard
import numpy

1. Do poetry run test.py; keyboard imports fine, but on import numpy you receive (partial) output below:

PS D:\path\to\poetry\project\testProject> poetry run .\test.py
Traceback (most recent call last):
File "C:\Users\######\AppData\Local\Programs\Python\Python38\Lib\site-packages\numpy\core\__init__.py", line 23, in <module>
from . import multiarray
File "C:\Users\######\AppData\Local\Programs\Python\Python38\Lib\site-packages\numpy\core\multiarray.py", line 10, in <module>
from . import overrides
File "C:\Users\######\AppData\Local\Programs\Python\Python38\Lib\site-packages\numpy\core\overrides.py", line 6, in <module>
from numpy.core._multiarray_umath import (
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'

1. As can be seen, poetry attempts to run numpy from the global Python 3.8 site-packages folder.
2. Remove system environment variable PYTHONPATH created above, and in new power shell window, run poetry run test.py again, with no error.

Continue reading...