[Python] Cloudera Data Science Workbench Not Using Virtual Environment's Python

Question:
I am working in Cloudera Data Science Workbench (CDSW) and have created a virtual environment named "testenv". I started a session and activated my virtual environment using:

source testenv/bin/activate


After activation, when I run:

which python


I get the expected output:

/home/cdsw/testenv/bin/python


However, when I check the Python executable inside the workbench using:

import sys
print(sys.executable)


I get:

/usr/local/bin/python3.10


This indicates that the workbench is still using the system Python instead of the virtual environment’s Python.

What I’ve Tried:

• Ensuring the virtual environment is activated before running Python commands.
• Restarting the CDSW session and reactivating the virtual environment.
• Checking if the virtual environment is correctly created and contains the necessary dependencies.

Question:


How can I configure Cloudera Data Science Workbench to use the Python executable from my virtual environment (testenv) instead of the system Python?

Any guidance would be greatly appreciated!

Continue reading...