[Python] pip-install Python Package from Azure Artefact on a Databricks Cluster

I have a small demo package that I've published my to Azure Devops - I'm able to pip install this locally by spinning up a virtual environment and adding the specific global setting to my pip.conf file.

Now, I want to install this into my databricks cluster - I've been unable to find any documentation to do so.

Things to note, this databricks cluster is not within the same tenant as my devops. I've created a token and given it access to read packages within my devops.

I assume I need to add my token & username somewhere in my artifact index url and install it in the cluster PyPi area?

however I can't figure this out

pip install mypackage==0.01 https://tokenname:tokenpassword@pkgs.dev.azure.com/myorg/mypackage/_packaging/mypackage/pypi/simple/




Continue reading...