[Python] Default credentials with WorkspaceClient hosted in an Azure Web App with managed...

I have a fast API hosted in an azure web app. I am trying to get some data from DataBricks and return it from the FastAPI. This is my context:

DbxContext.py:

import time
import pandas as pd
from databricks import sql
from databricks.sdk import WorkspaceClient
from pydantic import SecretStr
# from databricks.sdk.service import users

class DataBricksOptions:
def __init__(self, client_id: str, client_secret: SecretStr, tenant_id: str, workspace_resource_id: str, http_path: str, host: str):
self.client_id = client_id
self.client_secret = client_secret
self.tenant_id = tenant_id
self.workspace_resource_id = workspace_resource_id
self.http_path= http_path
self.host = host

class DataBricksContext:
def __init__(self, options: DataBricksOptions):
self.options = options

def get_databricks_connection(self):
try:
# Initialize Databricks Workspace Client using default credentials
w = WorkspaceClient()
me2 = w.current_user.me()
print(f"Me: {me2.display_name}")
#w.users.me()
# Create Databricks Token
token = w.tokens.create(comment=f"sdk-{time.time_ns()}", lifetime_seconds=3600)

# Establish SQL Connection
connection = sql.connect(
server_hostname=self.options.host,
http_path=self.options.http_path,
access_token=token.token_value
)
return connection
except Exception as e:
print(f"Error in get_databricks_connection(): {e}")
return None

def exec_query_to_df(self, connection, query: str):
try:
# Use the connection to execute the query
df = pd.read_sql(query, connection)
return df
except Exception as e:
print(f"Error in exec_query_to_df(): {e}")
return None


When I run it locally in VS code everything works as expected. When I run it in Azure I get following error:

Error in get_databricks_connection(): default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication
​

I have set System Assigned Identity to On on the azure web app. Any suggestions?

Continue reading...