[Python] How to get all jobs associated with an experiment in Azure ML Python SDK v2?

I am trying to get a list of all the jobs in my workspace belonging to a certain experiment. This is a trivial matter using the v1 of the SDK (relevant documentation). However, I am unable to do that using the v2.

More in general, I can't figure out how to explore jobs applying filters of any kind. The closest thing I can find in the documentation is this list method combined with list comprehension, but that is way too slow.

What would be the best way to do this using the Azure ML SDK v2? Is it possible to do it at all?

First, this is the code using the v1. It does what I want and it takes less than a second to complete:

#My version of azureml-core is 1.49.0
import azureml.core
from azureml.core import Workspace, Experiment

ws = Workspace.from_config()
exp = Experiment(workspace=ws, name="my_experiment_name")
runs = exp.get_runs()


Now this is the closest thing I came up with using the version 2 of the sdk, using the same workspace as the previous example. This also works but it takes around seven minutes to complete:

# My version of azure-ai-ml is 1.5.0
from azure.ai.ml import MLClient
from azure.identity import DefaultAzureCredential, InteractiveBrowserCredential

ml_client = MLClient(
credential= DefaultAzureCredential(),
subscription_id= "my_subscription_id",
resource_group_name= "my_resource_group_name",
workspace_name= "my_workspace_name",
)

runs = [run for run in ml_client.jobs.list() if run.experiment_name == "my_experiment_name"]

Continue reading...