Starting to try to use Ray to parallelize a number of task-parallel jobs. I.e. each task takes in an object from a data frame, and then returns a list. Within the function, there is a check for a property of the object though, and if that property if fulfilled I want the task to be canceled gracefully. (I know one could hack around it by setting the retries per task to 0, and then set the number of permitted task failures to infinity) The structure of the function is sort of like this: import ray import numpy as np ray.init() @ray.remote def test_function(x): if np.random.rand() < 0.5: raise Exception("blub") return [x, x*x, "The day is blue"] futures = [test_function.remote(i) for i in range(10000)] print(ray.get(futures)) Is there a best-practice, or a graceful way to let the individual tasks fail? Continue reading...
