[Python] How to return a generator using joblib.Parallel()?

I have a piece of code below where the joblib.Parallel() returns a list.

import numpy as np
from joblib import Parallel, delayed

lst = [[0.0, 1, 2], [3, 4, 5], [6, 7, 8]]
arr = np.array(lst)
w, v = np.linalg.eigh(arr)

def proj_func(i):
return np.dot(v[:,i].reshape(-1, 1), v[:,i].reshape(1, -1))

proj = Parallel(n_jobs=-1)(delayed(proj_func)(i) for i in range(len(w)))


Instead of a list, how do I return a generator using joblib.Parallel()?

Edit:

I have updated the code as suggested by @user3666197 in comments below.

import numpy as np
from joblib import Parallel, delayed

lst = [[0.0, 1, 2], [3, 4, 5], [6, 7, 8]]
arr = np.array(lst)
w, v = np.linalg.eigh(arr)

def proj_func(i):
yield np.dot(v[:,i].reshape(-1, 1), v[:,i].reshape(1, -1))

proj = Parallel(n_jobs=-1)(delayed(proj_func)(i) for i in range(len(w)))


But I am getting this error:

TypeError: can't pickle generator objects


Am I missing something? How do I fix this? My main gain here is to reduce memory as proj can get very large, so I would just like to call each generator in the list one at a time.

Continue reading...