[Python] How to vectorize computation of spearman correlation between a 2D array and a...

I have a matrix M with size (37, N) and an additionnal 1D reference vector of size (37,1)

I am looking for a way to compute the spearman correlation between each sample of M and my reference to obtain a (N, 1) matrix as a result. Using a for loop I can do :


from scipy.stats import spearmanr
import numpy as np

M = np.random
rng = np.random.default_rng()
M = rng.standard_normal((37, 100))
reference = rng.standard_normal((37, 1))

expected_result = []
for element in M.T:
expected_result.append( spearmanr(element, reference).statistic)

print(np.array(expected_result).shape)


It gives what I want (shape (100,1)) but I would like it to be vectorized to speed up computations. I tried using the following code :


from scipy.stats import spearmanr
import numpy as np

M = np.random
rng = np.random.default_rng()
M = rng.standard_normal((37, 100))
reference = rng.standard_normal((37, 1))
res = spearmanr(M, reference)


but the res.statistic ends up in the shape (101,101), which is even more confusing to me.

Is there a way to vectorize this ?

Thank you

Continue reading...