[Python] Cython GroupBy Function much slower then similar Numba Function

I have a very basic group by function that i want to use in a Cython object but it's something like 400 times slower then then a similar function in Python JITed by Numba

This is my cython function

@cython.boundscheck(False) # Deactivate bounds checking
@cython.wraparound(False) # Deactivate negative indexing.
@cython.ccall
def groupby(
in_index: cython.long[:], in_values: cython.float[:], number_of_results: cython.long
) -> cython.double[:]:
results = np.zeros(number_of_results, dtype=np.float64)
results_view: cython.float[:] = results
index: cython.long[:] = in_index
val: cython.float[:] = in_values
index_max: cython.Py_ssize_t = in_index.shape[0]
n: cython.Py_ssize_t
for n in range(index_max):
results_view[index[n]] += val[n]

return results


but this function with Numba


@jit(nopython=True, fastmath=True)
def agg_sims(group_idx, a, number_of_sims):
output = np.zeros(number_of_sims, dtype=np.float64)
for i in range(group_idx.shape[0]):
output[group_idx] += a
return output


I'd like to use cython because Numba doesn't deal well with the object I built hold the index and values part.

I isolated out the function into it's own file and compiled it with cython (on windows with VS build tools) and using a set of random data but the the index is ordered

this is an example of the data sim_index=np.array([0,1,2,2,3,3,3,3,3,4,4,5,5,5,5,6,6,7,7,7,7,7,8,8,9,9,9])

loss_values=np.array([983218.798545568,78773.246069412,427915.701586101,384401.565066934,976474.931385909, 491002.57968459,449277.841899304,2157814.69086177,634849.51236713,683151.164470444,951717.254327894, 338593.160096285,111153.600572457,327118.874300972,585857.183862426,1793115.50344123,775898.742468206, 236320.911958699,412790.930873261,894075.886633843,400713.081209467,1142732.59966746, 651378.893317142,238021.483213581,50557.1083702021,75622.454060533,153180.479257057])

The actual data is about 2.5m values groupby in cython took gave me this with timeit 805 ms ± 30.9 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

but the numba version with the same data

took this 4.35 ms ± 667 μs per loop (mean ± std. dev. of 7 runs, 1 loop each)

i'm at a looss on why the cython function is so much slower then the numba function.

The cython function doesn't show any major python interactions in the annotation html file. really just the numpy function.

Any suggestions would really be helpful.

Continue reading...