[Python] in python, how to check 250 Trillion combinations of values, run function, get best...

I have an algorithm for which I need to find the best parameter combo given from some ranges.

r1 = np.arange(30, 150, 1)
r2 = np.arange(0, 1, 0.05)
r3 = np.arange(0, 10, 0.1)
r4 = np.arange(1, 8, 1)
r5 = np.arange(3, 100, 1)
r6 = np.arange(6, 100, 1)


@nb.njit(parallel=True)
def _check_params(p1, p2, p3, p4, p5, p6, data):
tp = 0
for i in nb.prange(len(data)):
if p1 > data:
tp += 3
if p2 < data:
tp += 1
if p3 < data:
tp += 3
if p4 < data:
tp += 3
if p5 < data:
tp += 4
if p6 < data:
tp += 5
return tp


@nb.njit(parallel=True)
def _loop_combos(r1, r2, r3, r4, r5, r6, data):
best_result = 0
best_params = [0, 0, 0, 0, 0, 0]
for p1 in nb.prange(r1):
for p2 in range(r2):
for p3 in range(r3):
for p4 in range(r4):
for p5 in range(r5):
for p6 in range(r6):
result = _check_params(p1, p2, p3, p4, p5, p6, data)
if result > best_result:
best_result = result
best_params[0] = p1
best_params[1] = p2
best_params[2] = p3
best_params[3] = p4
best_params[4] = p5
best_params[5] = p6
print(best_result)
print(best_params)


_loop_combos(r1, r2, r3, r4, r5, r6, some_data)


Almost everything about this can be parallelized except result > best_result because since the parallelization cannot simultaneously check & update the same value.

I also tried creating an original array of size r1*r2*r3*r4*r5*r6 such that each parallel run saves to a different location, but due to the combinatorics explosion this array would be too big.

I also checked and there does not seem to be any kind of semaphore/mutex lock available in the parallelization, so i cannot really "wait" for best_result to be available to check.

Is there anything I'm missing or is this simply cannot be done in a parallel way because of the best_result = result check?

Continue reading...