So I am working on a project whereby I have to quantize my model's weights and biases to integers and perform subsequent operations using integers. The output of my model can be either (int8 or int16) values (in this case, logits) and I need to call softmax on this logits output/array. I was able to find an integer implementation of softmax written in C (https://github.com/ARM-software/CMSIS-NN/tree/main/Source/SoftmaxFunctions). The problem I'm having is trying to evaluate that this C implementation is accurate (or more specifically, that I am using it accurately). The way I'm thinking of doing that is detailed below: In Python Take my integer logits -> call an integer python implementation of softmax on the logits -> get a result (python_integer_prediction_probabilities). In C (using CMSIS-NN's) Take the same integer logits -> call the C softmax implementation on my logits -> get a result (CMSIS_NN_prediction_probabilities) Finally, I compare these two results to see if they are close enough. The main problem I'm having is, I assumed there would be information about how to implement a softmax function, using Python, which takes integer inputs, but I can't find anything online. Does anyone have an idea of how to implement this in python or is aware of resources that I could use to figure this out? thank you. Continue reading...
