For example, for Softmax, there's an efficient formula that uses the output value to compute its derivative. However, JAX grad creates a derivative function that accepts inputs. For softmax, that means recomputing the exponentials. So should I stick with the JAX grad or write the custom derivative with the more efficient formula? Or am I missing info about how to use JAX grad? Continue reading...
