[Python] How do I tell tensorflow to throw an error if I am trying to do a non-differentiable...

I am learning tensorflow and spent a good amount of time trying to find what is causing this error:

No gradients provided for any variable.


In the end I tracked that it was caused by using argmax at the very start of my gradient calculation.

Is there some sort of guard or wrapper that can be put on my tensors and will throw at the moment when I lose the differentiability? This should be transitive of course, so if I do this:

raw_output = my_model((input_data))
tf.make_sure_all_steps_are_differentiable(raw_output)
# this is differentiable of course, although I am not sure if it's correct in this context
multiplied = raw_output * 5
# This is not differentiable and should raise
# Non differentiable operation attempted on tensor [name]
# or something like that
# The guard should have been transferred to the new tensor during previous operation
argmaxed = tf.argmax(multiplied, other, arguments)


Does tensorflow has some helper like that? I could find a lot of questions about what's differentiable and what isn't. This would help find out the moment you use it.

Continue reading...