I previously created metrics for a TensorFlow Classifier referencing predictions['logits'] to calculate the metrics. I have changed the model from a Classifier to an Estimator in order to enable multi-objective learning (using MultiHead). However, this has caused Python to throw an error since now the elements of predictions are keyed by pairs of the head name and original key, e.g. ('label1','logits') for a head with name 'label1'. I'd like to allow for dynamic generation of metrics based on a configuration file in order to more easily train and test a variety of models with different label combinations. The problem now is that the metric_fn parameter to tf.estimator.add_metrics does not take any additional parameters to allow for dynamically determined or constructed metrics. How can I generate an estimator with multiple heads and custom metrics for each head? Continue reading...
