[Python] Why does average_precision_score ignore predictions where y_true = 0?

Consider this example:

import numpy as np
from sklearn import metrics

y_true = np.array([[1, 1, 1],[0, 0, 0],[0, 0, 0]])
y_pred = np.array([[1, 1, 1],[.9, .9, .9],[.8, .8, .8]])

aps = metrics.average_precision_score(y_true, y_pred, average='micro')
roc_auc = metrics.roc_auc_score(y_true, y_pred, average='micro')
print(f"average_precision_score = {aps:.1f}, ROC AUC = {roc_auc:.1f}\n")

for threshold in np.arange(.8, 1.01, .1):
y_pred_copy = y_pred.copy()
y_pred_copy[y_pred_copy >= threshold] = 1
y_pred_copy[y_pred_copy < threshold] = 0
precision = metrics.precision_score(y_true, y_pred_copy, average='micro', zero_division=0)
recall = metrics.recall_score(y_true, y_pred_copy, average='micro', zero_division=0)
print(f"At threshold {threshold:.1f}, {precision=:.1f} and {recall=:.1f}")


The output is as follows:

average_precision_score = 1.0, ROC AUC = 1.0

At threshold 0.8, precision=0.3 and recall=1.0
At threshold 0.9, precision=0.5 and recall=1.0
At threshold 1.0, precision=1.0 and recall=1.0


The non-zero values in y_pred after the first row are ignored by average_precision_score and roc_auc_score. But isn't it relevant that there are large non-zero false positives in those cases, as indicated by the output from the threshold loop? Is there a way to account for those?

Continue reading...