[Python] `make_scorer(needs_sample_weight=True)` wrongly injects `needs_sample_weight` into...

When using make_scorer(..., needs_sample_weight=True), the generated scorer unexpectedly passes needs_sample_weight=True as a keyword argument to the scoring function itself, leading to TypeError.

Minimal example:

import numpy as np
from sklearn.datasets import make_regression
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import make_scorer

def weighted_mape(y_true, y_pred, sample_weight=None):
return np.average(np.abs((y_true - y_pred) / (y_true + 1e-8)), weights=sample_weight)

scoring = make_scorer(weighted_mape, greater_is_better=False, needs_sample_weight=True)

X, y = make_regression(n_samples=100, n_features=5, random_state=0)
weights = np.random.rand(100)

model = GradientBoostingRegressor()
grid = GridSearchCV(model, param_grid={"n_estimators": [10]}, scoring=scoring, cv=3)
grid.fit(X, y, sample_weight=weights)

Expected Results


make_scorer(..., needs_sample_weight=True) should cause sample_weight to be passed during cross-validation scoring.

The scoring function should not receive needs_sample_weight=True as a kwarg.

Actual Results


The scoring function raises:

TypeError: weighted_mape() got an unexpected keyword argument 'needs_sample_weight' unless manually patched with **kwargs.
​

Any workaround on how to use weights in a custom scorer?

Environment details:
Python version : 3.10.12 (main, Feb 4 2025, 14:57:36) [GCC 11.4.0]
Platform : Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
scikit-learn : 1.2.2
numpy : 1.26.0
pandas : 1.5.3
xgboost : 2.1.2

Continue reading...