[Python] Constructing custom loss function in lightgbm

I have a pandas dataframe that records the outcome of F1 races:

data = {
"Race_ID": [0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4],
"Racer_Number": [1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4],
"Win_odds": [2.3, 4.3, 1.3, 5.0, 3.4, 1.1, 2.3, 2.0, 7.4, 3.1, 1.8, 6.0, 2.4, 5.2, 3.5, 3.0, 1.4, 3.2, 2.5, 5],
"Rank": [3, 2, 1, 4, 2, 1, 4, 3, 4, 2, 1, 3, 4, 2, 3, 1, 1, 2, 3, 4]
}
df_long = pd.DataFrame(data)


and I want to train a lightgbm classifier using the LGBMClassifier API to predict the winner of a race using Win_odds as feature. To do that, we convert the data to wide format:

# Reshape into wide format (one row per race, racers as columns)
df_wide = df_long.pivot(
index="Race_ID",
columns="Racer_Number",
values=["Win_odds","Rank"]
)
df_wide.columns = [f"{col[0]}_{col[1]}" for col in df_wide.columns]
df_wide = df_wide.reset_index()
df_wide['target'] = df_wide.apply(lambda row: np.argmin(row['Rank_1':'Rank_4']) , axis=1)


And then we can use the columns Win_odds_1, Win_odds_2, Win_odds_3, Win_odds_4 as features and the target as our target. In multi-class classification, typically we use the (multi-class) log loss as our loss function as documented here: https://scikit-learn.org/stable/modules/model_evaluation.html#log-loss



where in our case K=4, N=5. However, I would like to construct a custom loss function along with its gradient and hessian defined by the following formula:



where o_{i,k} is the Win_odds of sample i with label k and lambda is a positive constant (say 1) to determine the relative significance of the regularization term.

To simplify notation, let



To construct the custom loss function for use in lightgbm, we need to compute the gradient and hessian and here is my attempt:



However, I am not sure if there is any way to simplify the above expression, and much less to write it in code. And I have no idea how to compute the hessian since I am stuck at the gradient.

So for example the total custom loss of the above dataset is given by

\frac{-1}{5}((\log(p_{0,2})-(p_{0,0}-\frac{1}{2.3})^2-(p_{0,1}-\frac{1}{4.3})^2-(p_{0,2}-\frac{1}{1.3})^2-(p_{0,3}-\frac{1}{5})^2)+
(\log(p_{1,1})-(p_{1,0}-\frac{1}{3.4})^2-(p_{1,1}-\frac{1}{1.1})^2-(p_{1,2}-\frac{1}{2.3})^2-(p_{1,3}-\frac{1}{2})^2)+
(\log(p_{2,2})-(p_{2,0}-\frac{1}{7.4})^2-(p_{2,1}-\frac{1}{3.1})^2-(p_{2,2}-\frac{1}{1.8})^2-(p_{2,3}-\frac{1}{6})^2)+
(\log(p_{3,3})-(p_{3,0}-\frac{1}{2.4})^2-(p_{3,1}-\frac{1}{5.2})^2-(p_{3,2}-\frac{1}{3.5})^2-(p_{3,3}-\frac{1}{3})^2)+
(\log(p_{4,0})-(p_{4,0}-\frac{1}{1.4})^2-(p_{4,1}-\frac{1}{3.2})^2-(p_{4,2}-\frac{1}{2.5})^2-(p_{4,3}-\frac{1}{5})^2))


I am not sure how to do it as my target variable is 1-dimensional with only the label of the winner, and LightGBM's API doesn't support passing additional data to the loss function directly.

Continue reading...