[Python] Maximizing corretc classification with PuLP

In python, I’m trying to solve a problem of linear optimization with the pulp package. I have N continuous indicators and a dichotomous target. I need to select a subset of M indicators and a threshold for each indicator so that if at least P indicators are above their threshold the record is classified as 1. The optimization problem must determine these variables:

• The number M of the subset variable
• The number P of minimum indicator who can be breached
• The threshold for each variable

The final scope is to have misclassified as 1 no more than 10 0s of the target, maximizing the classification of 1s

Hereafter my example code… I don’t understand why it is setting very high thresholds and no records are classified as 1.

import pandas as pd
import numpy as np
from imblearn.over_sampling import SMOTE
from pulp import LpMaximize, LpProblem, LpVariable, lpSum, value


# Sample data
np.random.seed(42)
data = pd.DataFrame(np.random.rand(1000, 10), columns=[f'X{i}' for i in range(10)]) # 10 continuous indicators
data['target'] = np.random.choice([0, 1], size=1000, p=[0.7, 0.3]) # Binary target (30% ones)

# Parameters
M = 5 # Number of selected indicators
S = 3 # Minimum number of indicators above threshold to classify as 1
max_false_positives = 100 # Maximum allowed misclassified 0s

# Define model
model = LpProblem("Indicator_Selection", LpMaximize)

# Decision variables
X = {i: LpVariable(f'X_{i}', cat='Binary') for i in range(data.shape[1] - 1)} # Indicator selection
T = {i: LpVariable(f'T_{i}', lowBound=-100, upBound=100, cat='Continuous') for i in range(data.shape[1] - 1)} # Thresholds
Y = {j: LpVariable(f'Y_{j}', cat='Binary') for j in range(len(data))} # Predicted labels

# Auxiliary binary variables to represent whether each indicator exceeds its threshold
Z = {(j, i): LpVariable(f'Z_{j}_{i}', cat='Binary') for j in range(len(data)) for i in X}

# Constraint: Select exactly M indicators
model += lpSum(X for i in X) <= M

# Constraint: Define Z[j, i] (1 if data.iloc[j, i] >= T, 0 otherwise)
M_large = 1 # Upper bound for continuous values
for j in range(len(data)):
for i in X:
model += Z[j, i] >= (data.iloc[j, i] - T) / M_large
model += Z[j, i] <= X # Z[j, i] can only be 1 if the feature is selected

# Constraint: Ensure classification rule
for j in range(len(data)):
model += Y[j] * S <= lpSum(Z[j, i] for i in X)

# Constraint: Limit false positives
model += lpSum(Y[j] for j in range(len(data)) if data.iloc[j, -1] == 0) <= max_false_positives

# Objective: Maximize correct classifications
model += lpSum(Y[j] for j in range(len(data)) if data.iloc[j, -1] == 1)

# Solve the model
model.solve()

# Output results
selected_features = [i for i in X if value(X) == 1]
thresholds = {i: value(T) for i in selected_features}

Continue reading...