I have been trying for the last few days to train a neural network on an extremely unbalanced dataset, but the results have not been good enough, there are 10 classes and for 4 or 5 of them it does not obtain good results. I could start to group them but I want to try to get at least decent results for the minority classes. This is the dataset Kaggle Dataset The pre processing I did was the following one: -Obtain temporal data from the time the loan has been on datos_crudos['loan_age_years'] = (reference_date - datos_crudos['issue_d']).dt.days / 365 datos_crudos['credit_history_years'] = (reference_date - datos_crudos['earliest_cr_line']).dt.days / 365 datos_crudos['days_since_last_payment'] = (reference_date - datos_crudos['last_pymnt_d']).dt.days datos_crudos['days_since_last_credit_pull'] = (reference_date - datos_crudos['last_credit_pull_d']).dt.days Drop columns which have 40% or more NaN Imputation for categorical and numerical data categorical_imputer = SimpleImputer(strategy='constant', fill_value='Missing') numerical_imputer = IterativeImputer(max_iter=10, random_state=42) One Hot Encoding, Label Encoder and Ordinal Encoder Also did this -Feature selection through random forest -Oversampling and Undersampling techniques, used SMOTE Current 361097 Fully Paid 124722 Charged Off 27114 Late (31-120 days) 6955 Issued 5062 In Grace Period 3748 Late (16-30 days) 1357 Does not meet the credit policy. Status:Fully Paid 1189 Default 712 Does not meet the credit policy. Status:Charged Off 471 And the code to do SMOTE and undersampling undersample_strategy = { 'Current': 100000, 'Fully Paid': 80000 } oversample_strategy = { 'Charged Off': 50000, 'Default': 30000, 'Issued': 50000, 'Late (31-120 days)': 30000, 'In Grace Period': 30000, 'Late (16-30 days)': 30000, 'Does not meet the credit policy. Status:Fully Paid': 30000, 'Does not meet the credit policy. Status:Charged Off': 30000 } Computed class weights Focal loss function I am watching F1 Macro because of the unbalanced data This is the architecture model = Sequential([ Dense(1024, activation="relu", input_dim=X_train.shape[1]), BatchNormalization(), Dropout(0.4), Dense(512, activation="relu"), BatchNormalization(), Dropout(0.3), Dense(256, activation="relu"), BatchNormalization(), Dropout(0.3), Dense(128, activation="relu"), BatchNormalization(), Dropout(0.2), Dense(64, activation="relu"), BatchNormalization(), Dropout(0.2), Dense(10, activation="softmax") # 10 clases ]) And the report classification, the biggest problems are class 3,6 and 8 some epochs obtain really low metrics for those clases Epoch 7: F1-Score Macro = 0.5840 5547/5547 [==============================] - 11s 2ms/step precision recall f1-score support 0 1.00 0.93 0.96 9125 1 0.99 0.85 0.92 120560 2 0.94 0.79 0.86 243 3 0.20 0.87 0.33 141 4 0.14 0.88 0.24 389 5 0.99 0.95 0.97 41300 6 0.02 0.00 0.01 1281 7 0.48 1.00 0.65 1695 8 0.02 0.76 0.04 490 9 0.96 0.78 0.86 2252 accuracy 0.87 177476 macro avg 0.58 0.78 0.58 177476 weighted avg 0.98 0.87 0.92 177476 Any idea what could be missing to obtain better results? Continue reading...
