[Python] How to further reduce MSE/RMSE in CatBoost after tuning, data augmentation, and...

I am using CatBoost model with 100 data and I have done data augmentation, hyperparameter tuning,cross validation, added isolation forest and randomizedsearchCV and at the end I could have this result as minimum:

Tuned CatBoost with Isolation Forest + Augmentation Model → MSE: 4062.61, RMSE: 63.74, R²: 0.9989
Tuned CatBoost with Isolation Forest + Augmentation 10-Fold CV RMSE: [319.47092218 36.47791126 468.55095141 133.94190436 45.46218705
104.49212969 93.89201169 177.19812971 283.95153831 114.13156753]
Mean RMSE: 177.76, Std Dev: 131.05

Best Hyperparameters: {'random_strength': 0.5, 'learning_rate': 0.03, 'l2_leaf_reg': 1, 'iterations': 1500, 'depth': 8, 'boosting_type': 'Ordered', 'bagging_temperature': 0.5}



However, I know that this result is not enough for training mode so what else should I do to make the MSE and RMSE decrease?

Continue reading...