I want to undersample 3 cross-validation folds from a dataset, using say, RandomUnderSampler from imblearn, and then, optimize the hyperparameters of various gbms using those undersampled folds as input. The code I have so far is: def train_model_with_undersampling(undersampler, estimator, scale, params, X_train, y_train): # we need the resampler within a pipeline, because we are # using cross-validation to optimize hyperparameters, which # means that we need a left out fold without resampling to # evaluate the model. # The only way is with imblearn's pipeline (see imblearn docs) # annoying cause we need to resample every time if scale is True: pipe = Pipeline([ ("scaler", MinMaxScaler()), ("sampler", undersampler), ("model", estimator), ]) else: pipe = Pipeline([ ("sampler", undersampler), ("model", estimator), ]) search = HalvingRandomSearchCV( estimator=pipe, param_distributions=params, n_candidates="exhaust", factor=3, # only a third of the candidates are promoted resource='model__n_estimators', # the limiting resource max_resources=500, # max number of trees min_resources=10, scoring='roc_auc', cv=3, random_state=10, refit=True, n_jobs=-1, ) search.fit(X_train, y_train) return search However, this function will undersample the data to tune each gbm model. This is inefficient, because the undersampling is the same. What I would like is to be able to pass to HalvingRandomSearchCV the undersampled folds and the test fold somehow. In short, I want to undersample 3 different folds of X_train, and then be able to use those to optimize the hyperparameters of xgboost, catboost, gradientboostingclassifier and other models. Is there a way to do so? Continue reading...
