I have this code of a XGBoost model: temp_dataset_pos = dataset_vehicle_pos.copy() # temp_dataset_pos = temp_dataset_pos.drop(['sum_passenger'], axis=1) # temp_dataset_pos = temp_dataset_pos[['humidity','temperature', 'vehicle_avg_speed_pos']] X_pos = temp_dataset_pos.drop('vehicle_avg_speed_pos', axis=1) y_pos = temp_dataset_pos['vehicle_avg_speed_pos'] # Split the data into train and test sets X_train, X_test, y_train, y_test = train_test_split(X_pos, y_pos, test_size=0.20, random_state=42) # Create an XGBoost model xgboost_model = xgb.XGBRegressor(objective='reg:squarederror', random_state=42) # You can adjust hyperparameters # Fit the model xgboost_model.fit(X_train, y_train) # Ensure the predicted data aligns with the test set X_test['vehicle_count_pos'] = predicted_vehicle_count_pos['predicted_vehicle_count_pos'] # Make predictions on the test set y_pred_xgboost = xgboost_model.predict(X_test) # Round the predicted values to integers y_pred_rounded = y_pred_xgboost.round().astype(int) # Evaluate the model mse_xgboost = mean_squared_error(y_test, y_pred_rounded) print(f'Mean Squared Error (XGBoost): {mse_xgboost}') r2_xgboost = r2_score(y_test, y_pred_rounded) print('R-squared value:', r2_xgboost) importance = xgboost_model.feature_importances_ # Create an XGBoost model xgboost_model = xgb.XGBRegressor(objective='reg:squarederror', random_state=42) # You can adjust hyperparameters # Define a K-Fold cross validator kf = KFold(n_splits=5, shuffle=True, random_state=42) # Define the scorer based on Mean Squared Error mse_scorer = make_scorer(mean_squared_error, greater_is_better=False) # Perform cross-validation and calculate MSE mse_scores = cross_val_score(xgboost_model, X_pos, y_pos, scoring=mse_scorer, cv=kf) # Convert MSE scores to positive; cross_val_score returns negative values for 'greater_is_better=False' mse_scores = -mse_scores # Calculate mean and standard deviation of MSE mean_mse = np.mean(mse_scores) std_mse = np.std(mse_scores) print(f'Mean MSE from CV: {mean_mse}') print(f'Std MSE from CV: {std_mse}\n') # If you also want to include R-squared in your results summary r2_scorer = 'r2' r2_scores = cross_val_score(xgboost_model, X_pos, y_pos, scoring=r2_scorer, cv=kf) mean_r2 = np.mean(r2_scores) std_r2 = np.std(r2_scores) print(f'Mean R-squared from CV: {mean_r2}') print(f'Std R-squared from CV: {std_r2}') print('\n') And I get these results: Mean Squared Error (XGBoost): 395.3362869635255 R-squared value: 0.37072522417952525 Mean MSE from CV: 6.902625450153782 Std MSE from CV: 0.8885273860917627 Mean R-squared from CV: 0.990070278719086 Std R-squared from CV: 0.000623638887270274 How come that the R-Squared is so different with and without CV? Am I missing something in my code? Continue reading...
