[Python] Reproduce a particular tree from the random forest using DecisionTreeRegressor

I am trying to replicate a specific decision tree trained by a RandomForestRegressor class, using DecisionTreeRegressor. However, I cannot get the exact results, even with using the exact same bootstrapped samples and the same hyperparameters. Has anyone tried this before and been successful?

See my code below:

# Import packages
import numpy as np
from sklearn.ensemble import RandomForestRegressor
from sklearn.tree import DecisionTreeRegressor
from sklearn.datasets import make_regression
import matplotlib.pyplot as plt
from sklearn.tree import plot_tree

# Parameters
n_samples = 160
n_features = 20


# Create training data
Xs, y = make_regression(n_samples = n_samples, n_features = n_features, random_state = 0, shuffle = False)

# Create testing Xs
np.random.seed(0)
Xs_test = np.random.standard_normal((40, n_features))


#%%
# Random forest
n_estimators = 10
max_depth = 4
min_samples_leaf = 17
min_samples_split = 3
rf = RandomForestRegressor(random_state = 0, oob_score = False, max_features = None,
n_estimators = n_estimators,
max_depth = max_depth,
min_samples_leaf = min_samples_leaf,
min_samples_split = min_samples_split)
rf.fit(Xs, y)
y_pred = rf.predict(Xs_test)

# Plotting the 1st tree structure
tree = rf.estimators_[0]
plot_tree(tree, filled = True, rounded = True, node_ids = True, fontsize = 16)

plt.figure(figsize=(20, 10)
plt.show()

#%%
# Replicate the 1st tree
# Get the bootstrapped samples
sample_indices = rf.estimators_samples_[0]
bootstrap_Xs = Xs[sample_indices, :]
bootstrap_y = y[sample_indices]

decision_tree = DecisionTreeRegressor(random_state = tree.random_state, max_features = None,
max_depth = max_depth,
min_samples_leaf = min_samples_leaf,
min_samples_split = min_samples_split)
decision_tree.fit(bootstrap_Xs, bootstrap_y)
y_pred_check = decision_tree.predict(Xs_test)


# Plot the replicated decision tree
plot_tree(decision_tree, filled = True, rounded = True, node_ids = True, fontsize = 16)
plt.figure(figsize=(20, 10))
plt.show()



With above, I got different predicted ys, and different tree structures. Can anyone shed some lights? I am curious what additional settings has RandomForestRegressor done on top of the DecisionTreeRegressor.

Continue reading...