[Python] Why do accuracy scores vary in my RF even though I set the random state?

I am using scikit learns random forest and if I run the analysis again the accuracy scores stay mostly the same except for the last couple of digits. Is this something I should be worried about or am I maybe doing the random state thing wrong? I pasted how I do it below:

rf = RandomForestClassifier(random_state=42)


This is what I do at the beginning of the code and then I do something like this:

# Prepare variables for cross-validation
X = df[included_columns] # Features
y = df['strategy_encoded'] # Target variable

# Identify participants for each strategy
unique_strategies = df['strategy'].unique()
strategy_participants = {strategy: df[df['strategy'] == strategy]['participant_number'].unique() for strategy in unique_strategies}

# Generate participant combinations for cross-validation
participant_combinations = list(product(*strategy_participants.values()))

# Initialize lists for results
accuracies = []
training_accuracies = []
confusion_matrices = []
classification_reports = []

# Cross-validation with participant combinations
for combo in participant_combinations:
# Train-test split based on participant combinations
test_indices = df[df['participant_number'].isin(combo)].index
train_indices = df[~df.index.isin(test_indices)].index

X_train, X_test = X.loc[train_indices], X.loc[test_indices]
y_train, y_test = y.loc[train_indices], y.loc[test_indices]

# Train the model
rf = RandomForestClassifier()
rf.fit(X_train, y_train)

# Training accuracy
y_train_pred = rf.predict(X_train)
train_accuracy = accuracy_score(y_train, y_train_pred)
training_accuracies.append(train_accuracy)

# Test accuracy
y_pred = rf.predict(X_test)
test_accuracy = accuracy_score(y_test, y_pred)
accuracies.append(test_accuracy)

# Confusion matrix and classification report
conf_matrix = confusion_matrix(y_test, y_pred)
confusion_matrices.append(conf_matrix)

class_report = classification_report(y_test, y_pred, output_dict=True)
classification_reports.append(class_report)

# Calculate average results across folds
average_test_accuracy = np.mean(accuracies)
average_training_accuracy = np.mean(training_accuracies)
average_conf_matrix = np.mean(confusion_matrices, axis=0)

# Output the results
print("Average Test Accuracy:", average_test_accuracy)
print("Average Training Accuracy:", average_training_accuracy)
print("Average Confusion Matrix:\n", average_conf_matrix)

Continue reading...