[Python] Randomized Search CV with Neural Network

I want to tune a neural network but since i would be doing a lot of test runs i decided to use Randomized search CV

I first started by freeing up some space.

# Reseting layers and freeing up some space
tf.keras.backend.clear_session()


I then created a function to hold my neural network model.

def create_model(learning_rate=0.02):
model = Sequential([
Dense(128, activation='relu', input_shape=(23,)), #Input Layer
Dense(64, activation='relu'), #Hidden Layer
Dense(7, activation='softmax') #Output Layer
])
# Compiling Model
from tensorflow.keras.optimizers import Adam
model.compile(optimizer=Adam(learning_rate=learning_rate), loss='sparse_categorical_crossentropy', metrics=['accuracy', map_3])

return model


I wanted an early stopping parameter so I created a variable to hold that.

# Early Stopping
from tensorflow.keras.callbacks import EarlyStopping
early_stopping = EarlyStopping(monitor = 'val_map_3',
patience = 5,
mode = 'max',
restore_best_weights=True)


I then define my parameter dictionary for my Randomized Search CV.

# Setting Parameters for Randomized Search CV
param_dict = {'epochs': [20, 40, 60, 80, 100, 120, 150],
'batch_size': [32, 64, 128, 256, 512, 1024],
'learning_rate': [0.04, 0.06, 0.08, 0.1, 0.2, 0.5, 1.0, 2.0]}


I used Scikeras to wrap my neural network.

from scikeras.wrappers import KerasClassifier
tunned_model = KerasClassifier(
model=create_model,
epochs=param_dict['epochs'],
batch_size=param_dict['batch_size'],
learning_rate=param_dict['learning_rate'], verbose=1)


Then called RandomizedSearchCV and fit my data and i got an error.

# Loading Randomized Search CV
random_search = RandomizedSearchCV(tunned_model, param_distributions=param_dict, n_iter= 20, scoring='accuracy', verbose=2, cv=5, n_jobs=1, refit=True)

# Fitting Neural Network
random_search.fit(
X_train,
y_train,
validation_data=(X_val, y_val),
callbacks=[early_stopping]
)

Output:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-40-1796011897> in <cell line: 0>()
3
4 # Fitting Neural Network
----> 5 random_search.fit(
6 X_train,
7 y_train,

4 frames
/usr/local/lib/python3.11/dist-packages/sklearn/base.py in __sklearn_tags__(self)
538
539 def __sklearn_tags__(self):
--> 540 tags = super().__sklearn_tags__()
541 tags.estimator_type = "classifier"
542 tags.classifier_tags = ClassifierTags()

AttributeError: 'super' object has no attribute '__sklearn_tags__'


I have done my best to figure out what the problem is and nothing seems to work. I need help with the code.

Continue reading...