Grid Search in Multi class classification problems using Neural networks - neural-network

Question

I'm trying to do grid search for a multi class problem in neural networks.
I am not able to get the optimum parameters, the kernel keeps on compiling.
Is there any problem with my code? Please do help
import keras
from keras.models import Sequential
from keras.layers import Dense
# defining the baseline model:
def neural(output_dim=10,init_mode='glorot_uniform'):
model = Sequential()
model.add(Dense(output_dim=output_dim,
input_dim=2,
activation='relu',
kernel_initializer= init_mode))
model.add(Dense(output_dim=output_dim,
activation='relu',
kernel_initializer= init_mode))
model.add(Dense(output_dim=3,activation='softmax'))
# Compile model
model.compile(loss='sparse_categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
return model
from keras.wrappers.scikit_learn import KerasClassifier
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold
from sklearn.model_selection import GridSearchCV
estimator = KerasClassifier(build_fn=neural,
epochs=5,
batch_size=5,
verbose=0)
# define the grid search parameters
batch_size = [10, 20, 40, 60, 80, 100]
epochs = [10, 50, 100]
init_mode = ['uniform', 'lecun_uniform', 'normal', 'zero',
'glorot_normal', 'glorot_uniform', 'he_normal', 'he_uniform']
output_dim = [10, 15, 20, 25, 30,40]
param_grid = dict(batch_size=batch_size,
epochs=epochs,
output_dim=output_dim,
init_mode=init_mode)
grid = GridSearchCV(estimator=estimator,
scoring= 'accuracy',
param_grid=param_grid,
n_jobs=-1,cv=5)
grid_result = grid.fit(X_train, Y_train)
# summarize results
print("Best: %f using %s" % (grid_result.best_score_,
grid_result.best_params_))
means = grid_result.cv_results_['mean_test_score']
stds = grid_result.cv_results_['std_test_score']
params = grid_result.cv_results_['params']
for mean, stdev, param in zip(means, stds, params):
print("%f (%f) with: %r" % (mean, stdev, param))

Answer 1

There's no error in your code.
Your current param grid has 864 different combinations of parameters possible.
(6 values in 'batch_size' × 3 values in 'epochs' × 8 in 'init_mode' × 6 in 'output_dim') = 864
GridSearchCV will iterate over all those possibilities and your estimator will be cloned that many times. And that is again repeated 5 times because you have set cv=5.
So your model will be cloned (compiled and params set according to the possibilities) a total of 864 x 5 = 4320 times.
So you keep seeing in output that the model is being compiled those many times.
To check if GridSearchCV is working or not, use its verbose param.
grid = GridSearchCV(estimator=estimator,
scoring= 'accuracy',
param_grid=param_grid,
n_jobs=1,cv=5, verbose=3)
This will print the current possible params being tried, the cv iteration, time taken to fit on it, current accuracy etc.