K-fold Cross validation in Keras, do you need to create a new model for each fold?

[ravi...@personalis.com]

Hi,

I am trying to run 4-fold cross validation in Keras. I use sklearn cross_validation module for this. My question is, for each fold of training, do I have to create a new model? Can I create the model once, outside the loop and then call fit on the model 4 times? I am unsure if this paradigm causes the model to store learnt weights across the folds.

inner_cv = cross_validation.LabelKFold(groups, 4)
nn_model = create_model(params)

early_stopping = EarlyStopping(monitor='val_loss', patience=10, verbose=0, min_delta=0.0001)



for inner_train, inner_stop in inner_cv:
 nn_model.fit(x[inner_train], y[inner_train], batch_size=256, nb_epoch=500, callbacks=[early_stopping],validation_data=(x[inner_stop], y[inner_stop]), verbose=0)
 y_pred = nn_model.predict(x[inner_stop])
 comb_score = combined_score(y[inner_stop], np.ravel(y_pred))

[madera...@gmail.com]

Your model keeps all the weights, so if you want to compare the different splits independently for cross validation, you need to create a new model each time. 

inner_cv = cross_validation.LabelKFold(groups, 4)

early_stopping = EarlyStopping(monitor='val_loss', patience=10, verbose=0, min_delta=0.0001)

comb_score = []

for inner_train, inner_stop in inner_cv:

 nn_model = create_model(params)

 nn_model.fit(x[inner_train], y[inner_train], batch_size=256, nb_epoch=500, callbacks=[early_stopping],validation_data=(x[inner_stop], y[inner_stop]), verbose=0)
 y_pred = nn_model.predict(x[inner_stop])

 comb_score.append(combined_score(y[inner_stop], np.ravel(y_pred)))


mean_score = numpy.mean(comb_score)
std_deviation = numpy.std(comb_score)