Select the best number of convolutional and dense layers using Hyperopt

[Lucas Lins]

Hi, I have a question about the use of the hyperopt with convolutional neural networks.

Is it possible to use it to select the best number of convolutional and dense layers in a achitecture? I'm trying in the following way:

hyperopt_parameters = {
    'choice1': hp.choice('num_layers',
                    [ {'layers':'one'},
                    {'layers':'two',
                              'filter2': hp.choice('filter2', [32, 64, 128])}
                    ]),
    'choice2': hp.choice('num_layers2',
                    [
                      {'layers':'one'},
                      {'layers':['one','two']},
                      {'layers':'three',
                                'filter3': hp.choice('filter3', [32, 64, 128])}
                    ]),
    'choice3': hp.choice('num_layers3',
                    [
                      {'layers':'one'},
                      {'layers':['one','two']},
                      {'layers':['one','two','three']},
                      {'layers':'four',
                                'filter4': hp.choice('filter4', [32, 64, 128])}
                    ]),
    'choice4': hp.choice('num_layers4',
                    [
                      {'layers':'one'},
                      {'layers':['one','two']},
                      {'layers':['one','two','three']},
                      {'layers':['one','two','three','four']},
                      {'layers':'five',
                                'units1': hp.choice('units1', [4, 8, 16, 32, 64])}
                    ]),
    'l2_drop': hp.choice('l2_drop', [0.0, 0.5, 1]),
    'batch_size': hp.choice('batch_size', [10, 20]),
    'epochs': hp.choice('epochs', [5, 10]),
}

def function(args):
   
    model = Sequential()
   
    #1
    model.add(Conv2D(32, kernel_size=(3, 3),
                     input_shape=input_shape))
    model.add(Activation('relu'))
   
    #2
    if args['choice1']['layers']== 'two':
        model.add(Conv2D(filters = args['choice1']['filter2'], kernel_size=(3, 3)))
        model.add(Activation('relu'))
   
    #3
    if args['choice2']['layers']== 'three':
        model.add(Conv2D(filters = args['choice2']['filter3'], kernel_size=(3, 3)))
        model.add(Activation('relu'))
       
    #4
    if args['choice3']['layers']== 'four':
        model.add(Conv2D(filters = args['choice3']['filter4'], kernel_size=(3, 3)))
        model.add(Activation('relu'))    
   
    model.add(Flatten())
   
    #3
    if args['choice4']['layers']== 'five':
        model.add(Dense(units = args['choice4']['units1']))
        model.add(Activation('relu'))
        model.add(Dropout(args['l2_drop']))
   
    model.add(Dense(num_classes, activation='softmax'))
   
    model.summary()

    model.compile(loss='binary_crossentropy',
                    optimizer=Adam(),
                    metrics=['accuracy'])
        #model.summary()

    model.fit(x_train, y_train,
                batch_size=args['batch_size'],
                epochs=args['epochs'],
                verbose=1,
                validation_split=0.2)
    evaluation = model.evaluate(x_test, y_test, batch_size=args['batch_size'], verbose=0)
    return {'loss': -evaluation[1], 'status': STATUS_OK, 'model': model}

What shows me that is right is the best_model.summary() print, but the space_eval(hyperopt_parameters,best) print sometimes is wrong, like this one:

{'batch_size': 20,
 'choice1': {'filter2': 128, 'layers': 'two'},
 'choice2': {'filter3': 128, 'layers': 'three'},
 'choice3': {'filter4': 32, 'layers': 'four'},
 'choice4': {'layers': 'one'},
 'epochs': 5,
 'l2_drop': 1}

In choice4, it should have printed one, two, three, four, and not one

So, I'd like to know that if it's possible to check the best number of layers too 

besides the number of filters, if yes, if i'm doing correct.

Thanks, and congratulation for the work in hyperopt.