Neural Network Train loss increases while training.

[Zohair Zahid]

I am using a sinmple Neural network to classify characters but the perfomance is vey poor and the train loss also start to increase after several epochs. Any suggestions to improve. Below is the code for the configuration of Neural network.

net1 = NeuralNet(
    layers=[  # three layers: one hidden layer
        ('input', layers.InputLayer),
        ('hidden', layers.DenseLayer),
        ('output', layers.DenseLayer),
        ],
    # layer parameters:
    input_shape=(None, 400),  # 20x20 input pixels per batch
    hidden_num_units=300,  # number of units in hidden layer
    output_nonlinearity=lasagne.nonlinearities.softmax,  # output layer uses identity function


    output_num_units=60,  # 1 target values

    # optimization method:
    update=nesterov_momentum,
    update_learning_rate=0.01,
    update_momentum=0.9,

    regression=False,  # flag to indicate we're dealing with regression problem
    max_epochs=100,  # we want to train this many epochs
    verbose=1,
    )

[Sander Dieleman]

I don't see the training loss decreasing in the graph you posted? It's hard to see but it looks like the validation loss is increasing, which indicates overfitting.

Sander

[Zohair Zahid]

I am sorry i meant validation loss. How to tackle the problem of overfitting. Should I decrease the number of neurons in the hidden layer? or any other suggestion ?