Should I do learnLayerUnsupervised before learnLayerSupervised?

[dosxp]

Hi, I am learning to use Cesar's DNN sanple.

Should I pre-train the network by use learnLayerUnsupervised or learnNetworkSupervised function before I use learnLayerSupervised function?

[César]

Hi there,

It would be advisable to pre-train the network using learnLayerUnsupervised first. This method will train the network as if it were just a stack of Restricted Boltzmann Machines, and will constrain visible and hidden weights to have the same value.

Afterwards, when you call learnNetworkSupervised you will be essentially just performing gradient descent using Backpropagation. The network will be seen as a simple, multi-layer activation network, and no constraints will be enforced.

Hope it helps!

Best regards,

Cesar

[dosxp]

thanks for your reply.

If I have a network with layers(255,30,4), I should train layer 1 (hidden) and layer 2 (output) at first, am I right? 

I met a problem with this network. My inputs have 1000 samples, learnnetworksupervised function can reach

low error level such as 3% after 150 run, but learnlayersupervised function always stay high error level such as 

30% even after 50000 run. Do you have any advise for me to make learnlayersupervised reach a lower error level?

[dosxp]

And, I use doeble[]{1,0}, doeble[]{1,1} etc as outputs to train the network, but I got output like doeble[]{0.56,0.36} when I use network.compute(inputs[][]).

Is there anything wrong? I expect that the output value should be 1/0.

[César]

Hi there,

I forgot to add that usually you should not learn the last layer (the output layer) using learnLayerUnsupervised. You can either leave this layer untrained and just call learnNetworkSupervised, or you can add another layer to be the final, actual output layer and then call learnNetworkSupervised.

By the way, since you specified 255 inputs, 30 neurons in the hidden layer, and 4 neurons in the output layer, it will indeed generate output vectors containing 4 values. If you wish your network's output to be binary (i.e. 1 or 0), then what you can do is what I mentioned above: add another layer with only a single neuron, and call learnNetworkSupervised directly. Please see if it helps!

Best regards,

Cesar

[dosxp]

thanks.