Deep neural networks learning sample application

[César]

Hi all,

I just wanted to announce that I have included the deep learning sample application in the repository. It can be either browsed online at 

  https://code.google.com/p/accord/source/browse/trunk#trunk%2FSamples%2FNeuro%2FDeep

or you may also want to checkout the repository directly using 

  https://accord.googlecode.com/svn/trunk/

The sample application will included in the next release, but since there seems to be some immediate interest on it and I am not sure when a new release will be available, I am therefore including the SVN paths here.

Hope you will find it interesting!

Best regards,

Cesar

[mu li]

Hi,

Thanks very much!!!

I am very looking forward it!

[mu li]

hi,

Does it need windows 8?

I am using win7 maybe not supported...

[César]

No, it does not needs Windows 8. You can run it on Windows 7.

[mu li]

Hi Cesar,

I think I missed so many third party dll file...

would you please add them into your project file.

the lib link address in your project file is static linked to you D:\XXXXX

can you make a little modification(include all external add in) that everyone can use it easier.

Many many thanks!

[mu li]

Sorry Sesar, 

I finally find those files in your package....

I will try to make a successful build.

[mu li]

Hi,,,,

I got it runs!

but I still hope you to modify the .proj file to indicate all the dll file without any adjust.

Thanks!

[César]

Hi mu!

Certainly, the next release will have this sample application proper configured. Sorry about the mess :-)

Best regards,

Cesar

[mu li]

Hi Cesar,

I think for hand writing digits recognition task by using the pure pixel is not a very good idea.

I try to use the application to recognized my writing digit, the result is not good. But if I try to write the digits like the training set the result is perfect.

I am not sure about if the training set is made by you or it's public data. I think I can provide a more adaptable method to deal with any writing style.

If I have free time I will make one and provide it to you just as myself practice of using you wonderful deep learning code.

Anyway, This work is just for showing the powerful of deep learning maybe you not just want it to do a perfect digits recognition job.

And do you have any plan in the future with the deep learning framework? I am very happy if you share that with me, I think I can get some help for you.

Cheers!

[pene...@gmail.com]

Hi Cesar!

The sample app looks really great.
There is an application called Visual-RBM which can train RBMs at a really high speed using the GPU. It can export out the results in the following format:
float[v] Visible biases
float[h] Hidden biases
float[v * h] Weights

In this way I can get all the layers trained unsupervised using Visual-RBM. Could you point me in the right direction to import these trained parameters into the Accord.NET application? What can you think of what objects/classes should be modified so I can import these trained RBMs?

Thank you for your help,
Peter

[César]

Hi Peter,

Very cool, I wasn't aware of such application! I hope the following can help!

    int v = 0, h = 0;
    float[] visibleBiases = new float[v];
    float[] hiddenBiases = new float[h];
    float[,] weights = new float[v, h];

    // Create a new deep belief network with v inputs and h outputs
    DeepBeliefNetwork network = new DeepBeliefNetwork(inputsCount: v, hiddenNeurons: h);

    // Load visible thresholds (often are negative of the bias)
    StochasticLayer visibleLayer = network.Machines[0].Visible;
    for (int i = 0; i < visibleBiases.Length; i++)
        visibleLayer.Neurons[i].Threshold = -visibleBiases[i];

    // Load hidden thresholds (often are negative of the bias)
    StochasticLayer hiddenLayer = network.Machines[0].Hidden;
    for (int i = 0; i < hiddenBiases.Length; i++)
        visibleLayer.Neurons[i].Threshold = -hiddenBiases[i];

    // Load the weights into the hidden layer
    for (int i = 0; i < v; i++)
        for (int j = 0; j < h; j++)
            hiddenLayer.Neurons[i].Weights[j] = weights[i, j];


    // Copy the reversed weights from the hidden layers. This makes
    // the weights tied so inputs can be reconstructed from outputs.
    network.UpdateVisibleWeights();

Please note, however, that this network assumes Bernoulli-style activation functions. There could be some other parameters which also have to be matched... But the above should give a start.

If you need any further help, please ask!

Best regards,

Cesar

[César]

Oops.. I've just seen I made a mistake when copy and pasting some of the lines. The line after "// Load hidden thresholds" should have been "hiddenLayer.Neurons" instead of "visibleLayer.Neurons". I didn't actually run the code, but it should be something very similar.

Regards,

Cesar