BFGS

[adamenk...@gmail.com]

Hi. I'm have beural net with backpropagation learning. I view in Statistica BFGS algoritm, but dont know how use him. Where i should call this method, in my program ? In derivative compute?

[César]

Hi there,

Instead of using BFGS, you may want to use Levenberg-Marquardt, which is seldom a superior alternative to BFGS. It is available in the framework as the LevenbergMarquardtLearning class.

Hope it helps!

Best regards,

Cesar

[Artem Adamenko]

Hi. Thanks for response ) LevenbergMarquardtLearning using for classification ? 20 input and 2-3 classes, for example?

[Artem Adamenko]

help with BFGS

[César]

Hi there,

I have created an example for multi-class classification to put in the documentation. Please update your libraries to at least v2.14.3-alpha in order to use some of the methods included in the example!

int numberOfInputs = 3;
int numberOfClasses = 4;
int hiddenNeurons = 5;


double[][] input =
{
    new double[] { -1, -1, -1 }, // 0
    new double[] { -1,  1, -1 }, // 1
    new double[] {  1, -1, -1 }, // 1
    new double[] {  1,  1, -1 }, // 0
    new double[] { -1, -1,  1 }, // 2
    new double[] { -1,  1,  1 }, // 3
    new double[] {  1, -1,  1 }, // 3
    new double[] {  1,  1,  1 }  // 2
};


int[] labels =
{
    0,
    1,
    1,
    0,
    2,
    3,
    3,
    2,
};


double[][] outputs = Accord.Statistics.Tools
    .Expand(labels, numberOfClasses, -1, 1);


var function = new BipolarSigmoidFunction(2);
var network = new ActivationNetwork(function,
    numberOfInputs, hiddenNeurons, numberOfClasses);


new NguyenWidrow(network).Randomize();


var teacher = new LevenbergMarquardtLearning(network);


double error = Double.PositiveInfinity;
for (int i = 0; i < 10; i++)
    error = teacher.RunEpoch(input, outputs);


for (int i = 0; i < input.Length; i++)
{
    int answer;
    double[] output = network.Compute(input[i]);
    double response = output.Max(out answer);


    int expected = labels[i];
    // At this point, the contents of the variable 'answer' will be the same as 'expected'.
}

Hope it helps! I am uploading the new documentation soon.

Best regards,

Cesar