BfgsMultiDimensions optimizer

[Vincent Cahais]

Hi everyone,

I'm trying to optimize a likelihood function with BFGS. I have two models, M0 with 1 parameter to optimize and M1 with three parameters to optimize.
The optimization works fine with M0 (same result as in R). But with M1 result is slightly different while likelihood function return the good result with parameters chosen by R.

In both models, parameters values returned doesnt correspond to the optimized value. I think it's because i use ReparametrizationFunctionWrapper, how to retrieve the good values ?

Vincent

[Laurent Guéguen]

Hi Vincent,

and is this bug specific with BFGS optimizer? Have you tried with
other
optimizers?

L

[Julien Yann Dutheil]

Hi Vincent,

May I additionally ask what is the code you used to retrieve the
optimized parameter values?

Cheers,

J.

> --
> You received this message because you are subscribed to the Google Groups "Bio++ Usage Help Forum" group.
> To post to this group, send email to biopp-he...@googlegroups.com.
> To unsubscribe from this group, send email to biopp-help-for...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--
Julien Y. Dutheil, Ph-D
0 (+49) 6421 178 986

§ Max Planck Institute for Terrestrial Microbiology
Department of Organismic Interactions
Marburg -- GERMANY

§ Intitute of Evolutionary Sciences - Montpellier
University of Montpellier 2 -- FRANCE

[Yifei Huang]

I came across some similar problems actually. I think the problem with the BFGS optimizer in Bio++ is that it does not perform a linear search. In PAML, the BFGS (ming2 function) function performs a linear search along a reasonable direction specified by the gradient and the (approximated) hessian matrix. This should be the standard algorithm. In Bio++, the gradient and the (approximated) hessian matrix are used for a "one step" optimization, which frequently leads to a higher negative log likelihood and then the optimization fails. You might try PseudoNewtonOptimizer in Bio++, which is better in my view.

Yifei

Yifei Huang
Department of Biology
McMaster University

[Cahais Vincent]

Thanks for your answers,

here is my code :   

    Logvrais1 log1(&mystat,0.001);
    ReparametrizationFunctionWrapper rpf1(&log1,false);
    ParameterList pl1 = rpf1.getParameters();
    ThreePointsNumericalDerivative tpnd1(&rpf1);
    tpnd1.setParametersToDerivate(pl1.getParameterNames());
    BfgsMultiDimensions optim1(&tpnd1);
    optim1.setVerbose(0);
    optim1.setProfiler(0);
    optim1.setMessageHandler(0);
    optim1.setConstraintPolicy(AutoParameter::CONSTRAINTS_AUTO);
    pl1.setParameterValue("xi",0.8028201);
    pl1.setParameterValue("p1",0.999);
    pl1.setParameterValue("p2",0.001);
    optim1.init(pl1);
    double ml1 = optim1.optimize();
    cerr << "ml1=" << ml1 << "\tin R : 105.8411" << endl;
    cerr << "p1=" << optim1.getFunction()->getParameterValue("p1")<< "\tin R : 0.5999006" << endl;
    cerr << "p2=" << optim1.getFunction()->getParameterValue("p2")<< "\tin R : 0.6335156" << endl;
    cerr << "xi=" << optim1.getFunction()->getParameterValue("xi")<< "\tin R : 0.8031929" << endl;
    //calcul of likelihood whith R returned parameters values
    cerr << "logvrais1=" << mystat.logvrais1(0.5999006, 0.6335156, 0.8031929) << " in R : 105.8411" << endl;

It returns :

ml1=112.064    in R : 105.8411
p1=-10.115    in R : 0.5999006
p2=8.7112    in R : 0.6335156
xi=-48.6576    in R : 0.8031929
logvrais1=105.841 in R : 105.8411

According to parameters constraints, p1, p2 and xi can't be negatives :
addParameter_(Parameter("p1", prec, new IncludingInterval(prec, 1-prec, prec)));
addParameter_(Parameter("p2", 1-prec, new IncludingInterval(prec, 1-prec, prec)));
addParameter_(Parameter("xi", 0.5, new IncludingInterval(0.5, 1-prec, prec)));

PseudoNewtonOptimizer return the good value for ml1, but not for parameters, in addition with a fourth parameter it also return a wrong result.

Vincent

[Yifei Huang]

Hi Vincent,

Sorry that I misunderstood your first email. You used the ReparametrizationFunctionWrapper class so the parameters in object rpf1 are transformed to remove constraints. Therefore, when you called optim1.getFunction()->getParameterValue("p1"), you got the transformed p1 instead of the original one. Can you try to get the parameters by something like log1.getParameterValue("p1")?

Yifei

[Vincent Cahais]

Hi,

log1.getParameterValue("p1")  returns the initialized values (0.999).

Vincent