SO,QN=0 MCSCF solver sometimes performs much better than the defaults

[Tibor Győri]

Dear Molpro community,

Here I would like to share an example where the default solvers for MCSCF and DF-MCSCF have much worse convergence than SO,QN=0.

1) MCSCF

If I understand correctly, MCSCF defaults to the WMK second-order solver with the L-BFGS acceleration activated. As seen in the attached outputs (default.out, default_fail.out), this results in chaotic convergence. Re-running the same input multiple times sometimes results in convergence after 20+ iterations, but sometimes convergence is not achieved within the hardcoded maximum of 40 iterations.

The run-to-run variability suggests that something is numerically unstable in L-BFGS, allowing the small non-deterministic noise of parallel execution to have a large impact. Either way, I think it can be said that WMK + L-BFGS does not work well in this specific case.

Disabling the L-BFGS acceleration with QN=0 solves the problem completely, and the WMK solver converges reliably in 6 iterations. (noQN.out)

2) DF-MCSCF

DF-MCSCF seems to default to the combined first- and second-order optimization method, SO-SCI. This struggles to converge even more than the default solver of non-DF MCSCF, and reproducibly does not converge in 40 iterations. (DF-default.out)

Enabling second order solvers with SO fixes this, and convergence is reliably achieved in 6 iterations. Curiously, unlike non-DF MCSCF,  L-BFGS works perfectly fine here and saves a little bit of time. (DF-SO.out, DF-SOnoQN.out)

Based on these results, it looks like the default solvers of MULTI can have poor convergence in some edge cases, in both DF and canonical computations.

The first step should of course be to verify that the wf, occ, etc. directives are correct, but perhaps trying SO,QN=0 should be a recommended course of action for resolving MCSCF convergence issues.

Best regards,

Tibor Győri