Help with speeding up the excited state optimization calculations

[Dhane]

Hello everyone,

I am trying to run an excited state optimization to get the emission max using NWChem. I tried a few input files with the help of the NWChem documentation and the posted information in this forum. However, my runs are running really slow. 

I installed the NWChem using conda. I am trying to understand if this is due to the installation or any wrong keywords I am using here in the input.

I am planning to install the NWChem from source to use the optimized BLAS and other libraries. But I would be grateful if you can help me to find if I am doing anything wrong in this input file in terms of the keywords.

Any help/advice would be greatly appreciated.

________________________

start tddft_opt

memory total 2000 mb stack 1300 mb heap 100 mb global 600 mb

title "Benzanthrone 2 in pbe0/def2-tzvpbasis set"

geometry units angstroms

C 5.57092200 -1.03649200 -0.25994500 

C 4.91130300 0.17210300 -0.23224400 

C 3.52472200 0.22508200 -0.08950400 

C 2.76982700 -0.95383500 0.02853200 

C 3.46232300 -2.17106400 -0.00158500 

C 4.83371400 -2.21250600 -0.14289900 

C 2.88436300 1.55368300 -0.06601000 

C 1.31692400 -0.88566700 0.17566400 

C 0.67502600 0.37948100 0.20572600 

C 1.41655300 1.58103700 0.09505300 

C 0.78257800 2.80409800 0.13601500 

H 1.39505600 3.69376700 0.04837600 

C -0.60084400 2.88562500 0.29609700 

C -1.34412900 1.73587800 0.39996400 

C -0.73866900 0.46723100 0.34617400 

H 6.64812800 -1.07371400 -0.37041600 

H 5.44146900 1.11282300 -0.31888400 

H 2.92401800 -3.10543100 0.08769500 

H 5.33789600 -3.17228500 -0.16198100 

H -1.08526200 3.85369700 0.34547500 

H -2.41645500 1.79131600 0.54170400 

C -1.51979500 -0.72678000 0.43466300 

C 0.52367900 -2.01042200 0.29280100 

C -0.86239600 -1.93946400 0.42420600

H 0.97520200 -2.99419600 0.27799400 

H -1.44677100 -2.84712400 0.51489800 

O 3.53374800 2.58019300 -0.17462700 

N -2.88612200 -0.67513700 0.60968900 

C -3.70534400 -0.38471600 -0.33322700 

N -5.02146900 -0.21304700 -0.02398400 

C -3.30493700 -0.21158200 -1.77035200 

H -3.60472700 0.76748800 -2.14975400 

H -2.22555000 -0.30371200 -1.86462500 

H -3.77147400 -0.97250000 -2.39996500 

C -5.43819100 -0.45580300 1.33690100 

H -5.53457300 -1.52743100 1.55133400 

H -4.70001800 -0.04713000 2.02367800 

H -6.40329600 0.02543200 1.50119000 

C -6.05895800 -0.27757800 -1.02207700 

H -5.72792400 0.15420000 -1.96358800 

H -6.39403900 -1.30683800 -1.21141800 

H -6.92054700 0.29752500 -0.67834600

end

basis

    H library def2-tzvp

    O library def2-tzvp

    C library def2-tzvp

    N library def2-tzvp

end

dft

semidirect memsize 48000000 filesize 0

noprint "final vectors analysis"

xc pbe0

mult 1

end

driver

clear

maxiter 99

end

cosmo

solvent methanol

end

tddft

nroots 5

notriplet 

thresh 1e-5

maxiter 150

civecs

grad

root 1

end

end

task tddft optimize

task tddft energy

_________________

Thank you very much in advance,

Best,

Dhaneeka 

[jeff.science]

I don't know what Conda does with NWChem, but if it leads to the use of MPI-TS, that's the biggest performance issue.  You need MPI-PR or ARMCI-MPI to get decent performance.

You didn't post your output file so it's impossible to say more.

I am running on 16 cores of AMD 7950X and these are the timings so far, which seem reasonable given the size of this system.

   convergence    iter        energy       DeltaE   RMS-Dens  Diis-err    time
 ---------------- ----- ----------------- --------- --------- ---------  ------
     COSMO gas phase
 d= 0,ls=0.0,diis     1   -993.3502627623 -2.91D+03  1.01D+01  5.36D+01   331.2
 d= 0,ls=0.0,diis     2   -985.3136575154  8.04D+00  1.69D-01  8.64D+01   387.9

Jeff

[jeff.science]

Okay, I made some progress here.  I can't make the NWChem DFT code faster, but I figured out how to improve the rate of convergence.

The first thing you do is run HF/6-31G* to get an initial guess, then project into HF/def2-TZVP, then use that to start the PBE0 iterations.

I also found it helped to converge PBE0 without COSMO first.

I haven't done the optimization, just the TDDFT energy.  Below are the modifications to your input file that I used.  If you want me to share the log files, I can.

...

basis "631Gp"
    * library 6-31G*
end

basis "tzvp"
    * library def2-tzvp
end

set "ao basis" "631Gp"

scf
  direct
  vectors input atomic output dhane_scf_631Gp.movecs

  noprint "final vectors analysis"
end

task scf energy ignore

set "ao basis" "tzvp"

scf
    direct
    vectors input project "631Gp" dhane_scf_631Gp.movecs output dhane_scf_tzvp.movecs

    maxiter 3 # the fourth iteration takes a really long time

    thresh 1e-3

    noprint "final vectors analysis"

end

task scf energy ignore

dft
    vectors input dhane_scf_tzvp.movecs output dhane_pbe0_tzvp.movecs 
    xc pbe0
    mult 1
    noprint "final vectors analysis"
    #semidirect memsize 48000000 filesize 0
    direct
end

task dft energy ignore

driver
    clear
    maxiter 99
end

cosmo
    solvent methanol
end

dft
    vectors input dhane_pbe0_tzvp.movecs 
    xc pbe0
    mult 1
    noprint "final vectors analysis"
    #semidirect memsize 48000000 filesize 0
    direct

end

tddft
    nroots 5
    notriplet
    thresh 1e-5
    maxiter 150
    civecs
    grad
        root 1
    end
end

task tddft energy
#task tddft optimize

[Dhane]

Dear Jeff,

Thank you very much for your prompt response and for the help in testing my input file. I appreciate it very much. 

I checked my log file for the original input file I posted. Beyond the 4th convergence iteration it takes around 1500-5000 time units.

Thank you very much for posting this work around for the calculation. I will try this option and see how it works.

Yes sure, can you send your log files so I can compare with mine when I run them.

Thank you very much for your help,

Best,

Dhaneeka

[Edoardo Aprà]

You can install the faster MPI-PR NWChem variant from conda with the command

micromamba install -c conda-forge -y nwchem=*=*mpi_pr*

The only suggestion I have for the original input file is to change the basis set from the default cartesian to spherical

https://nwchemgit.github.io/Basis.html#spherical-or-cartesian

By doing so, the code should run faster and be more numerically stable.

In other words, the line

basis

should be changed to

basis spherical

Other than this, the input looks good to me.

[Dhane]

Dear Edoardo,

Thank you very much for suggesting another solution for my issue. I appreciate it very much. I installed NWChem through conda including the mpi-pr and it is running now. I used 8 processors in the mpirun. However, only 7 is actually used in the run. It is also shown in the output file as 7. I am wondering how it happens.

I also had another question with the NWChem installation with conda. I was trying to see if my installation with conda actually using optimized blas, which is openBLAS, Lapack and scalarpack.

May I know how to find that? When I install the NWChem through conda do they use the optimized openBLAS, Lapack and scalarpack or do I need to set the environment variables before the nwchem installation?

I found this build.sh in the here https://github.com/conda-forge/nwchem-feedstock/blob/main/recipe/build.sh

Is it the one actually used in the installation?

[Edoardo Aprà]

When using MPI-PR, one process is used for communication instead of computation on each node https://nwchemgit.github.io/ARMCI.html

NWChem is built on Conda using OpenBLAS and Scalapack and https://github.com/conda-forge/nwchem-feedstock/blob/main/recipe/build.sh is used to generate the conda packages

[Ravithree Senanayake]

Dear Edoardo,

Thank you very much for your explanation and for your help. Much appreciated.

Thank you,

Best,

Dhaneeka

--
You received this message because you are subscribed to the Google Groups "NWChem Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nwchem-forum...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nwchem-forum/752ef15c-67ba-4e2a-8ce2-12a21c678fc9n%40googlegroups.com.