access one- and two- electron integrals from a DFT calculation

[Oleksii]

Hi everyone,

Can anyone suggest if there is a way to output the one- and two-electron integrals from a DFT calculation? These are the core Hamiltonian, Coulomb and exchange integrals.

I have tried to run the calculation with the option "print graddebug fcore", as was previously suggested on the forum, but the output in the .out file was identical to the one from the calculations with default printing settings. Also, in the .out file, it is written:

"Integral file          = ./dft_calc.aoints.0"

However, this file is absent in my working folder, and I suspect it was deleted by the program right after the calculation as a temporary file. 

I was wondering if there is any way to keep this file in the folder and access it after the run has finished? Alternatively, is there any way to output the integrals directly to the .out file?

Your help will be highly appreciated.

Best regards,

Oleksii

[jeff.science]

Try https://nwchemgit.github.io/FCIDUMP.html

[Edoardo Aprà]

Keep in mind that FCIDUMP currently writes the integrals that are transformed using the molecular orbitals. 

https://nwchemgit.github.io/FCIDUMP.html#orbitals-orbitals-specifications

[Oleksii]

Dear Edoardo and Jeff,

Many thanks.

I need the one-electron Hamiltonian matrix in the atomic orbitals basis set; do I understand correctly that I need to do the following:

If |Psi^MO>=C*|psi^AO>, where C is a matrix of expansion coefficients (constructed from the "MO coefficients analysis" section), then the Hamiltonian in the atomic orbital basis set will be:

H^AO=(S^-1)*(C^T)*H^MO*C*(S^-1),

where C^T is a transposed expansion coefficient matrix, and S^(-1) is the inverse overlap matrix.

Is the procedure above correct, or did I miss something?

Best regards,

Oleksii

середа, 5 лютого 2025 р. о 00:40:31 UTC+2 Edoardo Aprà пише:

[jeff.science]

I can't remember if there is already an AO dump command, but it's easy for me to implement this.

Do you need this in a specific file as binary or can it be written to stdout?

Jeff

[jeff.science]

It seems "atomic" is an option for fcidump but the docs say it is not implemented yet.  I will try to finish it (or correct the docs if someone else already finished it).  https://nwchemgit.github.io/FCIDUMP.html#orbitals-orbitals-specifications

I'm pretty busy right now so it might take a week or two...

Jeff

[Oleksii]

Dear Jeff,

Thanks a lot. I would appreciate it if this module will be implemented.

At the same time, could you please give me some hints if the strategy I mentioned above is correct? In that case, my group members could write a temporary script for matrix conversion while the main module is being prepared. In that case, we could forward with our simulation analysis.

Thanks and best regards,

Oleksii

середа, 5 лютого 2025 р. о 18:58:28 UTC+2 jeff.science пише:

[Jeff Hammond]

I think what you wrote is right but it's wasteful to do a O(N^5) 4-index transform to get the MOs and then do another O(N^5) transform to get back to the AOs, which are only O(N^4) to generate in the first place.

My goal is to write this code over the weekend.  I don't know if I'll get the format right on the first try, but if you are flexible on parsing, it will be fine.

Jeff

--
You received this message because you are subscribed to a topic in the Google Groups "NWChem Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nwchem-forum/uUjemdqLeiI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nwchem-forum...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/nwchem-forum/73ce969e-a549-4421-8925-d539d41b7f2dn%40googlegroups.com.

--

Jeff Hammond
jeff.s...@gmail.com
http://jeffhammond.github.io/

[Oleksii]

Dear Jeff,

I see, in that case, I agree that converting the matrices back is not reasonable because of the precision loss.

Many thanks, we would very much appreciate the new module.

Best regards,

Oleksii

пʼятниця, 7 лютого 2025 р. о 17:14:33 UTC+2 jeff.science пише:

[Jeff Hammond]

I thought we had this already and we did, but it took me too long to find it.

You can add print "kinetic" "potential" "overlap" "ao2eints" to your SCF block to dump the integrals to stdout.

If you want this reformatted, it's easy enough to modify the code.  Look at src/ddscf/print_*

If you need other information, src/util/nwmolden.F documents programmatic access to basis sets, etc.

Jeff

echo

start h2o_integrals

memory stack 2000 mb heap 100 mb global 2000 mb noverify

permanent_dir .
scratch_dir /tmp

geometry units angstrom
  O      0.00000000     0.00000000     0.11726921
  H      0.75698224     0.00000000    -0.46907685
  H     -0.75698224     0.00000000    -0.46907685
end

basis "ao basis" spherical noprint
  * library cc-pvdz
end

scf
  rhf
  singlet
  thresh 1e-7
  maxiter 100
  noprint "final vectors analysis" "final vector symmetries"
  print "kinetic" "potential" "overlap" "ao2eints"
end

task scf energy

To view this discussion visit https://groups.google.com/d/msgid/nwchem-forum/54c9c2cf-f9c9-43a3-93c5-fa02080ff525n%40googlegroups.com.

[Oleksii]

Dear Jeff,

Many thanks for your timely help.

This was exactly what I was looking for.

Best regards,

Oleksii

субота, 8 лютого 2025 р. о 17:23:46 UTC+2 jeff.science пише:

[Oleksii]

Dear Jeff,

Thanks again for your advice. However, we are facing now another problem. With a large number of basis functions (3000 AO in our case), the direct assessment of the integrals becomes useless, as we have too many of them and are running out of memory very quickly. I was wondering if any effective ways in NWChem to reduce the number of one-electon integrals in the output? For example, with the resolution of identity approximation, etc.

Thanks again and best regards,

Oleksii

понеділок, 10 лютого 2025 р. о 14:33:12 UTC+2 Oleksii пише:

[Oleksii]

Dear Jeff,

Apologies for the typo in my last message: it was concerning the two-electron integrals, of course.

My task is to compute expressions (required for electron transfer calculations with the method of tunnelling currents) which contain a linear combination of the Fock matrix and two-electron integrals (for example, expression 6.8 here: https://pubs.aip.org/aip/jcp/article-abstract/118/17/7898/185389/Tunneling-currents-in-long-distance-electron?redirectedFrom=fulltext ). I fully understand the memory bottleneck. However,  all these integrals are anyway computed during the SCF calculation, the only problem is storing all of them at the same time. I was wondering if you could give me some hints on whether I can access the integrals directly during the calculation, in order to compute the above expressions. I fully understand that it might be a challenging task.

Another (trivial) possibility I was thinking of is to decrease the printing threshold in "print_aoints.F":

 if (abs(g(l,k,j,i)).gt.1d-8) then
                     write(6,1) i,j,k,l,g(l,k,j,I)

Thus, setting it to several orders of a magnitude lower results in a smaller number of integrals in the .out file. However, re-computing all of them is a very long process which takes an unreasonable amount of time; non-comparable with the relatively fast SCF calculation.

I would very much appreciate your advice on calculating the linear combination of two-electron integrals.

Thanks and best regards,

Oleksii

вівторок, 25 лютого 2025 р. о 00:46:45 UTC+2 Oleksii пише:

[Jeff Hammond]

I suspect that I can hack the code to let you call NWChem to generate a subset of 2-e AOs every time you call it, which would let you use it like a semidirect integral engine, but that's not particularly efficient.  At the very least, you should first switch from text I/O to binary I/O, since that will be quite a bit more efficient.  I think the current code uses 45 bytes per integral, and a binary version would use 24 bytes (8 if we don't print the indices).  Also, writing them to a binary file should be faster than printing to standard out, due to buffering differences.

I can reduce the threshold (I'll just make it configurable), but then you can't really use the AOs for anything accurate.  Before you go that route, you ought to consider just developing your code inside of NWChem so that you can access these objects directly.

The other option here is to use a code like PySCF that can be scripted almost arbitrarily.  (NWChem has a Python interface but I'm not aware of features for accessing integrals, at least with the same ease as PySCF.)

Jeff

To view this discussion visit https://groups.google.com/d/msgid/nwchem-forum/312e0ace-21c6-4ef2-95cb-1de1011d164an%40googlegroups.com.

[Oleksii]

Dear Jeff,

Many thanks.

Changing the output of "print_aoints.F" to the binary format is also an option, but, as you also mentioned, it will require decreasing a threshold and the accuracy could go down dramatically. The "print_aoints" function outputs it to .stdout. Is there any function which does it to binary?

If you could make it possible to access the 2-electron integrals from the last SCF step, the same ones that were used for the energy calculation (without printing them to a file), that would be very helpful. In that case, as you mentioned, I can write my own function (a linear combination of a Fock matrix, 1- and two-electron integrals). I would appreciate it very much.

PySCF seems not to have a well-tested constraint DFT module, so it would be more reasonable to try that with NWchem, if possible.

Thanks and best regards,

Oleksii

вівторок, 25 лютого 2025 р. о 14:06:17 UTC+2 jeff.science пише:

[Jeff Hammond]

The AOs are a function only of the basis set and geometry so they do not vary throughout the SCF solver.  However, if you are also consuming the Fock matrix, then I suppose it matters.

I haven't done binary I/O in Fortran before, only C, but I know it's not hard and I'll figure it out if that's necessary.

If you are using constrained DFT, then I suppose NWChem is the way to go.

I do think you'll be better off learning to develop within NWChem, versus dumping quantities to standard out.

Jeff

To view this discussion visit https://groups.google.com/d/msgid/nwchem-forum/96f0212a-4725-469a-bdc0-26825f462eeen%40googlegroups.com.

[Oleksii]

Dear Jeff,

I was wondering if you had a chance to look at the binary output functions for 2-electron integrals?

Another topic we discussed is how to access two-electron integrals as they are evaluated during the NWChem calculation. This would allow us to use them efficiently in expressions involving the Fock matrix and two-electron integrals. If you could point to the relevant functions in the source code, I would insert the necessary expressions, so that they are computed on-the-fly, avoiding any additional computational cost.

I completely understand that you might be currently very busy, but if you have a moment to look into this, we would greatly appreciate your help.

Best regards,

Oleksii

середа, 26 лютого 2025 р. о 06:41:06 UTC+11 jeff.science пише:

[Jeff Hammond]

src/ddscf/scf.F is the Hartree-Fock driver.  From there you find calls to fock_2e, most likely.

I haven't had any time to work on binary output.

Jeff

To view this discussion visit https://groups.google.com/d/msgid/nwchem-forum/b2ea70f5-769b-47e2-9832-e3c0379a8fd5n%40googlegroups.com.