FFTW "Wisdom" support removed?

[M. Brehm]

Dear Community,

in older versions of CP2k, there was a small script called "cp2k-wisdom" in the tools subdirectory of the CP2k trunk, which was able to produce planning data for FFTW, such that CP2k runs were able to profit from optimal algorithms for a given machine without the need to use "FFTW_PLAN_TYPE PATIENT" each time.

However, this script was removed from the trunk some time ago, and there seems to be no replacement. Why was it removed? Was there some problem with Wisdom in CP2k? I would like to use this feature for my calculations. Would it be a good idea to just use the old version of the script from an older revision?

Best regards,
Martin

[Iain Bethune]

Hi Martin,

The cp2k-wisdom script was removed because it was unmaintained and (to our knowledge) unused. I had a look at it and it appears to generate only wisdom for 3D FFTs which is not usually what you want for parallel runs, and also only support cubic cells.

In any case, CP2K will at the end of a run write a wisdom file for you to the location specified in GLOBAL%FFTW_WISDOM_FILE_NAME, which defaults to /etc/fftw/wisdom. This will contain wisdom for the FFT sizes and types that are relevant for your calculation. So best thing to do it to set this to be somewhere writable for non-root users, then run once with FFTW_PLAN_TYPE PATIENT or even EXHAUSTIVE. Then subsequent runs you should find the planning time significantly reduced, as the wisdom file should be read. Let me know if it’s not working for you!

- Iain

--

Iain Bethune
Project Manager, EPCC

Email: ibet...@epcc.ed.ac.uk
Twitter: @IainBethune
Web: http://www2.epcc.ed.ac.uk/~ibethune
Tel/Fax: +44 (0)131 650 5201/6555
Mob: +44 (0)7598317015
Addr: 2404 JCMB, The King's Buildings, Peter Guthrie Tait Road, Edinburgh, EH9 3FD

> --
> You received this message because you are subscribed to the Google Groups "cp2k" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+uns...@googlegroups.com.
> To post to this group, send email to cp...@googlegroups.com.
> Visit this group at http://groups.google.com/group/cp2k.
> For more options, visit https://groups.google.com/d/optout.


--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

[M. Brehm]

Hi,

thank you, this sounds very useful and interesting. I did not even know about this feature. I was running CP2k many times as root, so it always wrote this wisdom file, and I never noticed it :-) I will try this out.

Can you tell me on which factors it depends which sizes of FFTs CP2k performs? I assume there are many such factors (basis set, atom count, and even number of MPI processes...). So this Wisdom file which is written for a certain run will then only be useful for the exactly same calculation of the same system, right?

Best regards,
Martin

[Iain Bethune]

Hi Martin,

The FFT size depends directly on the number of points on the planewave grids for Quickstep calculations. It will also be used for ewald summation in classical or QM/MM simulations, and for various other things. For QS, if you turn the print level up from the default of MEDIUM, you will see a section which shows exactly which grid configurations are used. Typically there are 4 grid levels, the finest is maybe 125 - 300 grid points across, and the coarsest could be as little as ~10-20. This depends on a number of factors, including the size of the simulation box, the accuracy of gaussian-grid mapping (see http://www.cp2k.org/howto:converging_cutoff for discussion of the control of this, and the MGRID section of the input: http://manual.cp2k.org/trunk/CP2K_INPUT/FORCE_EVAL/DFT/MGRID.html), the sharpness or smoothness of basis functions in your system. Once the grid sizes are chosen (say x1*y1*z1 .. x4*y4*z4) then the FFTs can be planned.

For a parallel 3D FFT each dimension is done independently, with a parallel transpose of the data between successive 1D FFTs. Since the length of the 1D FFT depends only on the grid size, not the decomposition (this only affects how many 1D FFTs are performed per process), then the wisdom file should be portable between calculations on the same grid size.

From my understanding of FFTW, wisdom is accumulated i.e. start with no wisdom, run a calculation (saving wisdom to a file), then run another calculation with different grid (read old wisdom, save wisdom to file), after the second run the wisdom file contains wisdom for the transform lengths used by both calculations. So you should probably just set FFTW_WISDOM_FILE_NAME to some file in e.g. your home directory, and over time you will build up wisdom for a wide range of relevant transform lengths.

As I said before, this feature is not often used (since most people will run as non-root!), so please do feed back if you think it’s not working as described.

Cheers

- Iain

—

Iain Bethune
Project Manager, EPCC

Email: ibet...@epcc.ed.ac.uk
Twitter: @IainBethune
Web: http://www2.epcc.ed.ac.uk/~ibethune
Tel/Fax: +44 (0)131 650 5201/6555
Mob: +44 (0)7598317015
Addr: 2404 JCMB, The King's Buildings, Peter Guthrie Tait Road, Edinburgh, EH9 3FD

[M. Brehm]

Hi,

thank you for the detailed explanation. I just tried a small example with just one single calculation. I was able to create a Wisdom file with FFTW_PLAN_TYPE PATIENT just by specifying FFTW_WISDOM_FILE_NAME. Running the same calculation without FFTW_PLAN_TYPE PATIENT, but reading that Wisdom file instead, gives exactly the same performance as above. That's great. Thank you again for enlightening this :-)

Best regards,
Martin

[Iain Bethune]

Hi Martin,

To really check it is working, I would run again with FFTW_PLAN_TYPE PATIENT and FFTW_WISDOM_FILE_NAME set to the location of your file - this should still read the wisdom, and the planning time should be much shorter, while still giving the expected performance.

- Iain

--

Iain Bethune
Project Manager, EPCC

Email: ibet...@epcc.ed.ac.uk
Twitter: @IainBethune
Web: http://www2.epcc.ed.ac.uk/~ibethune
Tel/Fax: +44 (0)131 650 5201/6555
Mob: +44 (0)7598317015
Addr: 2404 JCMB, The King's Buildings, Peter Guthrie Tait Road, Edinburgh, EH9 3FD

[M. Brehm]

Hi,

just checked it, and indeed it works. Calculation starts very fast in spite of FFTW_PLAN_TYPE PATIENT (probably because all the planning was already found in the Wisdom file). This is even better - then it is possible to always leave the FFTW_PLAN_TYPE PATIENT parameter (or even "EXHAUSTIVE") switched on. Very nice. Thanks again.

On my machine (AMD Opteron 6380, four CPUs on one mainboard), the calculations with careful FFT planning are more than three times faster (!) than without this feature.

Best regards,
Martin

[Santiago Movilla]

Hi CP2K community, 

I am trying to take advantage of the fftw pre-planning in my QM/MM calculation. However, I am unable to get CP2K to write or read the wisdom file.  

If I start without wisdom file and defining PREFERRED_FFT_LIBRARY FFTW3, FFTW_PLAN_TYPE PATIENT and FFTW_WISDOM_FILE_NAME ./wisdom.dat, at the end of the cycles it throws the error message:

Error exporting wisdom to file wisdom.dat
pw/fft/fftw3_lib.F:161
 ===== Routine Calling Stack =====
            1 CP2K    

On the other hand, if I generate a generic wisdom using fftw-wisdom (same version, architecture, fftw plan type and OMP set up that I then use with CP2K), the error is:

Error importing wisdom from file wisdom_fftw-wisdom.dat
pw/fft/fftw3_lib.F:222
 ===== Routine Calling Stack =====
            1 CP2K  

I have tested with the container version cp2k_2024.1_mpich_generic_psmp.sif (arch=linux-rocky8-sapphirerapids) and also with the CP2K version 2024.1 available in spack for arch=linux-rocky8-skylake_avx512. The version of fftw in both cases is 3.3.10.

Does anyone have any hints as to what might be causing the problem?
By removing FFTW_WISDOM_FILE_NAME keyword from the input, the job terminates without error.  But I can't take advantage of the wisdom pre-planning.

Best,

Santiago Movilla