FFTW Wisdom File Error

[Santiago Movilla]

Hi CP2K community, 

I am trying to take advantage of the fftw pre-planning in my QM/MM calculation. However, I am unable to get CP2K to write or read the wisdom file.  

If I start without wisdom file and defining PREFERRED_FFT_LIBRARY FFTW3, FFTW_PLAN_TYPE PATIENT and FFTW_WISDOM_FILE_NAME ./wisdom.dat, at the end of the cycles it throws the error message:

Error exporting wisdom to file wisdom.dat
pw/fft/fftw3_lib.F:161
 ===== Routine Calling Stack =====
            1 CP2K    

On the other hand, if I generate a generic wisdom using fftw-wisdom (same version, architecture, fftw plan type and OMP set up that I then use with CP2K), the error is:

Error importing wisdom from file wisdom_fftw-wisdom.dat
pw/fft/fftw3_lib.F:222
 ===== Routine Calling Stack =====
            1 CP2K  

I have tested with the container version cp2k_2024.1_mpich_generic_psmp.sif (arch=linux-rocky8-sapphirerapids) and also with the CP2K version 2024.1 available in spack for arch=linux-rocky8-skylake_avx512. The version of fftw in both cases is 3.3.10.

Does anyone have any hints as to what might be causing the problem?
By removing FFTW_WISDOM_FILE_NAME keyword from the input, the job terminates without error.  But I can't take advantage of the wisdom pre-planning.

Best,

Santiago Movilla

[Frederick Stein]

Dear Santiago,

Thank you for your report. This is apparently a bug. I am currently trying to fix it such that it will still be part of the next release.

Best,

Frederick

[Frederick Stein]

As a workaround, stick to the default ( FFTW_PLAN_TYPE ESTIMATE). This is well tested. I am able to reproduce the issue locally but currently do not know why it actually fails.

[Santiago Movilla]

Dear Frederick,
Thank you very much for the answers. Even using the default (ESTIMATE) I still get the same errors. I have decided to attach the input of my calculation in case you can find any other error that may be related or, taking advantage of your expertise, if you can suggest any other kind of change that may speed up the calculation time.
Best,
Santiago Movilla

[Frederick Stein]

Can you try it without the line with FFTW_WISDOM_FILE_NAME?

[Santiago Movilla]

Sure!, I did.

Without this line the calculation works perfectly. It finishes without error. The problem is when I want to read or write the wisdom file.

[Frederick Stein]

I will push a bugfix. Now, I can get it running, if a wisdom file created from fftw-wisdom is used, but not if it was created during a CP2K run (better than nothing).

Can you provide the timing report (last part of the output file) to check the performance of your setup?

[Santiago Movilla]

Dear Frederick,
I would like to express my gratitude for all the assistance provided.
I have conducted two trials with FFTW3 and FFTSG, and I have attached the timings report. It appears that FFTSG results in superior performance.
I have not yet tested with the wisdom file, as I have not yet generated it for the estimate plan type.

Best,

Santiago Movilla

[Frederick Stein]

Dear Santiago,

If you have compiled CP2K yourself, can you recompile with the current master trunk and run CP2K with a wisdom file optimized from the script fftw-wisdom (delivered with FFTW3). With my bugfix, I was not able to run CP2K with a wisdom file optimized from a CP2K run but with an FFTW-optimized one (I have no idea why).

Best,

Frederick

[Santiago Movilla]

Dear Frederick,

I have tried to do it now. I got the same error again (Error importing wisdom from file...) I attach the file by which I create the wisdom file (using fftw-wisdom), in case there is any irregularity in this one. I also attach the CP2K output in case in the technical details printed by the program, at the beginning of the execution, you can detect any error in the installation.

I don't know why it works for you and not for me.

Best,

Santiago Movilla

[Frederick Stein]

Dear Santiago,

That is interesting. I used "./fftw-wisdom -v -c -o wisdom", i.e. I optimized in patient mode (default). I do not know the internals of FFTW3 too well. Unfortunately, they do not provide any hint what the actual issue is.

According to your CP2K output, you are using the latest release (2024.1), not the development version? If you use an official release, is it possible for you to pull and compile the current master trunk of the CP2K Github repository? This one has the fix (which will hopefully work for you). Your version is too old for sure.

Best,

Frederick

[Santiago Movilla]

Dear Frederick,

Yes, I am using version 2024.1. Right now it is impossible for me to recompile the one from the Github repository (ITS must do it). Tomorrow as soon as they can I will do it and let you know what happens.

On the other hand, if you create the wisdom with the default patient, then the pre-planned routines cannot be used in CP2K in ESTIMATE type mode, or is it possible? I thought FFTW was strict about the architecture, the work plan and the number of threads.

Thanks for the help today, when I have new tests I will let you know here.

Best,

Santiago

[Frederick Stein]

Dear Santiago,

Regarding the version: There will be a new version in 1-2 weeks if that is more suitable for your compute center.

I have just tried. It did work for me (at least with different planning types and a wisdom file for patient). I didn't check the number of threads. The details of the library are unknown to me. It is also difficult to check because there is no direct source code of FFTW3 available as it is the case with other libraries.

Tell me as soon as you have any news.

Best,

Frederick