LIBXSMM (cp2kflags?)
605 views
Skip to first unread message
Luiz Fernando Lopes Oliveira
Jun 21, 2017, 11:18:38 PM6/21/17
to cp2k
Hi all.
I compiled CP2K with LIBINT, LIBXC, ELPA and PLUMED2, reading throughout the forum I found many people (e.g. here) saying that LIBXSMM also helps to speed up CP2K.
I installed it (without errors) with
make -f /path/to/libxsmm/Makefile
following the info from: https://github.com/hfp/libxsmm/blob/master/README.md
And on my arch file I have:
LIBXSMMROOT = /lflo/cp2k2017/cp2k/cp2k/libxsmm
ifneq (,$(LIBXSMMROOT))
DFLAGS += -D__LIBXSMM
IFLAGS += -I$(LIBXSMMROOT)/include
MIC_LDFLAGS += -offload-option,mic,ld,"$(LIBXSMMROOT)/lib/mic/libxsmm.a"
endif
I then compile CP2K (also without errors).
However, when running a regtest, I can only see information about the other libraries mentioned above:
cp2kflags: omp libint fftw3 libxc elpa=201611 elpa_qr parallel mpi3 scalapack libderiv_max_am1=5 libint_max_am=6 plumed2
Also, the number of tests performed is exactly the same as before the LIBXSMM compilation.
My questions are: should the cp2kflags show something for LIBXSMM? Are there any tests for LIBXSMM?
Thanks!
Nico Holmberg
Jun 26, 2017, 1:27:26 AM6/26/17
to cp2k
Hi,
You seem to be missing the library flags for LIBXSMM. I have set the following flags (gfortran)
LIBXSMM_LIB=/homeappl/home/nholmber/appl_sisu/lib/libxsmm-1.7.1/lib
LIBXSMM_INC=/homeappl/home/nholmber/appl_sisu/lib/libxsmm-1.7.1/include
DFLAGS += -D__LIBXSMM
FCFLAGS += -I$(LIBXSMM_INC)
LDFLAGS = $(FCFLAGS)
LIBS += -L$(LIBXSMM_LIB) -lxsmm -lxsmmext -lxsmmgen -lxsmmf
There are no regtests from libxsmm, but you should see the libxsmm flag listed in CP2K output files once you've successfully linked the library.
BR,
Nico
You seem to be missing the library flags for LIBXSMM. I have set the following flags (gfortran)
LIBXSMM_LIB=/homeappl/home/nholmber/appl_sisu/lib/libxsmm-1.7.1/lib
LIBXSMM_INC=/homeappl/home/nholmber/appl_sisu/lib/libxsmm-1.7.1/include
DFLAGS += -D__LIBXSMM
FCFLAGS += -I$(LIBXSMM_INC)
LDFLAGS = $(FCFLAGS)
LIBS += -L$(LIBXSMM_LIB) -lxsmm -lxsmmext -lxsmmgen -lxsmmf
There are no regtests from libxsmm, but you should see the libxsmm flag listed in CP2K output files once you've successfully linked the library.
BR,
Nico
Dominik 'Rathann' Mierzejewski
Jun 27, 2017, 8:25:51 AM6/27/17
to cp2k
Hello,
On Monday, 26 June 2017 at 07:27, Nico Holmberg wrote:
> Hi,
>
> You seem to be missing the library flags for LIBXSMM. I have set the
> following flags (gfortran)
>
> LIBXSMM_LIB=/homeappl/home/nholmber/appl_sisu/lib/libxsmm-1.7.1/lib
> LIBXSMM_INC=/homeappl/home/nholmber/appl_sisu/lib/libxsmm-1.7.1/include
>
> DFLAGS += -D__LIBXSMM
> FCFLAGS += -I$(LIBXSMM_INC)
> LDFLAGS = $(FCFLAGS)
> LIBS += -L$(LIBXSMM_LIB) -lxsmm -lxsmmext -lxsmmgen -lxsmmf
Out of curiosity, why are you adding -lxsmmext and -lxsmmgen? They don't
seem to be required by libxsmm or libxsmmf.
Regards,
Dominik
--
Fedora http://fedoraproject.org/wiki/User:Rathann
RPMFusion http://rpmfusion.org
"Faith manages."
-- Delenn to Lennier in Babylon 5:"Confessions and Lamentations"
On Monday, 26 June 2017 at 07:27, Nico Holmberg wrote:
> Hi,
>
> You seem to be missing the library flags for LIBXSMM. I have set the
> following flags (gfortran)
>
> LIBXSMM_LIB=/homeappl/home/nholmber/appl_sisu/lib/libxsmm-1.7.1/lib
> LIBXSMM_INC=/homeappl/home/nholmber/appl_sisu/lib/libxsmm-1.7.1/include
>
> DFLAGS += -D__LIBXSMM
> FCFLAGS += -I$(LIBXSMM_INC)
> LDFLAGS = $(FCFLAGS)
> LIBS += -L$(LIBXSMM_LIB) -lxsmm -lxsmmext -lxsmmgen -lxsmmf
seem to be required by libxsmm or libxsmmf.
Regards,
Dominik
--
Fedora http://fedoraproject.org/wiki/User:Rathann
RPMFusion http://rpmfusion.org
"Faith manages."
-- Delenn to Lennier in Babylon 5:"Confessions and Lamentations"
Nico Holmberg
Jun 27, 2017, 9:05:08 AM6/27/17
to cp2k
Hi,
-lxsmmext is for the optional OpenMP support
Not sure about -lxsmmgen. The flag was copied from an installation template used by my HPC center. I built some static kernels for specific matrix sizes when I installed libxsmm. Maybe the libxsmmgen library is related to them, but you'd have to ask the maintainers of libxsmm to be sure.
BR,
Nico
-lxsmmext is for the optional OpenMP support
Not sure about -lxsmmgen. The flag was copied from an installation template used by my HPC center. I built some static kernels for specific matrix sizes when I installed libxsmm. Maybe the libxsmmgen library is related to them, but you'd have to ask the maintainers of libxsmm to be sure.
BR,
Nico
Luiz Fernando Lopes Oliveira
Jun 27, 2017, 12:19:04 PM6/27/17
to cp2k
Thank you very much, Nico.
Well, I'm using ifort, and, I think (based on my installation of ELPA and on the howto pages) that
IFLAGS += -I$(LIBXSMMROOT)/include
is used instead of
FCFLAGS += -I$(LIBXSMM_INC)
LDFLAGS = $(FCFLAGS).Following your advice I inserted
LIBS += -L$(LIBXSMM_LIB) -lxsmm -lxsmmext -lxsmmgen -lxsmmf
then compiled CP2K without a problem but I still can't see any libxsmm flag on my CP2K outputs.
Any ideas?
Thanks!
Nico Holmberg
Jun 27, 2017, 1:41:13 PM6/27/17
to cp2k
Hi Luiz,
Looking at this intel arch-file from the CP2K Intel branch, it seems that you indeed need to set
However, you should also add it to the FCFLAGS (see the link):
FCFLAGS += $(OPTFLAGS) -diag-disable $(DIAG_DISABLE) $(DFLAGS) $(IFLAGS)
Hopefully that helps.
BR,
Nico
Looking at this intel arch-file from the CP2K Intel branch, it seems that you indeed need to set
IFLAGS += -I$(LIBXSMMROOT)/include
However, you should also add it to the FCFLAGS (see the link):
FCFLAGS += $(OPTFLAGS) -diag-disable $(DIAG_DISABLE) $(DFLAGS) $(IFLAGS)
Hopefully that helps.
BR,
Nico
Hans Pabst
Jun 30, 2017, 4:29:21 AM6/30/17
to cp2k
Hello Dominik,
I just wanted to add to Nico's nice explanation, and give some details about what the various libraries are meant for:
I just wanted to add to Nico's nice explanation, and give some details about what the various libraries are meant for:
- libxsmm: LIBXSMM's main library, and typically sufficient to link with. CP2K+LIBXSMM needs to link against libxsmm.a.
- libxsmmf: only needed for FORTRAN code, which uses the libxsmm.mod file. CP2K+LIBXSMM needs to link against libxsmmf.a.
- libxsmmext: contains functions with internal OpenMP-based threading. CP2K+LIBXSMM uses OpenMP-threads by itself (xSMP builds), and there is no need to link against libxsmmext.a.
- libxsmmgen: contains LIBXSMM's code generator (backend), and the library can be used if only the low-level code generator is used. This library is also used to organize LIBXSMM's code, and is actually contained by libxsmm.a. There is no need for CP2K+LIBXSMM to link against libxsmmgen.a.
- libxsmmnoblas: dummy library to deliver BLAS symbols if it is certain that LIBXSMM does not fallback to BLAS. CP2K+LIBXSMM should not link against libxsmmnoblas.a, and there is no need.
Btw, in case of using Intel Compiler the Linux-x86-64-intel.* ARCH file(s) can be used even with non-Intel branch of CP2K by copying these files into the arch-folder (I think Nico pointed to Linux-x86-64-intel.x). In case of these "special" arch files there is the choice to intercept *all* xGEMM calls with LIBXSMM, and filter-out some (minor) additional performance opportunity. On BigCore this is definitely negligible, but perhaps beneficial on KNL. However this "choice" is sort of experimental, and can be enabled with LIBXSMM=2. For example:
$ make ARCH=Linux-x86-64-intel VERSION=psmp LIBXSMM=2
To also rely on LIBXSMM's transpose routine in some cases, one may use LIBXSMM=3. However, the latter requires also the Intel-branch of CP2K.
Hans
Dominik 'Rathann' Mierzejewski
Jun 30, 2017, 4:37:27 AM6/30/17
to cp2k
Hello, Hans.
On Friday, 30 June 2017 at 10:29, Hans Pabst wrote:
> Hello Dominik,
>
> I just wanted to add to Nico's nice explanation, and give some details
> about what the various libraries are meant for:
>
> - *libxsmm*: LIBXSMM's main library, and typically sufficient to link
> with. *CP2K+LIBXSMM needs to link against libxsmm.a*.
> - *libxsmmf*: only needed for FORTRAN code, which uses the libxsmm.mod
> file. *CP2K+LIBXSMM needs to link against libxsmmf.a*.
> - libxsmmext: contains functions with internal OpenMP-based threading.
> CP2K+LIBXSMM uses OpenMP-threads by itself (xSMP builds), and there is *no
> need* to link against libxsmmext.a.
> - libxsmmgen: contains LIBXSMM's code generator (backend), and the
> libxsmmgen.a.
> - libxsmmnoblas: dummy library to deliver BLAS symbols if it is certain
Thank you!
That's an excellent explanation and very useful to packagers.
Could you have it added to CP2K documentation?
On Friday, 30 June 2017 at 10:29, Hans Pabst wrote:
> Hello Dominik,
>
> I just wanted to add to Nico's nice explanation, and give some details
> about what the various libraries are meant for:
>
> with. *CP2K+LIBXSMM needs to link against libxsmm.a*.
> - *libxsmmf*: only needed for FORTRAN code, which uses the libxsmm.mod
> file. *CP2K+LIBXSMM needs to link against libxsmmf.a*.
> - libxsmmext: contains functions with internal OpenMP-based threading.
> CP2K+LIBXSMM uses OpenMP-threads by itself (xSMP builds), and there is *no
> need* to link against libxsmmext.a.
> - libxsmmgen: contains LIBXSMM's code generator (backend), and the
> library can be used if only the low-level code generator is used. This
> library is also used to organize LIBXSMM's code, and is actually contained
> by libxsmm.a. There is *no need* for CP2K+LIBXSMM to link against
> library is also used to organize LIBXSMM's code, and is actually contained
> libxsmmgen.a.
> - libxsmmnoblas: dummy library to deliver BLAS symbols if it is certain
> that LIBXSMM does not fallback to BLAS. CP2K+LIBXSMM should not link
> against libxsmmnoblas.a, and there is *no need*.
Thank you!
That's an excellent explanation and very useful to packagers.
Could you have it added to CP2K documentation?
Reply all
Reply to author
Forward
0 new messages