CP2K on intel xeon phi
693 views
Skip to first unread message
Aman Jindal
Jun 3, 2016, 10:21:29 AM6/3/16
to cp2k
Dear Users,
Has anyone of you tried installing cp2k on Intel xeonphi. if yes, which arch file should be used (with what modifications)?
thanks in advance,
Aman
Has anyone of you tried installing cp2k on Intel xeonphi. if yes, which arch file should be used (with what modifications)?
thanks in advance,
Aman
Hans Pabst
Jun 10, 2016, 3:37:00 AM6/10/16
to cp...@googlegroups.com
Dear Aman,
I am assuming you are talking about Xeon Phi x100 family (KNC). To prepare an ARCH file for Xeon Phi "native" is about cross-compiling the code using the "-mmic" flag. I think there was some practical experience at EPCC / University of Edinburgh. On the other hand there was also an attempt to fit into the offload infrastructure, which is in place for the GPGPU / ACC backend. This code did not make it into production, and it is hence stripped from the main branch. It is still present in the intel branch, but I have not put any more work into it recently.
If you are talking about Xeon Phi x200 family (KNL), this is a whole different story and it works nicely. Let me summarize the steps for KNL:
Hans
I am assuming you are talking about Xeon Phi x100 family (KNC). To prepare an ARCH file for Xeon Phi "native" is about cross-compiling the code using the "-mmic" flag. I think there was some practical experience at EPCC / University of Edinburgh. On the other hand there was also an attempt to fit into the offload infrastructure, which is in place for the GPGPU / ACC backend. This code did not make it into production, and it is hence stripped from the main branch. It is still present in the intel branch, but I have not put any more work into it recently.
If you are talking about Xeon Phi x200 family (KNL), this is a whole different story and it works nicely. Let me summarize the steps for KNL:
- It works since CP2K 3.0 (released), and onward (CP2K 4.0-development). If you want to use the release or the master/trunk:
- git clone https://github.com/cp2k/cp2k.git cp2k.git (or from SourceForge via Subversion or download of the official release)
- ln -s cp2k.git/cp2k cp2k
- It works since pre-CP2K 3.0 and onward using the (current) CP2K/intel branch.
- git clone --branch intel https://github.com/cp2k/cp2k.git cp2k.git
- ln -s cp2k.git/cp2k cp2k
- I strongly suggest to use LIBXSMM (although you can run LIBSMM, which currently only supports up to AVX2).
- git clone https://github.com/hfp/libxsmm.git
- git checkout tags/1.4.3 (if you don't want the master)
- I suggest to use my outgrown ARCH file(s), but you can also build your own (minimal) ARCH file.
- The ARCH files work for CP2K 3.0, and development (regardless of whether it is the master or the intel branch).
- For 3.0/master/trunk branch, you have to download the following ARCH files into the cp2k/arch folder:
https://github.com/cp2k/cp2k/raw/intel/cp2k/arch/Linux-x86-64-intel.x (needed for PSMP, POPT, SOPT, and SSMP)
https://github.com/cp2k/cp2k/raw/intel/cp2k/arch/Linux-x86-64-intel.psmp
POPT, SOPT, and SSMP configurations are available as well. - Building CP2K
- LIBXSMM is auto-detected if the "cp2k" folder (perhaps a symlink; see above) and the "libxsmm" folder are located in parallel,
if not: use LIBXSMMROOT=/path/to/libxsmm along with the make command. - source /opt/intel/compilers_and_libraries_2017.0.064/linux/bin/compilervars.sh intel64
- make ARCH=Linux-x86-64-intel VERSION=psmp AVX=3 MIC=1
- There are quite some additional key-value pairs supported:
- LIBXSMM=2: intercepts all GEMM calls to go through LIBXSMM (gives some minor/additional performance)
- SYM=1: add symbols "-g" (it also enables VTune support in LIBXSMM if Intel VTune is source'd, see here)
- DBG=1: enables unoptimized code along with symbols (not for production runs, but for debug)
- MKL=2: enables parallel MKL (default is sequential, but sequential should be fine)
- I recommend to use LIBINT, LIBXC, and ELPA (unless you know that your workload cannot benefit).
If you want/need (optimal) recipes for these libraries, let me know. - LIBINTROOT=/path/to/libint
- LIBXCROOT=/path/to/libxc
- ELPAROOT=/path/to/elpa
Hans
Iain Bethune
Jun 13, 2016, 4:37:43 AM6/13/16
to cp...@googlegroups.com, Fiona Reid
Aman,
If you want to run in native mode on KNC you could use the attached arch file as a starting point. As Hans said, it’s basically a case of adding the -mmic flag to cross-compile for the KNC’s processors. As usual, when at the bleeding edge of ifort versions, there are a few files which need to be compiled at lower optimisation levels.
The main issue with CP2K on KNC is that in order to get good performance, you need to get close to 240 threads (hence why we use the MPI+OpenMP version of CP2K), but to find a problem where you can get good scaling to 240 threads, while still fitting in the KNC’s memory (6, 8 or 16 GB depending on the exact model) is hard.
There are some (slightly outdated now) performance results here http://www.prace-ri.eu/IMG/pdf/wp140.pdf
- Iain
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
If you want to run in native mode on KNC you could use the attached arch file as a starting point. As Hans said, it’s basically a case of adding the -mmic flag to cross-compile for the KNC’s processors. As usual, when at the bleeding edge of ifort versions, there are a few files which need to be compiled at lower optimisation levels.
The main issue with CP2K on KNC is that in order to get good performance, you need to get close to 240 threads (hence why we use the MPI+OpenMP version of CP2K), but to find a problem where you can get good scaling to 240 threads, while still fitting in the KNC’s memory (6, 8 or 16 GB depending on the exact model) is hard.
There are some (slightly outdated now) performance results here http://www.prace-ri.eu/IMG/pdf/wp140.pdf
- Iain
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
Reply all
Reply to author
Forward
0 new messages