[Rocks-Discuss] PGI and Intel Compiler Licensing and VASP
James Rudd
This is probably a little off topic, but I'm hoping answers are known
as PGI gets discussed regularly.
I manage a 20 node AMD Opteron cluster running Rocks 5.1 with SGE
queuing. At the moment it is only used for Gaussian 03 calculations.
We are planning on running VASP on the cluster, and from what I have
read this will require adding the HPC Roll (for OpenMP) and either the
Intel or PGI roll for Fortran compilers. (Probably PGI as from what I
have read it performs better than Intel on AMD chips)
My first question is too check if my assumptions above are correct and
ask if anyone using VASP on a ROCKS cluster has any suggestions. Also
do you use goVASP?
My second and most important question is about licensing for PGI or
Intel compilers. What type of license is required? Will a single
license work and we can compile VASP on the frontend and use binaries
on compute nodes; or does each node need a license, or is there a
special license required?
Thankyou,
James
James Rudd
http://jrudd.org/
---------------------
HPC Cluster Administrator
Centre of Excellence for Silicon Photovoltaics and Photonics
University of New South Wales
Sydney NSW 2052
AUSTRALIA
Backlund, Daniel
I can offer you some advice running VASP on an opteron cluster. We run both SIESTA and
VASP on our 5.1 cluster of about 50 processors (mostly Barcelona cores). We use the Pathscale
compiler and have experimented with Intel compilers, both OpenMPI. We stay away from
using threading in our applications, since jobs will often run on the same node. First off, we did
not notice an increase in performance using Intel versus the Pathscale compiler. Pathscale compiles
a lot faster and in my opinion is just as easy to control optimizations. Second, you can certainly
buy a single license for your compiler to use on the frontend. I recommend installing the
compiler in the /share/apps directory so your nodes can see the compiler libraries.
Third, you can use the AMD ACML performance libraries which are freely available. I can
tell you right now ACML is as fast as the Intel MKL libraries on opterons. We compiled our
own versions of OpenMPI for use with the Pathscale compiler, so you really won't need the HPC roll.
As far as VASP goes, it runs quite nicely and scales very well over a Gigabit network. The
performance of VASP really depends on your BLAS library. You can use any optimized version
you want, so long as it is for the compiler you use. From your signature, I see that you are working for
a University and you should be able to freely download a copy of GotoBLAS http://www.tacc.utexas.edu/resources/software/#blas
which is probably the fastest of all. If you use GotoBLAS, which is what we use, you will need to compile
your own version of lapack-3.1.1 with Goto's lapack patch. Before you fork up the money for goVASP, I recommend trying
some free viewers and scripts (XCrysDen, VMD, p4vasp, vtstscripts, vasputil, etc.) We do not or have
never used goVASP and we are doing just fine.
Hope this helps. E-mail me if you have and other questions about it.
Daniel
________________________________________
From: npaci-rocks-dis...@sdsc.edu [npaci-rocks-dis...@sdsc.edu] On Behalf Of James Rudd [james...@gmail.com]
Sent: Monday, February 16, 2009 7:37 AM
To: Rocks Cluster Discussion
Cc: Dirk Koenig
Subject: [Rocks-Discuss] PGI and Intel Compiler Licensing and VASP
Hamilton, Scott L.
I am not running VASP myself, but looking around it seems that you may
actually have to build openMPI yourself as several sites claim that VASP
will not link against mpi libraries compiled using gfortran, which I
believe was used to compile the HPC Roll. I don't know if a PGI roll
exists or not, but the Intel roll includes a working openMPI with it. I
would assume the PGI roll would as well.
I have a few users that are running VASP on my cluster without issue,
but I hand compiled my own openMPI using the Intel compilers and they
are using it.
Scott
-----Original Message-----
From: npaci-rocks-dis...@sdsc.edu
[mailto:npaci-rocks-dis...@sdsc.edu] On Behalf Of James Rudd
Sent: Monday, February 16, 2009 7:38 AM
To: Rocks Cluster Discussion
Cc: Dirk Koenig
Subject: [Rocks-Discuss] PGI and Intel Compiler Licensing and VASP
James Rudd
Thank you very much for your suggestions. They should make
implementation much more problem free and get everything up and
running faster.
I was unaware of the Pathscale compilers, the need to recompile
OpenMPI with the different compilers, and about ACML and BLAS.
Thanks,
James
James Rudd
http://jrudd.org/
---------------------
HPC Cluster Administrator
Centre of Excellence for Silicon Photovoltaics and Photonics
University of New South Wales
Sydney NSW 2052
AUSTRALIA
Bart Brashers
> queuing. At the moment it is only used for Gaussian 03 calculations.
> We are planning on running VASP on the cluster, and from what I have
> read this will require adding the HPC Roll (for OpenMP) and either the
> Intel or PGI roll for Fortran compilers. (Probably PGI as from what I
> have read it performs better than Intel on AMD chips)
>
> Intel compilers. What type of license is required? Will a single
> license work and we can compile VASP on the frontend and use binaries
> on compute nodes; or does each node need a license, or is there a
> special license required?
You can buy just one license, compile on the frontend, and run on the
compute nodes. If you use non-static libraries (most people do) then
you'll need a few files on the compute nodes too. If you install PGI in
/share/apps/pgi, then those needed libraries are shared. That's what I
would suggest for a cluster of your size.
Bart
This message contains information that may be confidential, privileged or otherwise protected by law from disclosure. It is intended for the exclusive use of the Addressee(s). Unless you are the addressee or authorized agent of the addressee, you may not review, copy, distribute or disclose to anyone the message or any information contained within. If you have received this message in error, please contact the sender by electronic reply to em...@environcorp.com and immediately delete all copies of the message.
James Rudd
Just a quick thanks for your suggestion. We ended up purchasing a
single PGI license for frontend and compiled Vasp, GotoBlas and
OpenMPI on it. I am using Environmental Modules for managing different
Vasp versions and other configuration options.
Thanks,
James
James Rudd
http://jrudd.org/
---------------------
Network Administrator
Sydney Boys High
---------------------
HPC Cluster Administrator
Centre of Excellence for Silicon Photovoltaics and Photonics
University of New South Wales
Sydney NSW 2052
AUSTRALIA
Bart Brashers
Glad I could help! This list is an amazing resource, thanks to all the good people (mostly smarter and better-educated than me) who willingly share their experience.
Bart