Compling Qwalk on BG/Q with IBM XL and GNU cross compilers

[Alex Findlater]

Hi Lucas,

I am attempting to compile Qwalk on a BG/Q machine at ANL (Vesta) with both the IBM XL and GNU cross compilers.
I am running into the following compile time errors:

properties/Average_ekt.cpp:838: error: 'nTest' was not declared in this scope
properties/Average_ekt.cpp:840: error: 'nelectrons' was not declared in this scope
make: *** [/gpfs/vesta-home/afindlat/qwalk/src/obj/Linux-mpi/Average_ekt.o] Error 1

Here are the mentioned line numbers:

837  ##int natoms=sample->ionSize();                                                                                                                                           
838  assert(accept_var.GetDim(0) >= nTest());                                                                                                                                                              
839 //assert(totalv.GetDim(0) >= nwf);                                                                                                                                                                        
840  assert(nelectrons == sample->electronSize());

I commented out the assert statements on lines 838 and 840 and was able to successfully compile, but I am not
sure what is causing this problem with the cross compilers.

I am not a C++ expert so I am not sure I can diagnose the problem, however after a little googling I came across the following from IBM:

"Earlier compiler releases sometimes allowed names to be used in a template definition before they were declared as long as they were declared before the template was instantiated."
(http://www-01.ibm.com/support/docview.wss?uid=swg21315623)

Maybe this is relevant?

Any incite would be helpful. Thank you.

-Alex

[Alex Findlater]

Follow up:

I have the "hacked" code running on the BG/Q (with lines 838 and 840 commented out).  I guess at this point
I am just missing some error handling due to the missing assert statements?

Thanks,
Alex

[Lucas Wagner]

Hi Alex,

It looks like you have debugging turned on(and some of the debugging routines are not compiling..we should fix that). You don't want to use that executable; although it should work, it will be slow. The BG/Q makefile that I've been using is the one appended here. It at least compiles on mira. I am using the  +mpiwrapper-xl.legacy to compile.

Also, please note that the individual cores on BG/Q are quite slow relative to modern x86 processors. I recommend running in c32 mode, or perhaps c64 mode, which takes advantage of hyperthreading, and running on at least 2048 threads for DMC, so that one walker/thread is used.

Best,

Lucas

######################################################################
# Compiler definitions for jaguar.ccs.ornl.gov (compute nodes)
#  all compiler specific information should be declared here

CXX:=mpixlcxx
CXXFLAGS:= -O3 -qstrict -qhot -qarch=qp -qtune=qp -qprefetch -qunroll=yes
CXXFLAGS += -DUSE_BLAS -DUSE_LAPACK -DUSE_MPI ${INCLUDEPATH}
SOFTPATH:=/soft/libraries/alcf/current/xl/
BLAS_LIBS := -L/soft/compilers/ibmcmp-feb2015/xlf/bg/14.1/lib64/  -L${SOFTPATH}/BLAS/lib -L${SOFTPATH}/CBLAS/lib/ -L${SOFTPATH}/LAPACK/lib -llapack -lcblas -lblas -lxlopt -lxlf90_t -lxl -lpthread -lmass
BLAS_INCLUDE := -I${ESSL}/include


DEBUG:= -DNO_RANGE_CHECKING   -DNDEBUG 
######################################################################
# This is the invokation to generate dependencies
DEPENDMAKER:=g++ -MM  $(INCLUDEPATH)

--
You received this message because you are subscribed to the Google Groups "QWalk user's group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qwalk-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alex Findlater]

Hi Lucas,

Thanks for the makefile!  I am at ATPESC right now, but once I get a chance tonight I will recompile with your flags and the compiler you recommend. 

With the executable I am using now, I am noticing that as I add more and more nodes (1 walker per core using c64), I am getting killed by the IO for the
*.config file.  When I cut the number of blocks in half while doubling the number of steps per block (so I think the same number of total steps) I get a significant speed-up since I am
writing to file less often.  Is this a common strategy?  I haven't thought much about how the statistics will differ between the two strategies.

Thanks again for your help,
Alex

[Lucas Wagner]

Hi Alex,

Could you post your input files (just the text), and what systems you're doing/how many processors?

If you're doing a small calculation, you may be dominated by writing the configs. Particularly if your blocks are taking just a few seconds. Putting more steps in the blocks is quite safe and will improve your statistics.

Cheers,

Lucas