SCALE-6.2 compilation in parallel mode

[Kuryndin Anton]

Hello,

I want to ask a question about installation of SCALE-6.2 with Parallel Processing function enabled on Linux cluster. As far as I understand from the manual it's necessary to recompile the source code in order to use SCALE in parallel mode. However the information that is located in manual is not enough for compiling SCALE in parallel mode on Linux OS. Following the instructions leads to failing of compilation (compilation errors occur). What is possible to do in that case?

The libraries and compilers that are described in the manual are installed. I’ve tried to install SCALE on Ubuntu and CentOS. I’d be very grateful if you shared your experience of SCALE installation with parallel function.

[Rob Lefebvre]

Hi, 

What is the compilation error you are observing?

Best regards,

Rob Lefebvre

[Kuryndin Anton]

As for Ubuntu the error is with libblas:
[  0%] Built target teuchoscore
[  1%] Built target teuchosparameterlist
[  2%] Built target teuchoscomm
make[2]: *** No rule to make target `/usr/lib/lapack/libblas.so', needed by `Trilinos/packages/teuchos/numerics/src/libteuchosnumerics.so.6.2'.

[Kuryndin Anton]

I' ve dealt with the previous error simply reinstalling lapack library. However now I get this:

[ 54%] Built target legend
Linking CXX executable wpng
libpng/liblibpng.so.6.2: undefined reference to `trunc'
libpng/liblibpng.so.6.2: undefined reference to `pow'
libpng/liblibpng.so.6.2: undefined reference to `floor'
collect2: error: ld returned 1 exit status

By now I couldn't find the reason for this.

[Rob Lefebvre]

This appears to be an issue in the system library the compiler is providing.

What is the compiler and version you are using?

Perhaps, conducting a

make VERBOSE=1

would provide better insight into what is missing?

Best regards,

Rob Lefebvre

[Kuryndin Anton]

Unfortunately Verbose didn't help.
I've tried to use make -n and the last messages in listing were:
 -c /home/ser/SCALE-6.2/packages/Newt/newt.f90 -o CMakeFiles/newt.dir/newt.f90.o
make[2]: *** no rule to make target `packages/Newt/libnewtlib.so.6.2', needed by `packages/Newt/newt'.
make[1]: *** [packages/Newt/CMakeFiles/newt.dir/all] error 2
make: *** [all] error 2

The gcc version is 4.8.4 and Ubuntu is 14.04.

Thanks in advance for your help.

[Rob Lefebvre]

Seems several things are going on here. The compiler version should be sufficient, but it isn't clear why you previously received a link error in wpng, and a compile error in newt. There are no interdependencies between wpng and newt. 

Are you conducting a parallel build? I.e., make -j8?

I need more information. Please post the compile and/or link error when executing the following command:

make -j1 VERBOSE=1

Please, note that the build is serial (-j1), and 'VERBOSE' is case sensitive. I hope to see the compiler/linker command line invocation and subsequent compiler/linker error message immediately following.

With this information I can compare to our equivalent test bed installations for any system differences which we can than investigate, and hopefully resolve.

Best regards,

Rob Lefebvre

[Kuryndin Anton]

After executing the command you mentioned the last step was:

Linking CXX executable wpng

And the last lines of quite a long listing are (again a link error in wpng):

/home/ser/SCALE-6.2/build/gcc/Trilinos/packages/teuchos/parameterlist/src:/home/ser/SCALE-6.2/build/gcc/Trilinos/packages/teuchos/core/src:

libpng/liblibpng.so.6.2: undefined reference to `trunc'
libpng/liblibpng.so.6.2: undefined reference to `pow'
libpng/liblibpng.so.6.2: undefined reference to `floor'
collect2: error: ld returned 1 exit status

make[2]: *** [packages/ScaleHeritage/Legend/wpng] Error 1
make[1]: *** [packages/ScaleHeritage/Legend/CMakeFiles/wpng.dir/all] Error 2
make: *** [all] Error 2

[Rob Lefebvre]

Hi, appears Ubuntu 14.04 has changed/corrected some of the dependencies in their system libraries. 

This link error should be resolved by updating the packages/ScaleHeritage//Legend/libpng//pngconf.h file to include the 'math.h'. 

Update the following section of code in packages/ScaleHeritage//Legend/libpng//pngconf.h (line ~52):

/* From libpng 1.6.0 libpng requires an ANSI X3.159-1989 ("ISOC90") compliant C
 * compiler for correct compilation.  The following header files are required by
 * the standard.  If your compiler doesn't provide these header files, or they
 * do not match the standard, you will need to provide/improve them.
 */
#include <limits.h>
#include <stddef.h>

to be as follows:

/* From libpng 1.6.0 libpng requires an ANSI X3.159-1989 ("ISOC90") compliant C
 * compiler for correct compilation.  The following header files are required by
 * the standard.  If your compiler doesn't provide these header files, or they
 * do not match the standard, you will need to provide/improve them.
 */
#include <limits.h>
#include <stddef.h>
#include <math.h>

Note the '#include <math.h>'. 

Best regards,

Rob Lefebvre

[Kuryndin Anton]

Thanks for your help. Unfortunately that didn't go properly. I still get the same error in "wpng". Listing is also quite the same.

[Rob Lefebvre]

Hi, you may have tried this already, and it may just be a rabbit hole with Ubuntu, but try adding the same includes to the wpng.c file.

Legend and WPNG are used to generate geometry plots in Keno via the plot block. If this feature is not needed, it might be possible to ignore the Legend and WPNG compile errors via a 'make -i' or a 'make -i -k'. This might be worth trying.

Unfortunately, Ubuntu is not a part of our continuous integration scheme. I will put in a request to add an Ubuntu machine to our continuous integration.

Best regards,

Rob Lefebvre

[Kuryndin Anton]

Hi again! The installation of Ubuntu 16.04 instead of 14.04 resolved the problem and SCALE compiled and installed successfully! But the new problem appeared (as it always happens): KENO module doesn't work. All other modules and sequences that don't use KENO execute fine. I believed that there is an issue with openmpi and tried to reinstall it using different versions and this didn't solve the problem out. 

The error message I get in console is:

sh: 1: makedirectory: not found

The message in output file is:  

Error opening file /tmp/scale.ser.15678/html/errorshead0001.html on unit 12 as unknown and formatted   

Action readwrite    position rewind

The open status was returned as            2

Processor error message -

Cannot open file '/tmp/scale.ser.15678/html/errorshead0001.html'

Execution will terminate 

stop code        200

KENO failed to execute. error code 51200

The input files were from the samples provided along the SCALE. The current openmpi version is 1.10.2.

I've also looked through the files in "tmp" directory and that gave insufficient info. 

What is the file that KENO looks for? 

Any help is greatly appreciated!

[Jan Frybort]

I believe that the first error "sh: 1: makedirectory: not found" can by solved by installing ksh: sudo apt-get install ksh
Jan Frybort

[Rob Lefebvre]

Great to hear some progress was made. 

The makedirectory is a bash script that should be located at SCALE/cmds/makedirectory. 

Is your terminal a bash terminal? I thought it should be by default on Ubuntu. 

I'm interested to hear if Jan Frybort's solution works. I'll up-vote it if it is.

Best regards,

Rob Lefebvre