What about FREEBSD or OpenSOLARIS

[Simeon Nifos]

Dear HashDist developers,

It would be great if you provide support for FREEBSD and OpenSolaris. 

HasDist is the backbone for building the Fenics project. I have found 

however, running benchmarks that FreeBSD and OpenSolaris for scientific

computing is more efficient than Linux or OSX. For example the same executable

compiled with the same compiler and linked to the same BLAS libraries both in 

Linux and OpenSolaris runs 5% faster on OpenSolaris. This was a great 

surprise. I would like to continue testing and benchmarking, but I cannot do 

that without being able to install FEniCS which uses hasdist. How hard is

it to port it to FreeBSD or OpenSolaris.

Best regards,

Simon.

[Aron Ahmadia]

Hi Simeon,

It would be great if you provide support for FREEBSD and OpenSolaris. 

HasDist is the backbone for building the Fenics project. I have found 

however, running benchmarks that FreeBSD and OpenSolaris for scientific

computing is more efficient than Linux or OSX.

This is a bold claim.

 

For example the same executable

compiled with the same compiler and linked to the same BLAS libraries both in 

Linux and OpenSolaris runs 5% faster on OpenSolaris.

I'd like more details on this.  The kernel is almost a negligible factor in high performance linear algebra.  Was this running single or multi-threaded?  On what architecture?  

 

This was a great 

surprise. I would like to continue testing and benchmarking, but I cannot do 

that without being able to install FEniCS which uses hasdist. How hard is

it to port it to FreeBSD or OpenSolaris.

It should be relatively trivial since we build everything from source.  I would try running the Linux builds and adjust them as needed.

A

[Chris Kees]

Simeon,

We'd be happy to help you. As Aron said, you can try building the linux  stacks for  fenics first. Unless  one of the fenics  guys on the list would  like  you to do it  differently, you can fork  the repositories here: https://github.com/hashdist/hashstack and  https://github.com/hashdist/hashdist and build a fenics stack with 

$ cd hashstack

$ cp examples/fenics-dev.debian.yaml fenics-dev.solaris.yaml

$ ../hashdist/bin/hit build fenics-dev.solaris.yaml

Post the issues that arise to  https://github.com/hashdist/hashstack

I agree that  comparison of the performance profiles could be interesting if you see large differences  in runtimes for  different  OS's on the same  hardware. 

--
You received this message because you are subscribed to the Google Groups "hashdist" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hashdist+u...@googlegroups.com.
To post to this group, send email to hash...@googlegroups.com.
Visit this group at http://groups.google.com/group/hashdist.
To view this discussion on the web visit https://groups.google.com/d/msgid/hashdist/CAPhiW4gd6q%3DgPiV7QNnw1Cb%2BmCPW%2BUReWDT3%2BNmd73s09RLqZQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Johannes Ring]

On Tue, Oct 20, 2015 at 5:18 PM Chris Kees <cek...@gmail.com> wrote:

Simeon,

We'd be happy to help you. As Aron said, you can try building the linux  stacks for  fenics first. Unless  one of the fenics  guys on the list would  like  you to do it  differently, you can fork  the repositories here: https://github.com/hashdist/hashstack and  https://github.com/hashdist/hashdist and build a fenics stack with 

$ cd hashstack

$ cp examples/fenics-dev.debian.yaml fenics-dev.solaris.yaml

The FEniCS examples in HashStack is unfortunately not up-to-date, so it would probably be better to start with one of the HashDist profiles from the fenics-developer-tools repository:

$ wget https://bitbucket.org/fenics-project/fenics-developer-tools/raw/ad773d09d5d2bf6f93d9fbd69efd49da399408ee/install/profiles/fenics.Linux.yaml -O fenics.solaris.yaml

$ ../hashdist/bin/hit build fenics.solaris.yaml

Johannes

PS: I will try to update the FEniCS examples in HashStack soon.

To view this discussion on the web visit https://groups.google.com/d/msgid/hashdist/CAOVFbFgc%2BrCKpFGSvh5K9xQcN81U%2B8oKAL6dnhifFS%2B6fsHGjA%40mail.gmail.com.

[Simeon Nifos]

On Tuesday, October 20, 2015 at 4:55:21 PM UTC+2, Aron Ahmadia wrote:

Hi Simeon,

It would be great if you provide support for FREEBSD and OpenSolaris. 

HasDist is the backbone for building the Fenics project. I have found 

however, running benchmarks that FreeBSD and OpenSolaris for scientific

computing is more efficient than Linux or OSX.

This is a bold claim.

No!

 

 

For example the same executable

compiled with the same compiler and linked to the same BLAS libraries both in 

Linux and OpenSolaris runs 5% faster on OpenSolaris.

I'd like more details on this.  The kernel is almost a negligible factor in high performance linear algebra.  Was this running single or multi-threaded?  On what architecture?  

It was an 8 Dual Core Opteron^TM CPUs 870 1MB L2 cache per core, 2GB RAM per core. The Linux kernel was recompiled from scratch and tuned for the particular machine. Several kernels with different patches were compared and the fastest one was used to compare against Solaris. The code was using OPENMP, sparse linear algebra, domain decomposition. Compiled with g++-4.2.2 and linked with ACML in both Linux and Solaris. About 5% faster code and higher scalability was observed for any number of processors especially for OMP_NUM_THREADS=8 and 16, for any number of partitions. Sorry but this is true.

[Aron Ahmadia]

On Tue, Oct 20, 2015 at 1:46 PM, Simeon Nifos <arch...@gmail.com> wrote:

Compiled with g++-4.2.2 and linked with ACML in both Linux and Solaris. About 5% faster code and higher scalability was observed for any number of processors especially for OMP_NUM_THREADS=8 and 16, for any number of partitions. Sorry but this is true.

I'd be interested in seeing these results with a newer compiler, g++-4.2 is more than 8 years old at this point.  I don't know if your hot spots were in ACML or in your OpenMP-based code, so it's hard to say where the performance improvements are coming from.

I'm not challenging your assertions or experience here.  After all, both IBM and Cray use their own custom light-weight kernels instead of the standard Linux kernel.  

As for HashDist support for OpenBSD/Solaris, it's definitely interesting to us.  I'm happy to help you fix any bugs you find in building for Solaris.

A

[Ondřej Čertík]

On Tue, Oct 20, 2015 at 11:57 AM, Aron Ahmadia <ar...@ahmadia.net> wrote:
>
> On Tue, Oct 20, 2015 at 1:46 PM, Simeon Nifos <arch...@gmail.com> wrote:
>>
>> Compiled with g++-4.2.2 and linked with ACML in both Linux and Solaris.
>> About 5% faster code and higher scalability was observed for any number of
>> processors especially for OMP_NUM_THREADS=8 and 16, for any number of
>> partitions. Sorry but this is true.
>
>
> I'd be interested in seeing these results with a newer compiler, g++-4.2 is
> more than 8 years old at this point. I don't know if your hot spots were in
> ACML or in your OpenMP-based code, so it's hard to say where the performance
> improvements are coming from.

To dig into this more, try to create a simple benchmark that still
reproduces this speedup. That way we can figure out which part of the
code is providing the speedup.
Definitely use the latest compilers if you can.

Ondrej

>
> I'm not challenging your assertions or experience here. After all, both IBM
> and Cray use their own custom light-weight kernels instead of the standard
> Linux kernel.
>
> As for HashDist support for OpenBSD/Solaris, it's definitely interesting to
> us. I'm happy to help you fix any bugs you find in building for Solaris.
>
> A
>

> --
> You received this message because you are subscribed to the Google Groups
> "hashdist" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to hashdist+u...@googlegroups.com.
> To post to this group, send email to hash...@googlegroups.com.
> Visit this group at http://groups.google.com/group/hashdist.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/hashdist/CAPhiW4haqi5bievsLye0U72P6CH8Wpyo4NASx54rgrKLYcAx%2Bw%40mail.gmail.com.

[Simeon Nifos]

I do not have access to his machine any more. The only thing I could do 

for you is to run a similar code under a virtual machine. Perhaps in some

time in the future I could install both OS in one of the machines we have.

But the computer these experiments were conducted is out of reach. Perhaps

you could try to perform similar tests. You only need to install PETSc based

FEniCS and solve a problem using a domain decomposition method under 

both Solaris and Linux.

[Ondřej Čertík]

On Tue, Oct 20, 2015 at 12:53 PM, Simeon Nifos <arch...@gmail.com> wrote:
> I do not have access to his machine any more. The only thing I could do
> for you is to run a similar code under a virtual machine. Perhaps in some
> time in the future I could install both OS in one of the machines we have.
> But the computer these experiments were conducted is out of reach. Perhaps
> you could try to perform similar tests. You only need to install PETSc based
> FEniCS and solve a problem using a domain decomposition method under
> both Solaris and Linux.

Unfortunately I do not have access to Solaris either.

Ondrej

[Simeon Nifos]

I will do it soon reinstall Solaris on the big workstation we have. The question is

which BLAS library does a better job. MKL was faster in my time than any other

competitor and also thread safe. ATLAS was not thread safe. ACML and SunPerf were

thread safe but not as efficient as MKL. However MKL does not support FreeBSD

or Solaris. These days we have OpenBLAS. Have you tried to compare FEniCS

built with MKL support and FEniCS with OpenBLAS for performance differences?

[Simeon Nifos]

I am trying to do that first on Linux Ubuntu 12.04 LTS and if I succeed I will try on Solaris.

I cannot do that:

 ../hashdist/bin/hit build fenics.solaris.yaml

So I followed the instructions on

 http://fenicsproject.org/download/installation_using_hashdist.html

doing 

$ install/fenics-install.sh fenics.solaris.yaml

Do I need to install hasdist? If yes, then what is this "pip" thing?

Is it something python related?

[Johannes Ring]

The steps for downloading HashDist and HashStack is missing in the steps from Chris. This should work:

git clone https://github.com/hashdist/hashdist.git

git clone https://github.com/hashdist/hashstack.git

cd hashstack
wget https://bitbucket.org/fenics-project/fenics-developer-tools/raw/ad773d09d5d2bf6f93d9fbd69efd49da399408ee/install/profiles/fenics.Linux.yaml

../hashdist/bin/hit build fenics.Linux.yaml

On Wed, Oct 21, 2015 at 12:19 PM Simeon Nifos <arch...@gmail.com> wrote:

I am trying to do that first on Linux Ubuntu 12.04 LTS and if I succeed I will try on Solaris.

I cannot do that:

 ../hashdist/bin/hit build fenics.solaris.yaml

So I followed the instructions on

 http://fenicsproject.org/download/installation_using_hashdist.html

That page is actually not official yet and I need to do some more work on it.

 

doing 

$ install/fenics-install.sh fenics.solaris.yaml

Do I need to install hasdist? If yes, then what is this "pip" thing?

Is it something python related?

Yes, pip is a tool for installing python packages.

Johannes

 

To view this discussion on the web visit https://groups.google.com/d/msgid/hashdist/444dee2a-8fc4-4cfb-b685-cfaac92718a8%40googlegroups.com.

[Johannes Ring]

On Wed, Oct 21, 2015 at 12:19 PM Simeon Nifos <arch...@gmail.com> wrote:

I am trying to do that first on Linux Ubuntu 12.04 LTS and if I succeed I will try on Solaris.

BTW, the upcoming 1.7 release of FEniCS (to be released in December/January) will require GCC 4.8, which is not available in Ubuntu 12.04 (at least in the official Ubuntu archive). Ubuntu 14.04 LTS might be a better choice.

Johannes

[Simeon Nifos]

Trying to compile on Linux Ubuntu 12.04 LTS I get the following:

[mpi] Building mpi/gmkx3ftk3tz2, follow log with:

[mpi]   tail -f /home/archon/.hashdist/tmp/mpi-gmkx3ftk3tz2/_hashdist/build.log

[mpi|ERROR] Command '[u'/bin/bash', '_hashdist/build.sh']' returned non-zero exit status 1

[mpi|ERROR] command failed (code=1); raising

The error is:

2015/10/21 13:27:14 - INFO: [package:run_job] checking whether fort77 accepts the FLIBS found by autoconf... no

2015/10/21 13:27:14 - INFO: [package:run_job] checking for valid entries in FLIBS...

2015/10/21 13:27:14 - INFO: [package:run_job] checking whether gcc links with FLIBS found by autoconf... yes

2015/10/21 13:27:14 - INFO: [package:run_job] checking whether Fortran 77 and C objects are compatible... no

2015/10/21 13:27:14 - INFO: [package:run_job] checking for file... file

2015/10/21 13:27:14 - INFO: [package:run_job] configure: error: Could not determine a way to link a Fortran test program!

2015/10/21 13:27:14 - INFO: [package:run_job] checking for linker for Fortran main program...

2015/10/21 13:27:14 - ERROR: [package:run_job] Command '[u'/bin/bash', '_hashdist/build.sh']' returned non-zero exit status 1

2015/10/21 13:27:14 - ERROR: [package:run_job] command failed (code=1); raising

This is because it doesn't automatically link with -lgfortran. It should, shouldn't it?

If I modify manually the script

/home/archon/.hashdist/tmp/mpi-gmkx3ftk3tz2/_hashdist/build.sh 

adding FC=gfortran, F77=gfortran, CC=gcc, CXX=g++, LIBS="-lgfortran"

(by default it selects Intel fortran compiler)

 

then the whole process will go on up to the 

make install

where it will complain that there are not privileges to install some mpl.so library

or something similar. But even if I modify the build.sh then I have to change dir

in /home/archon/.hashdist/tmp/mpi-gmkx3ftk3tz2/ and run 

$ sh _hasdist/build.sh

It doesn't work if I rerun fenics-install.sh

Ho can I instruct the script .yaml to use mph libraries of the system?

Any ideas?