Release of FDS 6.6.0

[Kevin]

We have just posted a minor release of FDS and Smokeview, 6.6.0. The release notes can be found here.

The installation and operation of the Windows version has not changed, but for Linux, we are now compiling with the Intel MPI library. Previously, we compiled with Open MPI. The reason for the change is that we have found Intel MPI to be slightly faster and more reliable on our own cluster at NIST, but we are interested in hearing from other users.

For those of you who want to compile FDS yourselves, as opposed to running with the executable that we compile, the makefile and build scripts have been streamlined. We are interested in feedback. Our goal is that you need only checkout the FDS GitHub repository and run the make script for your particular target, but, of course, there are always minor complications. So let us know how it goes.

[o...@aquacoustics.biz]

Hi Kevin - a timely post.

I have just completed the FDS 6.6.0 upgrade (28 Nov 17), generally following the guidelines in the https://github.com/firemodels/fds/wiki/Compiling-FDS-with-GNU-Fortran,-OpenMPI-and-Intel-Performance-Libraries-in-Ubuntu-Linux Wiki.  I haven't completed verification yet, but FDS 6.6.0 appears to be running just fine on my Linux (Ubuntu 16.04) cluster with OpenMPI 3.0.0.0 over IB compiled with gfortran 7.2.0.  I've only run three multi-mesh models to date (my own benchmarks) but the cluster appears to be running slightly faster (say 10%) than my previous 6.5.3 IB installation with essentially identical results.  

There were only a few issues with the install:

I don't run Slurm but this is the default configure resource manager.  If you don't disable it in the configure parameters (--without-slurm) then make will fail with a recursive error -19.

I had some issues with PATH and LIBRARY_PATH despite following the Wiki instructions.  I figure that I don't have modules and ended up setting my environment variables directly in .bashrc.

For some reason the bundled Linux FSD-SMV install (which I like to run to save some time for SmokeView and directory structures) locked Permissions to the FDS6 directory path.  Maybe this is a condition of my own making but it was simply resolved with chmod as superuser.

My initial install attempt was with Ubuntu 14.02 but I had some problems due to changes in the graphics hardware stack which required a kernel upgrade to 4.4.0.  Kernel upgrades require re-compliation of my Mellanox OFED.  Although I found a work around it was easier to start with a clean install of Ubuntu 16.04 and the latest Mellanox OFED source code (which was compiled under gfortran-5).

My thanks to NIST for the latest update.  The 3D heat conduction is a particular area of interest which I hope to be experimenting with over multi-meshes in the next few days.  While single mesh cases work fine on the cluster they don't have the processing speed advantages of OpenMPI.

With kindest regards,  t.

[Salah Benkorichi]

Tim,

Glad it's working for you.

As for permission, I believe you wont have issues if you don't install it as a superuser. Always, try to use the default installation directory as well. 

Btw, you can set up the module environment by following the instructions here: https://github.com/firemodels/fds/wiki/Modules

Let us know if you have issues with it. 

I understand the issues that can arises with the Kernel, but all in all, make sure to make a backup before you upgrade, and try to upgrade, it's anyway going to be better for your system security. 

A side note: I was reading your installation setup during this past weekends, and how you handled it infiniband, etc. I found you're compiling fds gnu on all nodes. You don't have to do that -- Just make sure the master node is exporting the path to openmpi directory and fds directory, similarly with making each slave node seeing the directory of the master node. Similarly with the fds budle. I have them installed and compiled only on the master node, without having to redo it on all nodes, and it works. -- Acknowledgement goes to Glenn for this. 

Let me know in case you try it.

Salah

--
You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.
To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/d09d5c10-5ce8-4797-821a-1d639731d0a6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[o...@aquacoustics.biz]

Thank you for your instant reply Salah.

Yes, I do my FDS install as a super user, so this doesn't explain why I experienced the permissions issue, but maybe I am doing something else wrong?

I'll look at the modules in due course.  My .bashrc edits are working just fine for now.

I always make a system image before embarking on an upgrade.  So when it all turns to toast I can recover the previous install in around 30 minutes for the entire cluster.  This is also useful when I need to re-run an old model on a previous version of FDS  (which has happened more than once).

I don't compile FDS on all nodes, just the Master (maybe my instructions on this aren't clear).  This is then copied to the Slave0 node (which is necessarily different from the Master Node).  Once FDS is running between the Master and Slave0 over IB I simply clone Slave0 to the other nodes and adjust their names, IP addresses and rebuild the authorized keys.

t.

[Salah Benkorichi]

Yes, you will have an issue of permission if you don't install in the user directory -- I mean if you try to install in the root directory, then it won't proceed. Try to use the default i.e. the home directory. ~/FDS/FDS6/bin/fds --> This shouldn't need any permission.

Well, we have made modules available for Linux and Mac users, in case you are interested in using them as they're handy.

I don't think it's necessary to copy the directories to the other nodes --where NFS can take care of this.

--

You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.
To post to this group, send email to fds...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/8a6756b6-4a8a-4e30-95c9-a3d82f848013%40googlegroups.com.

[o...@aquacoustics.biz]

Getting back to Kevin's original post: do you have any metrics for the speed improvement of the Intel Fortran compiler over GNU 7.2?  t.

On Wednesday, 29 November 2017 00:28:15 UTC+13, Salah Benkorichi wrote:

Yes, you will have an issue of permission if you don't install in the user directory -- I mean if you try to install in the root directory, then it won't proceed. Try to use the default i.e. the home directory. ~/FDS/FDS6/bin/fds --> This shouldn't need any permission.

Well, we have made modules available for Linux and Mac users, in case you are interested in using them as they're handy.

I don't think it's necessary to copy the directories to the other nodes --where NFS can take care of this.

On 28 November 2017 at 11:18, <o...@aquacoustics.biz> wrote:

Thank you for your instant reply Salah.

Yes, I do my FDS install as a super user, so this doesn't explain why I experienced the permissions issue, but maybe I am doing something else wrong?

I'll look at the modules in due course.  My .bashrc edits are working just fine for now.

I always make a system image before embarking on an upgrade.  So when it all turns to toast I can recover the previous install in around 30 minutes for the entire cluster.  This is also useful when I need to re-run an old model on a previous version of FDS  (which has happened more than once).

I don't compile FDS on all nodes, just the Master (maybe my instructions on this aren't clear).  This is then copied to the Slave0 node (which is necessarily different from the Master Node).  Once FDS is running between the Master and Slave0 over IB I simply clone Slave0 to the other nodes and adjust their names, IP addresses and rebuild the authorized keys.

t.

--
You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+u...@googlegroups.com.

To post to this group, send email to fds...@googlegroups.com.

[o...@aquacoustics.biz]

I've just completed run-time metrics on the FDS 6.6.0 install compiled under GNU Fortran 7.2 over IB.  The tests use my own benchmark models and involves about 100 core hours of processing.  See attachment for summary graphs.

For low allocated MPI processes (say 1 to 4) performance is generally worse than FDS 6.5.3 by about 10%, but it improves to equity above about 8 MPI processes.

Open MP performance is abysmal (essentially flat) on all models with no significant performance benefit with increasing Open MP processes.  This is not a platform resource issue so I expect that the problem lies with GNU Fortran 7.2 implementation of Open MPI.

FDS 6.6.0 performance is solid over extended periods of processing.  The output of the models are comparable with earlier systems and between simulations.  The installation passes the abridged verification suite tests.

Perhaps the Open MP issues is part of the reason why you have changed the compiler from GNU to Intel?

t.

[Salah Benkorichi]

Nice work Tim, you've done a good job.

I wish you could do also some testing for Nbr of threads more than 4. If you could at least reach the nbr of the cores you have. For instance, if you have 8 cores, then you could go up to 7 or 8. Also, for MPI 8 and 16, I wish you could go beyond 2 and 1 as well (but maybe you might not afford it on your CPU). This would give more insightful results.

Tell me, how many cores do you have per CPU. 

I don't think the difference is due to Open MP. Intel is a commercial product and has better optimization than GNU. We didn't have time to do a profound analysis for different cases benchmarking. It's good to see users work on this and share these nice feedback. we had faced quite issues with GNU (when it comes to initialization of parameters) and we have addressed couple of them --Whenever you receive an error with it, please report it to the issue tracker. 

I don't see on which Graph you're comparing GNU with INTEL. I'm assuming both 6.5.3 (4.8) and 6.6.0 (7.2) were compiled under GNU. 

Good job !

Salah

--

You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.

To post to this group, send email to fds...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/aba47c9a-3986-4f08-80b2-6ed903190a3b%40googlegroups.com.

[Salah Benkorichi]

Tim,

Let's move the discussion here.

Please reply back to this thread https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!topic/fds-smv/jLswX2L3VgM 

Thanks

S

[Andrew]

Hi

Just a note to say that the new linux version of FDS using the intel MPI runs loads better on our cluster.  I think the main reasons are due to the core pinning and slurm integration as i can now run out of the box so thanks for the work on this.

[Kevin]

Good to know. Thanks.

[Andreas]

Same here, the intel MPI library works flawless for us. Without having done a benchmark testing it also feels faster. Good job, thank you!

[fde]

I see  FDS 6.7.0 is ready to be downloaded but the link in download page still shows 6.6.0. Is fds+smv+userguides bundle going to be available soon?

[Randy McDermott]

That's what I call real-time monitoring of the site!

I was just in the middle of pushing up the webpage changes.  Try now.

--

You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.
To post to this group, send email to fds...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/d058a649-3bf8-4cfc-86a4-617576d81c0b%40googlegroups.com.

[fde]

Yay!

On Wednesday, June 27, 2018 at 3:40:19 PM UTC+2, Randy McDermott wrote:

That's what I call real-time monitoring of the site!

I was just in the middle of pushing up the webpage changes.  Try now.

On Wed, Jun 27, 2018 at 9:38 AM, fde <yilmaz....@gmail.com> wrote:

I see  FDS 6.7.0 is ready to be downloaded but the link in download page still shows 6.6.0. Is fds+smv+userguides bundle going to be available soon?

On Thursday, November 2, 2017 at 10:39:00 PM UTC+1, Kevin wrote:

We have just posted a minor release of FDS and Smokeview, 6.6.0. The release notes can be found here.

The installation and operation of the Windows version has not changed, but for Linux, we are now compiling with the Intel MPI library. Previously, we compiled with Open MPI. The reason for the change is that we have found Intel MPI to be slightly faster and more reliable on our own cluster at NIST, but we are interested in hearing from other users.

For those of you who want to compile FDS yourselves, as opposed to running with the executable that we compile, the makefile and build scripts have been streamlined. We are interested in feedback. Our goal is that you need only checkout the FDS GitHub repository and run the make script for your particular target, but, of course, there are always minor complications. So let us know how it goes.

--
You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+u...@googlegroups.com.

To post to this group, send email to fds...@googlegroups.com.