different machines -> different results

[Céline BLANC]

Hello,

I simulated a extinguishment of fire with sprinklers.

I used 8 cores.

I launched the job twice : on my laptop and on a cluster.

The results are not the same, for example on the HRR curve. The input data are the same and the job characteristics in file.out too.

I understand that there may be differences, but I've found that they are not insignificant.

Why do we get this behaviour?

[dr_jfloyd]

Fire is chaotic. If there is any tiny difference in the simulation at early times, that can lead to observable divergence in the results over time.  

A couple possible causes are a difference in optimization for the two different versions of the compiler or a real number constant being initialized without explicit typing.

[o...@aquacoustics.biz]

With the same computer, same allocated resources, and same model, same version of FDS the results will be identical.

But change the computer, the allocated hardware resources, or FDS  and you can expect differences in the FDS results.  These are due entirely to the iterative nature of the calculation, and hardware architecture of the allocated cores, and how these implement the underlying mathematics (in particular floating point arithmetic) which is in part influenced by the compiler.

I was interested to study this on a number of different platforms some time back.  While the simulation results were different, if a fire safety design was so finely balanced that the outcome depended on the simulation environment then the design was not sufficiently conservative.  Similarly, the simulation variation was significantly less than what you might expect from the stochastic nature of fire in a repeated experiment.

.

[fde]

   NOISE_VELOCITY might also have an impact.  

[Céline BLANC]

What is it NOISE_VELOCITY?

[Kevin McGrattan]

This issue comes up now and then. NOISE=F is one way to eliminate some randomness because NOISE=T, the default, introduces a small amount of random air movement at the start of the calculation. We do not insist in the code that the "seed" of the random number generator be consistent over all platforms. 

But even if you make all efforts to eliminate the butterfly effect, there is one thing left to consider. We optimize the compilation of the code to make it run as fast as possible. These optimizations often involve speeding up basic arithmetic and number truncation and so forth. There are compiler options to maintain strict conformance with IEEE (or similar) standards, but these slow the code down to some extent. We want to give the compiler every opportunity to make things fast, and demanding strict numerical compliance hinders the optimization to some extent. 

[Céline BLANC]

Yes, I understand the differences that come from the compiler, but I wasn't expecting such discrepancies.
If I have time, I'll try with NOISE=F to see if the differences are reduced.

Thank you

Le mar. 28 nov. 2023 à 15:01, Kevin McGrattan <mcgr...@gmail.com> a écrit :

This issue comes up now and then. NOISE=F is one way to eliminate some randomness because NOISE=T, the default, introduces a small amount of random air movement at the start of the calculation. We do not insist in the code that the "seed" of the random number generator be consistent over all platforms. 

But even if you make all efforts to eliminate the butterfly effect, there is one thing left to consider. We optimize the compilation of the code to make it run as fast as possible. These optimizations often involve speeding up basic arithmetic and number truncation and so forth. There are compiler options to maintain strict conformance with IEEE (or similar) standards, but these slow the code down to some extent. We want to give the compiler every opportunity to make things fast, and demanding strict numerical compliance hinders the optimization to some extent. 

--
You received this message because you are subscribed to a topic in the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/fds-smv/jSZFLBVNYdg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to fds-smv+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/CAAJimDEsOQr5XPT6GgPo%3DS4rbvh4c5%3DdXrrvF-XrkgueFKS2Xw%40mail.gmail.com.

[Céline BLANC]

Hello,

So I tried with the option NOISE=.F. and the results are more similar.

I forgot to say that I represent the HRR curve.

One computation is performed with a MAC and the second on Linux.

Thank you for your comments.

[o...@aquacoustics.biz]

Dear Celine, you have produced two visually similar HRR histories on different hardware platforms, but neither is an assurance of the absolute accuracy of your model.  While they may look similar, in fact they vary in high frequency structure HRR by about 20% at about 80 undefined time units.  The bigger question is how representative either plot is of the physical reality that you are trying to model.  We cannot accurately model the profile of water flowing from a faucet at an instant in time, but we can calculate the average flow rate within useful bounds.  

For NIST.  A number of fire models incorporate options for statistical variation of input parameters to estimate probabilistic bounds on fire behaviour.  I appreciate that the underlying distributions and parameters are problematic but is this an area of  research for FDS?

[dr_jfloyd]

I don't know that the plots shows any significant difference in spectra. Yes oscillaitons are occuring at different times, but without the raw data and with one dataset plotted as symbols and one as lines I don't think you can say that these results are significantly different in terms of the underlying spectral distribution of oscillations. The validation guide reports the predictive uncertainty of FDS. The variances in HRR in both plots would not result in temperature or heat flux predictions falling outside those bounds. 

FDS is fully determinstic except for the initial NOISE and in the case of sprinklers the droplet insertion locations over time. Those two items use random numbers which might vary between operating systems; however, on a given operating system the same random number sequence will be generated each time you run a given input file. For the typical fire scenarios used in design, varying input parameters is going to simply lead to expected variance in outputs. Increase the fire size - flame height will get larger to the 2/5 power of heat release rate, plume temperatures will increase to the 2/3 power, etc. 

[Randy McDermott]

My take on ob1's question is whether we are actively trying to suss out the sensitivities of the many parameters that go into a fire model (?).  I would say that, while we do not have an explicit focus on this, it is something that is being implicitly addressed in the area of fire growth and flame spread, where two-way coupling with the solid phase often shows extreme sensitivities to model parameters.  Really, I would say this is one of the primary goals of MaCFP, to wrap our arms around the problem and try to figure out some hierarchy of importance for parameters and submodels in order to achieve reasonably mesh independent solutions for fire growth, both in compartment fires and in WUI fires.

You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/92e588fe-aeef-4b07-9e12-08c922314c26n%40googlegroups.com.

[Céline BLANC]

Hi,

Hello,

This is only a demonstration case. I cannot validate the results.
I did these tests because I ran the simulations on two platforms and naturally compared the results.
I posted on the forum to get an idea of where the differences came from.

There are random parts: noise, droplets. Is there also in the speed limit conditions with the SEM?

I tested without noise and you can see differences before the droplet injections.

So this remains the compilation.... 

[o...@aquacoustics.biz]

Dear Jason,

would you please explain how the random number generator in Fortran 90 is dependent on the Operating System?  The seed for the pseudo random number is fixed in the FDS source code and the routine for generating the numbers is defined in the Fortran compiler.  So any random numbers generated in a particular simulation must be deterministic.  This includes the random initialization of the non-solid domain, without which a simple model becomes highly dimensionally symmetric, and the insertion of droplets.

I am of the view that the differences in FDS results from running the same model on different hardware is due entirely to minor variations in how different CPU’s implement core functions (the native assembler of the hardware core).

Perhaps NIST might consider defining the random number generator seed in an externally defined variable for experimentation?  Many years ago I did try changing the seed in the source code but this was time consuming, I don’t recall and didn’t document the results.  I will try and recover the files from archive.   

[dr_jfloyd]

Forgot we call RANDOM_SEED when doing NOISE. Interestingly we don't set a seed if NOISE=F. We should either move those lines to before the IF (NOISE) CALL INITIAL_NOISE or always call INITIAL_NOISE and put an IF(.NOT. NOISE) RETURN after setting the seed.  

Having the abiltiy to set the seed is I think a good idea. 

[dr_jfloyd]

As an addendum, even though we set the seed, different compilers may use different random number generators. The ISO standard for Fortran doesn't specify what generator to use only that RANDOM_NUMBER should generate pseudorandom values 0 <= x < 1.