OMP StackSize

962 views
Skip to first unread message

Eric MARCHAND

unread,
Jan 24, 2018, 1:06:13 PM1/24/18
to FDS and Smokeview Discussions
Hi,

If I could easily find in the FDS 6.6 Manule, how to manage the configuration of my Workstation which has 32 cores (Bi Xeon E5-2690), I do not know what value to give to OMP_STACKSIZE, which is a parameter I used with PyroSim , in the past. With the 64-bit OS Win 7, what value advise when the WorkStation has 64 GB of RAM?.

Thank you in advance for your advice.

Eric

Salah Benkorichi

unread,
Jan 24, 2018, 1:18:50 PM1/24/18
to fds...@googlegroups.com
We are using intel MPI in FDS6.6.0. 

You might think to upgrade your RAM to 128GB (just suggestion) since you have 32 Cores. 


--
You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.
To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/76b1ebbd-0942-42b8-8f64-2e7aee7cb17f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Eric MARCHAND

unread,
Jan 24, 2018, 2:23:19 PM1/24/18
to FDS and Smokeview Discussions
Thank U for this suggestion.
And for a configuration with 128 GB of RAM, what value to give to the instruction:
Set OMP_STACKSIZE = value?


Le mercredi 24 janvier 2018 19:18:50 UTC+1, Salah Benkorichi a écrit :
We are using intel MPI in FDS6.6.0. 

You might think to upgrade your RAM to 128GB (just suggestion) since you have 32 Cores. 

On 24 January 2018 at 18:06, Eric MARCHAND <pole.ri...@gmail.com> wrote:
Hi,

If I could easily find in the FDS 6.6 Manule, how to manage the configuration of my Workstation which has 32 cores (Bi Xeon E5-2690), I do not know what value to give to OMP_STACKSIZE, which is a parameter I used with PyroSim , in the past. With the 64-bit OS Win 7, what value advise when the WorkStation has 64 GB of RAM?.

Thank you in advance for your advice.

Eric

--
You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+u...@googlegroups.com.

fde

unread,
Jan 25, 2018, 2:48:39 AM1/25/18
to FDS and Smokeview Discussions
For one calculation, a model with few million cells would not exceed more than 15 GB ram. What benefit would one get with 128 GB ram?


On Wednesday, January 24, 2018 at 7:18:50 PM UTC+1, Salah Benkorichi wrote:
We are using intel MPI in FDS6.6.0. 

You might think to upgrade your RAM to 128GB (just suggestion) since you have 32 Cores. 

On 24 January 2018 at 18:06, Eric MARCHAND <pole.ri...@gmail.com> wrote:
Hi,

If I could easily find in the FDS 6.6 Manule, how to manage the configuration of my Workstation which has 32 cores (Bi Xeon E5-2690), I do not know what value to give to OMP_STACKSIZE, which is a parameter I used with PyroSim , in the past. With the 64-bit OS Win 7, what value advise when the WorkStation has 64 GB of RAM?.

Thank you in advance for your advice.

Eric

--
You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+u...@googlegroups.com.

Salah Benkorichi

unread,
Jan 25, 2018, 3:18:37 AM1/25/18
to fds...@googlegroups.com
It will make a difference when you such a machine if you are planning to use it. Think of it on how much core would require generally and you would understand why I suggested that.

To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.

To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/811725d9-2f31-493b-b1d5-ca37730994cd%40googlegroups.com.

fde

unread,
Jan 25, 2018, 3:42:20 AM1/25/18
to FDS and Smokeview Discussions
It will certainly be useful to run multiple runs at a time. On the other hand, you may divide the domain into 32 cores and finish the calculation as fast as possible. 


On Thursday, January 25, 2018 at 9:18:37 AM UTC+1, Salah Benkorichi wrote:
It will make a difference when you such a machine if you are planning to use it. Think of it on how much core would require generally and you would understand why I suggested that.
On 25 Jan 2018 7:48 am, "fde" <yilmaz....@gmail.com> wrote:
For one calculation, a model with few million cells would not exceed more than 15 GB ram. What benefit would one get with 128 GB ram?

On Wednesday, January 24, 2018 at 7:18:50 PM UTC+1, Salah Benkorichi wrote:
We are using intel MPI in FDS6.6.0. 

You might think to upgrade your RAM to 128GB (just suggestion) since you have 32 Cores. 


On 24 January 2018 at 18:06, Eric MARCHAND <pole.ri...@gmail.com> wrote:
Hi,

If I could easily find in the FDS 6.6 Manule, how to manage the configuration of my Workstation which has 32 cores (Bi Xeon E5-2690), I do not know what value to give to OMP_STACKSIZE, which is a parameter I used with PyroSim , in the past. With the 64-bit OS Win 7, what value advise when the WorkStation has 64 GB of RAM?.

Thank you in advance for your advice.

Eric

--
You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+u...@googlegroups.com.
To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/76b1ebbd-0942-42b8-8f64-2e7aee7cb17f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+u...@googlegroups.com.
To post to this group, send email to fds...@googlegroups.com.

Eric MARCHAND

unread,
Jan 26, 2018, 4:14:38 PM1/26/18
to FDS and Smokeview Discussions
Hi,

We do some multiple runs at a time. Finally, can anyone advise us on the value of OMP Stacksize for 64 or 128 Gb of RAM?

o...@aquacoustics.biz

unread,
Jan 27, 2018, 4:44:59 AM1/27/18
to FDS and Smokeview Discussions
Firstly the OMP_STACK_SIZE parameter is an allocation size PER OMP THREAD.  Refer to an Open Multi-Processor user guide for suggestions on size allocation.  Without MPI the mesh size and model will determine the required OMP stack size.  With MPI and OMP this is less about the size of the model and more about the size of the arrays required to support the largest mesh OMP processes.  Note that insufficient OMP stack size is likely to result in fatal sig-segv errors but this is not guaranteed.  Sometimes your model will proceed with stack overflow and you will get garbage results.

As a suggestion allocate a ridiculous amount of memory (say 2G) and use a resource monitor (HTOP in Linux or Resource Monitor in Windows) to determine actual memory usage, divide by the allocated processors and then double this to account for mesh imbalance.  My understanding is that the arrays are generated early and are not dynamic, so you will see the actual memory allocation requirements early in a simulation.

Secondly, if you have 32 cores on a single node, then allocating all of the cores will probably slow your model down.  Fully committing a node is generally not a good idea.  While this sounds counter-intuitive there are several reasons for this.  1.  You need some resource for OS, coms, and the master node (from which other processes are spawned).  2.  With high core count processors the clock speed will be throttled as concurrent core allocation increases (to reduce power dissipation in the CPU).  3.  Increased parallel processing comes with decreasing returns (but MPI is almost always better than OMP).

The best thing you can do is try some short run experiments to see what allocations work best for your platform.

Salah Benkorichi

unread,
Jan 27, 2018, 3:21:57 PM1/27/18
to fds...@googlegroups.com
I don't understand why you have to specify that parameter. Has it proven it performs better when not doing so? 
Just set your OMP_NUM_THREADS=1 when running multiple processes together, and you should get a faster run.

To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.

To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/7a0230c1-8c4c-4429-a9e8-e9d63eb868f6%40googlegroups.com.

Tim O'Brien

unread,
Jan 28, 2018, 3:07:04 AM1/28/18
to fds...@googlegroups.com

PER OMP THREAD is not a parameter Salah - just my crude attempt at adding emphasis.  t.


For more options, visit https://groups.google.com/d/optout.

 

--
You received this message because you are subscribed to a topic in the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/fds-smv/WFLhOhTbb_o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to fds-smv+u...@googlegroups.com.


To post to this group, send email to fds...@googlegroups.com.

Salah Benkorichi

unread,
Jan 28, 2018, 8:05:22 AM1/28/18
to fds...@googlegroups.com
I was referring to the OMP_STACKSIZE as a parameter of OpenMP. You may refer to it as an environmental variable which is the accurate term to it.

The value should be set by default to 16M which what is recommended. But, I don't see a reason for specifying it manually unless you have a reason.

To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.


To post to this group, send email to fds...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/fds-smv/WFLhOhTbb_o/unsubscribe.

To unsubscribe from this group and all its topics, send an email to fds-smv+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.

To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/000001d3980e%24f5d7a1b0%24e186e510%24%40biz.

Tim O'Brien

unread,
Jan 28, 2018, 1:49:24 PM1/28/18
to fds...@googlegroups.com

Why change this environmental variable?  Refer to https://github.com/firemodels/fds/wiki/OpenMP-Notes.  Recommendation is 200M.  t.

 

From: fds...@googlegroups.com [mailto:fds...@googlegroups.com] On Behalf Of Salah Benkorichi
Sent: Monday, 29 January 2018 2:05 a.m.
To: fds...@googlegroups.com
Subject: Re: [fds-smv] OMP StackSize

 

I was referring to the OMP_STACKSIZE as a parameter of OpenMP. You may refer to it as an environmental variable which is the accurate term to it.

 

The value should be set by default to 16M which what is recommended. But, I don't see a reason for specifying it manually unless you have a reason.

On 28 January 2018 at 08:06, Tim O'Brien <o...@aquacoustics.biz> wrote:

PER OMP THREAD is not a parameter Salah - just my crude attempt at adding emphasis.  t.

 


For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/fds-smv/WFLhOhTbb_o/unsubscribe.

To unsubscribe from this group and all its topics, send an email to fds-smv+u...@googlegroups.com.


To post to this group, send email to fds...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+u...@googlegroups.com.
To post to this group, send email to fds...@googlegroups.com.


For more options, visit https://groups.google.com/d/optout.

 

--

You received this message because you are subscribed to a topic in the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/fds-smv/WFLhOhTbb_o/unsubscribe.

To unsubscribe from this group and all its topics, send an email to fds-smv+u...@googlegroups.com.


To post to this group, send email to fds...@googlegroups.com.

Salah Benkorichi

unread,
Jan 28, 2018, 3:00:10 PM1/28/18
to fds...@googlegroups.com
Thanks for the link.
Yes the default is 16M.
This is what I was looking for : 

If Windows (64-bit System) reports error messages like

  • OMP: Error #136: Cannot create thread.
  • OMP: System error #8: Not enough storage is available to process this command. try to reduce your OMP_STACKSIZE value if it is "large" (e. g. 1G). This has solved the problem for some tests.

In case you face such an error then you would need to increase it. But, I have never changed the default value, and didn't have any issues related to OMP_STACKSIZE.

Salah

To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.


To post to this group, send email to fds...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/fds-smv/WFLhOhTbb_o/unsubscribe.

To unsubscribe from this group and all its topics, send an email to fds-smv+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.


To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/000001d3980e%24f5d7a1b0%24e186e510%24%40biz.


For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/fds-smv/WFLhOhTbb_o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to fds-smv+unsubscribe@googlegroups.com.

To post to this group, send email to fds...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+unsubscribe@googlegroups.com.

To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/000001d39868%24b2525fc0%2416f71f40%24%40biz.
Reply all
Reply to author
Forward
0 new messages