Characteristics of a PC dedicated to FDS computing.

[andrea...@gmail.com]

Hello,

What are the characteristics that a PC dedicated to FDS computing should have? 

I would like to know your opinions on the topic, and, if possible, get some basic explanation about why you reccomended a specific component/characteristic and/or links / key words to educate myself on the topic.

Thank you,

Andrea

[F-Sim.de]

Andrea,

FDS does a lot of floating point operations. A good system for doing FDS simulations:

will have a high performance in fp operations,

many cores,

a lot of fast ram, at best ECC

[Mikkel Lund Mortensen]

I mainly use FDS with building models that consider smoke as the primary objective and have between 1 and 10 million cells. I use dual processor workstations (HP z620) with xeon cpu's (E5-2680v2) and ecc ram. I'm not sure if the ram speed affects the speed very much, but I don't have different ram systems that are comparable, so can't say for sure. I can recommend buying used xeon processors, they use more power than the new ones, but have still have decently high frequency when all cores are working 100% and are much cheaper than a new cpu.

[Salah Benkorichi]

To have an appropriate answer, you need to explain clearly what you intend to do with fds, then people can give more suggestions.

The RAM is not going to speed your calculation, it will enable you to run big models with big cells number. In term of speed, you ought be looking for the clock speed of CPU, 

Are you interested in running parallel jobs or not. 

How generally you divide your model into mpi process, do you use frequently mpi parallel or not. 

You might find it effective and cost benefit if you are running simple models up to 1 million by simply having 6-8 cores machines, running one job wont take longer than 2 days, sometimes 15 hours only depending on what you have in your model.

--
You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+u...@googlegroups.com.
To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/767536b3-0108-4210-81a5-b68be70845b2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[andrea...@gmail.com]

Thank you for your answers.

I will need FDS for relatively small jobs initially ( about 500.000 cells), but the complexity of the jobs will increase, I guess up until millions of cells.

For smaller projects I may be interested in running parallel jobs.

"How generally you divide your model into mpi process, do you use frequently mpi parallel or not"

I've never used those function, I've always run FDS with the default settings; I know that these kind of choices are very important, and I intend to read the guide to understand them better (I've already done that, but I'm not so educated on the whole computer processing subject, so I need some time). My goal is to exploit those function at their best, once I know their function.

Andrea

[Michael Abbot]

I've found Xeon based machines at 2.2 GHZ and 8 cores seem to perform well when processing 8 multiple meshes. A similarly specc'd 8 core Intel I7 4.55 GHZ cores was observed to take longer for the same processing tasks. It's more about how a processor, ram, and HDD work together than it is about outright CPU speed. 

Single core processing of small FDS projects were (obviously) observed to process faster on the Intel I7 than on XEON processors.  

Regards,

Michael Abbot

--

You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+u...@googlegroups.com.
To post to this group, send email to fds...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/e1d7799d-8c97-4815-91c2-9970453ba2ec%40googlegroups.com.

[Sascha Gottfried]

Andrea,

getting results from FDS models involves a lot of tasks that take time and have their own challenges. You might gain some time with a fast PC, it might take even more time each time you are doing a manual results exploration/visualization or any other prepending or following task that is not well supported by good tools. Therefore the number of expected model iterations has more impact from a project management perspective.Many people, usually people rather new to FDS, focus on hardware specs. But usually the hardware is not a significant constraint for projects with smaller jobs.

There is one notable constraint: Each million cells require at least 1.0 GB up to 1.5 GB RAM. Given that you have 16 GB RAM you are limited to run a single model with up to 10 million cells - given that the baseline memory consumption of your operation system is rather low. The baseline memory consumption is rather high for a typical Windows 10 PC, where browsers, office suites etc. run simultaneously with FDS, Smokeview and/or PyroSim.

Just do, finish some smaller projects, identify constraints during a review. You might ask here as well how others remove these (technical) constraints. If necessary refine the (hardware) requirements for upcoming projects, specify the upcoming (parallel) workloads and tell us your budget as well. Given these specs you might get different answers, just as Salah Benkorichi previously mentioned for good reasons.

Good luck!

Sascha

[Chris Salter]

I undertook a number of benchmark tests previously using the hardware of volunteers to benchmark a simple FDS model that was distributed with FDS 5.

You should be able view the results here. These were split down between Multi core and Single core. The single core test is a simple FDS file that runs on a single core only and tests the speed, whilst the multi core looked at splitting the results across the different models. Part of this was done for testing to investigate the use of cloud computing for FDS models for industry (that paper is here and a similar one investigating costs by Edmund Ang is here).

I've found that the non server based chips, i.e. standard off the shelf desktop processors are quicker than the Intel workstation based chips, such as the Xeon range. However, they support a number of features such as ECC memory checking which can be beneficial.

Clock speed isn't the only thing you should look for, because as you can see from my benchmarks (albeit, not 100% scientific), that slower clock speeds from various CPU's are faster, though usually from an older generation of chips.

My personal preference is the new AMD Ryzen chips, as they are significantly cheaper than the Intel counterparts, are generally GHz for GHz are faster. However, they only really go up to 8 cores (or 32 if you use an AMD Threadripper, which I've not had the chance to test). Xeons can go up to 12 cores each IIRC and you can get dual processor machines, to up that to 24 total physical cores. You then have to consider the effect of hyper threading, which is essentially fake cores visible to the operating system. My experience that using these additional cores slows the model down if it's trying to use them, to the extent that we disabled hyper threading on a few machines in the office. My understanding is that hyperthreading attempts to use the different areas on the processor during each clock cycle, but if you're doing a lot of the same calculation, you'll see no benefit, as each core is trying to use the same part of the processor.

The number of cores comes in to play when debating if you're running via OpenMP or MPICH. For MPICH, you'll have to split the model into different meshes and assign the meshes to a processor - i.e. if you have a 4 core processor, you're best splitting the model in to no more than 4 meshes. If it's OpenMP, I've found (and I believe it's in the user guide as well) that the speed up past 4 cores is negligible (and MPICH is usually still quicker anyhow). The size and distribution of the meshes also plays a part. It's not an exact science.

Of course, other factors come in to play, such as what else is running on the system etc. In the past, I've tried running Linux and Windows benchmarks on the same machine and I think in a single case that I was able to test, I saw about a 10% increase in speed running Linux on the machine (though obviously that's a single data point and I've not been able to test much since then).

I've not gone in to how the different sub models effect the run time or how the other factors within the computer effect it (such as disk speed, RAM speeds, matched pairs of RAM, writing to a network file storage etc), so these are perhaps not the most scientific but purely just a download, install and run type affair.

Hope that helps.

[Andrew]

One of the main disadvantages with the AMD Ryzen chips is that other than the high end Threadripper cpus, you only get dual channel memory support.  This is unliekly to be an issue for the 4 core processors, however if you are running the larger core counts, particularly on big models you may find it slows down the calculations.

[o...@aquacoustics.biz]

When I set up my cluster I was faced with this very problem.  I wanted fast model processing at a budget price.

A really good place to start is to read the FDS Users Guide.  This will advise minimum hardware requirements and matters that will likely affect processing speed. 

 

NIST use Intel compilers and these are optimized for Intel hardware.  While there are other processor manufacturers and compilers, you might reasonably expect to minimize grief and optimize performance by sticking to Intel hardware.  In my experience, for FDS, the GNU compiler works but is slower than Intel, and may create all manner of issues that are best avoided entirely. 

 

For an Operating system I would recommend a Linux flavor.  Linux is cheap (or free), well supported, and has significantly less processor burden than Windows.  Linux boots in seconds while Windows takes several minutes (there are exceptions such Surface Pro which appear to use SRAM or EEPROM resident software).  Yes, I do run FDS on Windows, but primarily for building models using PyroSim, and for optimization.  My commercial models are run on my Linux cluster.

 

The major processing component of any FDS model is the cpu burden (over other stuff like RAM speed and size, disk access and communications).  While Xeon, and more recent processors may have many cores, large cashe, fast bus speeds and so forth, Xeon processors are relatively slow (~2 GHz), and others throttle the clock significantly under increasing core utilization and/or temperature.  You’ll need to read the CPU data sheets to understand this.

 

Hyper-threading is usually of no value with FDS.  While you may think you have more cores, expect the model to slow down.

 

With any FDS model, avoid fully committing all available cores on each node.  This may slow the model run-time down.  This is particularly important on the Master node from where FDS is launched.

 

I settled on Intel I7 4097K processors with a Gigabyte motherboards.  These have a stable 4.4 GHz  turbo clock and, with enhanced water cooling and BIOS settings, these don’t throttle despite 100% core utilization.  If necessary they will also over-clock above 5.3 GHz, but I don’t do this as a matter of course.

 

For expansion you will need relatively fast networking.  I use Mellanox Infiniband at 40 Gbps, but 1 Gbps Ethernet is relatively standard these days and improving communication speed comes with only a few percent in model run-time performance.

 

I also have two I9 nodes with 10 (real) cores on my cluster.  These are thermally hopeless and throttle under moderate loads or core utilization.  I use these for low-burden meshes and running concurrent models.  I won’t be getting any more I9’s and have contemplated changing these out for more I7’s.

 

Of course this is all meaningless without some form of bench-marking.  If you want some relative performance measures of my cluster verses a number of others (including HPC super-computers) then link to http://fire.aquacoustics.biz/html/publications.html and download the link FDS 6.7.0 Verification Report.  NIST have a number of standard speed benchmark models for strong and weak scaling and general performance.  I have only recently started using these so I don’t have a meaningful history or comparative performance at this time.

 

So, in my opinion, get the fastest Intel processor that you can afford and use Linux.  

[andrea...@gmail.com]

Thank you all for the advices. Now I have something to base my decisions on.

Andrea