Multiple MPI Sessions on a single Multi-CPU Workstation

[MScFire Student]

Question,

My workstation has two E5-Xeon CPUs, each with 8 Cores giving 16 Cores total (32 Threads total with Hyper Threading) running Windows 7 Professional 64Bit.

I have run a single 6 mesh FDS job using MPI by applying 1 mesh per MPI Process per 1 Thread but I want to run the same job 5 times, changing a variable slightly for each job. That would be 5 FDS jobs with 6 meshes each running simultaneously, each using MPI by applying 1 mesh per MPI process per 1 Thread, thus using 30 of the available 32 threads.

However, windows will not let me run multiple MPI sessions at a time on the workstation, it only allows me to run 1 FDS job through MPI and the other 4 FDS jobs through OpenMP (which is ridiculously slow). 

Is this a limitation of Windows, or MPI, or is there a command set that I can run to allow multiple sessions of MPI on the same platform with 2 CPUs?

If this is a limitation, has anyone tried to set up multiple virtual windows 7s on the same platform and run MPI across this?

If so, I was thinking of installing VirtualBox to setup a virtual windows 7 on a 2ndry HD, then clone it 3 times to give me 4 virtual windows 7. Setup a shared folder on the "true" windows 7 SSD and then run the 5 FDS jobs using MPI this way?

Anyone got any comments or ideas?

[Glenn Forney]

your saying you can't open up two dos windows and submit an MPI fds job in each window?

--
You received this message because you are subscribed to the Google Groups "FDS and Smokeview Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fds-smv+u...@googlegroups.com.
To post to this group, send email to fds...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fds-smv/7c0d7956-e35c-4291-80f5-0b90b0cf5bec%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Glenn Forney

[MScFire Student]

Glen,

Correct. I can have all 5 MPI FDS jobs ready to execute in their own DOS windows but the other 4 won't execute once the 1st MPI FDS job is running in its DOS window.

On Thursday, June 25, 2015 at 1:36:20 AM UTC+1, GF wrote:

your saying you can't open up two dos windows and submit an MPI fds job in each window?

--

Glenn Forney

[Kevin]

I cannot do it either. I started a 4 process MPI job in one command shell on my Windows 7 computer, but when I opened another shell and tried to run a second job, it would not start. My guess is that we are not using a job scheduler. On our linux cluster, we use PBS (Portable Batch System) to schedule the jobs; that is, choose appropriate nodes to run the cases and assign priority and so on. I don't know the equivalent under Windows, but I suspect that this is what Windows PowerShell is designed to handle. I've never used it, but I think PowerShell provides a Windows administrator with tools normally found on unix/linux systems.

Has anyone ever run the MPI version of FDS under Windows PowerShell? Is it possible to run multiple MPI jobs?

[MScFire Student]

Had a chat with the computer lab techs here and a few PhD coders, they all suggested that you cannot have more than one MPI job running on the same windows platform, you can queue the jobs but not run them simultaneously.

I'll have a go at setting up virtual windows on the HPC and see if I can run multiple MPI jobs through the various virtual desktops.

[MScFire Student]

Update.

Installed VirtualBox v4.3.28 onto the HPC and set up 3 virtual windows 7 pro workstations. Each got 8 threads, 16GB RAM, & 50GB SSD space.

I've successfully ran 4 versions of the same MPI FDS job on the HPC, with a single MPI job on the "4" PCs (original and 3 clones). So far the 3 MPI FDS jobs running on the clones are a little slower than the original.

Will update with times once I've got some definite comparisons.

[Bryan Klein]

Hello,

This is a screencast showing how to run two MPI cases at the same time through PyroSim.

http://screencast.com/t/hRGZ2pVUg3OP

Best Regards,

-Bryan Klein

[MScFire Student]

Bryan,

I constructed the full scale experimental geometry (10 storey) model and the extracted sensitivity study (2 storey including fire compartment) model using PyroSim 2015.2.0604 (student license, cheers!). No way I was going to be able to type in the full geometry using FDS 6.2.0, well not in any reasonable time frame!

The plan was to use the extracted 2 storey model to carry out a sensitivity study on 4 different FDS parameters. To do this I was going use FDS 6.2.0 to run 5 MPI cases simultaneously per parameter, with a slight change in the relevant parameter per MPI case. This would have given me 5 results for each of the four parameters from which I could develop the "optimum" scenario for the full 10 storey model. That plan has been modified slightly, as per the last post.

Following the study, I am going to run the full 10 storey model (32 meshes, 5.5 million cells) with the "optimum" FDS parameters using FDS 6.2.0 (with 32 meshes, 32 MPI processes, 32 threads, 3.6GHz per thread, with roughly 2GB RAM per mesh).

So far the virtual PCs seem to be doing the trick for the sensitivity study and then they will all be removed for the follow on full 10 storey model.

Paul.