Question on Linux clusters

[Hostikka Simo]

We are considering a purchase of small Linux cluster
for FDS computations. Now I should choose between
- a bunch of 8 identical computers (8 cores each)
- an actual cluster with one front 'node' and bunch of computing
'nodes'.

In the first option, the user needs to deside in which computer
to run his/her jobs. In the second, user only logs in to the
front node and the starts a job, using possibly some sort of
queue software. Actually, the user sees the cluster as a single
computer.

Questions to those of you using Linux clusters:
1. Which one of the above are you using? (or something else)
2. Are there obvious problems in either option?

One thing I'm a bit concerned is the running of parallel jobs.
In FDS, the first mesh both makes computation and controls others.
In real cluster, must the first process (first mesh) be running
on the first node of the cluster? How about traffic, is the network
and disk traffic going to reduce the speed, if all the traffic
takes place or goes through one machine?

Thanks,
Simo

Simo Hostikka
Senior Research Scientist
Fire Research
VTT Technical Research Centre of Finland
P.O.Box 1000, FI-02044 VTT, Finland
+358 20 722 4839
simo.h...@vtt.fi

[dr_jfloyd]

We use a head node that contains a RAID array and the remaining nodes
are diskless. Logins are to the head node, but jobs are submitted via
ssh to the other nodes. Multi-mesh jobs do not have to use the head
node as one of the nodes. For most FDS runs, there does not appear to
be a lot of demand on the head node or the network when writing output
files with the exception perhaps of restart files for large jobs.

> simo.hosti...@vtt.fi

[Kevin]

At NIST we use a cluster of Dell desktop computers running Red Hat
Linux (Fedora Core 6). We chose the Fedora Core option because it has
been more stable for us.

In terms of FDS, in the parallel version, the only thing special about
the first "node" or "thread" is that we "gather" all the global print
outs (like the .out, _hrr.csv, _devc.csv, etc.) to node 0 so that they
can be printed out to a single file. All the other output files are
written by the individual nodes. We refer to node 0 because MPI is
written by C programmers, who like to begin arrays with 0. So we
assign mesh 1 to node 0, mesh 2 to node 1, and so on. But really, the
meshes are all being processed in exactly the same way except for the
periodic print-outs. We do not use the "master-slave" model for
parallel computing. All threads are more or less the same.

FYI -- we're slowly removing the convention that the number of threads
must equal the number of meshes. For the evacuation "meshes", we want
to put them all within the same thread to eliminate the need to
exchange info machine to machine. It's a bit confusing because we now
must work with meshes, threads, CPUs, cores, machines, single or
distributed memory.

On Jan 11, 9:20 am, "Hostikka Simo" <Simo.Hosti...@vtt.fi> wrote:

> simo.hosti...@vtt.fi

[cwood]

We use a pre-configured dual, quad-core system (8 CPUs) w/16GB RAM &
1TB RAID6 disk (double-disk failure redundancy). The difference is
that we use clustermatic (see, www.clustermatic.org) as the operating
system, which is linux based. The system I describe is a deprecated
cluster meaning that it is setup to have multiple devices connected on
its private (2nd NIC card) network, but currently has only the master
node. Thus, the head or "master" node is a node with 8 CPUs.
However, to expand the system I need only add another node (with 1..n
CPUs), setup for PIXI boot, and usually a very small disk for local
storage (i.e. SWAP space). The key to this system is that there is no
node software maintenance. The PIXI boot causes the node to request
operating software from the master node so that updates only need
occur on the single master rather than across all nodes. In my
previous experience, we several times expanded cluster by making this
additional node purchase and just plugging it in. Node disk mounts
are automatic so that it just looks like one big machine with numerous
available processors. Job kickoff and data transfer is seemless
because everything appears to be on the master node. This also means
that inter-node security issues are non-existent and virus checkers
need not run on the private network because items can only come and go
through the master which is the only node to have an external
connection, CD/DVD/floppy drives, etc. Virus checkers can be a
substantial CPU drain when there is significant cluster chatter (i.e.
during normal parallel simulations). I have been involved in the
setup, running, and maintenance of many of these clusters as well as
WIN-XP OS based clusters. I prefer the clustermatic OS system. I
also prefer linux based systems for computational machines.

[Ilya]

We use Rocks Cluster http://www.rocksclusters.org
It is very simple in installation http://www.rocksclusters.org/rocks-documentation/4.3/
It bases on CentOS 4 (Red Hat Enterprise Linux 4) and contains very
powerful Rools like Sun Grid Engine job queueing system, Ganglia
Cluster monitoring system from UCB and others.

Best Regards,
Ilya Karkin

> simo.hosti...@vtt.fi

[Hostikka Simo]

Thanks everybody for the replies so far. Let them come...

> We use Rocks Cluster http://www.rocksclusters.org

This is exactly what our IT-folks at VTT suggested for our cluster.

Simo