Blue Mountain / Origin 2000
Bjorn Lindgren
SGI uses betwwen the O2000 "nodes" in LANL Blue Mountain
supercomputer, and that bandwidth it have (GByte/s).
I know they use CrayLink between the system boards inside
the O2000.
I'm also intrested to know what kind of "design" the
supercomputer has. I know O2000 is SMP/ccNUMA.
(single kernel image), but SMP/ccNUMA on that many CPUs?
I guess its a mix of MPP and SMP/ccNUMA. Someone can
maybe clarify for an layman. :)
- bln
John Stephens
wrote:
>Hi, I wonder if someone knows what kind of interconnect
>SGI uses betwwen the O2000 "nodes" in LANL Blue Mountain
>supercomputer, and that bandwidth it have (GByte/s).
>I know they use CrayLink between the system boards inside
>the O2000.
HIPPI.
>
>I'm also intrested to know what kind of "design" the
>supercomputer has. I know O2000 is SMP/ccNUMA.
>(single kernel image), but SMP/ccNUMA on that many CPUs?
>I guess its a mix of MPP and SMP/ccNUMA. Someone can
>maybe clarify for an layman. :)
>
>- bln
--
John Stephens (remove spaces for legal email address)
s t e p h e n s @ p o b o x . c o m
Nick Maclaren
John Stephens <step...@no-fixed-abode.com> wrote:
>On 8 Oct 1999 17:09:30 GMT, Bjorn Lindgren <b...@sunphocus.upp.se.pnu.com>
>wrote:
>
>>Hi, I wonder if someone knows what kind of interconnect
>>SGI uses betwwen the O2000 "nodes" in LANL Blue Mountain
>>supercomputer, and that bandwidth it have (GByte/s).
>>I know they use CrayLink between the system boards inside
>>the O2000.
>
>HIPPI.
Isn't it GSN (aka HiPPI 6400), rather than plain HiPPI 800? Or is
that the next upgrade?
>>I'm also intrested to know what kind of "design" the
>>supercomputer has. I know O2000 is SMP/ccNUMA.
>>(single kernel image), but SMP/ccNUMA on that many CPUs?
>>I guess its a mix of MPP and SMP/ccNUMA. Someone can
>>maybe clarify for an layman. :)
You have just done it! The system is an MPP cluster of SMP/ccNUMA
nodes. One way of programming such beasts is to use MPI between
nodes and OpenMP within the nodes - giving some delightfully complex
algorithmic and programming problems :-)
The Hitachi SR8000 and several other new supercomputers have similar
designs.
Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QG, England.
Email: nm...@cam.ac.uk
Tel.: +44 1223 334761 Fax: +44 1223 334679
Dan Nagle
Nick Maclaren wrote:
>
<snip ongoing discussion>
>
> You have just done it! The system is an MPP cluster of SMP/ccNUMA
> nodes. One way of programming such beasts is to use MPI between
> nodes and OpenMP within the nodes - giving some delightfully complex
> algorithmic and programming problems :-)
Why not use Co-array Fortran and leave the "delightfully complex
algorithmic and programming problems" to the compiler?
<snip rest and sig>
--
Cheers!
Dan Nagle dna...@erols.com
Purple Sage Computing Solutions, Inc.
Nick Maclaren
>
>Nick Maclaren wrote:
>>
>> You have just done it! The system is an MPP cluster of SMP/ccNUMA
>> nodes. One way of programming such beasts is to use MPI between
>> nodes and OpenMP within the nodes - giving some delightfully complex
>> algorithmic and programming problems :-)
>
>Why not use Co-array Fortran and leave the "delightfully complex
>algorithmic and programming problems" to the compiler?
Nice one. It was a joke, wasn't it?
Greg Lindahl
> >Why not use Co-array Fortran and leave the "delightfully complex
> >algorithmic and programming problems" to the compiler?
>
> Nice one. It was a joke, wasn't it?
Only problem is, nobody's laughing.
-- g
John Stephens
>In article <OaX+N=MU4unTlZ2ib...@4ax.com>,
>John Stephens <step...@no-fixed-abode.com> wrote:
>>On 8 Oct 1999 17:09:30 GMT, Bjorn Lindgren <b...@sunphocus.upp.se.pnu.com>
>>wrote:
>>
>>>Hi, I wonder if someone knows what kind of interconnect
>>>SGI uses betwwen the O2000 "nodes" in LANL Blue Mountain
>>>supercomputer, and that bandwidth it have (GByte/s).
>>>I know they use CrayLink between the system boards inside
>>>the O2000.
>>
>>HIPPI.
>
>Isn't it GSN (aka HiPPI 6400), rather than plain HiPPI 800? Or is
>that the next upgrade?
The next upgrade, but pretty soon now. It's possible some GSN links are
already in place, but it would have been quite recent.
>
>>>I'm also intrested to know what kind of "design" the
>>>supercomputer has. I know O2000 is SMP/ccNUMA.
>>>(single kernel image), but SMP/ccNUMA on that many CPUs?
>>>I guess its a mix of MPP and SMP/ccNUMA. Someone can
>>>maybe clarify for an layman. :)
>
>You have just done it! The system is an MPP cluster of SMP/ccNUMA
>nodes. One way of programming such beasts is to use MPI between
>nodes and OpenMP within the nodes - giving some delightfully complex
>algorithmic and programming problems :-)
>
>The Hitachi SR8000 and several other new supercomputers have similar
>designs.
>
>
>Regards,
>Nick Maclaren,
>University of Cambridge Computing Service,
>New Museums Site, Pembroke Street, Cambridge CB2 3QG, England.
>Email: nm...@cam.ac.uk
>Tel.: +44 1223 334761 Fax: +44 1223 334679
--
Per Ekman
> Hi, I wonder if someone knows what kind of interconnect
> SGI uses betwwen the O2000 "nodes" in LANL Blue Mountain
> supercomputer, and that bandwidth it have (GByte/s).
> I know they use CrayLink between the system boards inside
> the O2000.
HIPPI 6400 AFAIK. (6400 Mb/s)
> I'm also intrested to know what kind of "design" the
> supercomputer has. I know O2000 is SMP/ccNUMA.
> (single kernel image), but SMP/ccNUMA on that many CPUs?
> I guess its a mix of MPP and SMP/ccNUMA. Someone can
> maybe clarify for an layman. :)
I guess it's just a big cluster, you submit your jobs to a
scheduler and use MPI or whatever to communicate between
nodes. The O2K scales to 256 CPUs, although I think that
the Blue Mountain machines are 128 CPU ones. So they have
a number if SSI O2Ks in a fast network. If that should count as
one machine or not is another question entirely.
Pek
Bjorn Lindgren
: Bjorn Lindgren <b...@sunphocus.upp.se.pnu.com> writes:
: I guess it's just a big cluster, you submit your jobs to a
: scheduler and use MPI or whatever to communicate between
: nodes. The O2K scales to 256 CPUs, although I think that
: the Blue Mountain machines are 128 CPU ones. So they have
: a number if SSI O2Ks in a fast network. If that should count
: as one machine or not is another question entirely.
There are serval kernel images so it should count as serval
machines (think IBM SP2). I was a bit unsure about that part,
becoz I'd tought they'd (SGI) use some sort of special
shrink-warped version of IRIX (Celluar IRIX or what name is)
to build single kernel image systems out of serval Origion
2000 nodes. (SMP on a cluster)
- bln
Michael Woodacre
|> Bjorn Lindgren <b...@sunphocus.upp.se.pnu.com> writes:
|>
|> > Hi, I wonder if someone knows what kind of interconnect
|> > SGI uses betwwen the O2000 "nodes" in LANL Blue Mountain
|> > supercomputer, and that bandwidth it have (GByte/s).
|> > I know they use CrayLink between the system boards inside
|> > the O2000.
|>
|> HIPPI 6400 AFAIK. (6400 Mb/s)
|>
|> > I'm also intrested to know what kind of "design" the
|> > supercomputer has. I know O2000 is SMP/ccNUMA.
|> > (single kernel image), but SMP/ccNUMA on that many CPUs?
|> > I guess its a mix of MPP and SMP/ccNUMA. Someone can
|> > maybe clarify for an layman. :)
|>
|> scheduler and use MPI or whatever to communicate between
|> nodes. The O2K scales to 256 CPUs, although I think that
Actually, the Origin ccNUMA architecture scales to 1024 CPU's. Currently the
largest configurations that have actually been built are 512P.
Cheers,
Mike
|> the Blue Mountain machines are 128 CPU ones. So they have
|> a number if SSI O2Ks in a fast network. If that should count as
|> one machine or not is another question entirely.
|>
|> Pek
|>
--
Michael S. Woodacre
UUCP: wood...@sgi.com
Phone: +44 1118 925 7846 (work)
Home: +44 1249 721536 (home)
M/S: IGB-3110
Address : 1530 Lakeside
Arlington Business Park
Theale, Berkshire
Reading
RG7 4SB
United Kingdom
Marco Eschelwek
[snip]
> becoz I'd tought they'd (SGI) use some sort of special
> shrink-warped version of IRIX (Celluar IRIX or what name is)
Cellular IRIX ? As far as I know there ist still no final release
available. (But I'm not 100% sure) Because we've been waiting for this
OS for our Origin2000 for ages.
regards
marco
--
_________________________________
UNIX - using the computer smarter
_________________________________
John Stephens
wrote:
>On 12 Oct 1999, Bjorn Lindgren wrote:
>
>[snip]
>
>> becoz I'd tought they'd (SGI) use some sort of special
>> shrink-warped version of IRIX (Celluar IRIX or what name is)
>
>Cellular IRIX ? As far as I know there ist still no final release
>available. (But I'm not 100% sure) Because we've been waiting for this
>OS for our Origin2000 for ages.
You're going to wait a long time. The Cellular IRIX project was abandoned
at least 9 months ago.
Bjorn Lindgren
: On Tue, 2 Nov 1999 14:12:10 +0100, Marco Eschelwek <ma...@rz.tu-ilmenau.de>
: wrote:
:>On 12 Oct 1999, Bjorn Lindgren wrote:
:>
:>[snip]
:>
:>> becoz I'd tought they'd (SGI) use some sort of special
:>> shrink-warped version of IRIX (Celluar IRIX or what name is)
:>
:>Cellular IRIX ? As far as I know there ist still no final release
:>available. (But I'm not 100% sure) Because we've been waiting for this
:>OS for our Origin2000 for ages.
: You're going to wait a long time. The Cellular IRIX project was abandoned
: at least 9 months ago.
Do you know the reason why it was abandoned?
- bln
John Stephens
Not sure whether they figured they didn't know how to do it, or if the "rush
to Linux" got in the way.
SGI's planning has not been very consistent over the past year,
unfortunately. :-(
Dave Olson
| >: You're going to wait a long time. The Cellular IRIX project was abandoned
| >: at least 9 months ago.
| >
| >Do you know the reason why it was abandoned?
|
| Not sure whether they figured they didn't know how to do it, or if the "rush
| to Linux" got in the way.
|
| SGI's planning has not been very consistent over the past year,
| unfortunately. :-(
It was killed (as a major effort) about 15 months ago, well before any
of the earliest serious talk about moving towards linux.
It became clear that the customers who would benefit (a fairly
small percentage of origin customers) didn't justify the work, when
there was so much other work to do that was more broadly applicable,
and a limited number of people to do the work.
Some of the work did end up in irix 6.5 in various ways, CXFS,
HA, etc., so it wasn't completely abandoned/dropped. And some of
it may show up in future cluster work.
And some of the more general work shaped the origin follow-on
hardware and software as well.
And finally, IRIX 6.5 (as with 6.4) is still officially called
"cellular irix", even though the "true" cell work was killed,
just to confuse things further.
--