The SiCortex SC072-PDS seems to be going for less than $2000 on ebay these days.
It strikes me that it might be a fantastic mipsel compile farm machine [1], but I have zero experience in HPC, so have what is probably a dumb question:
Would there be an (obvious / easy) way to just run http://distcc.org on each of the 12 nodes / use all 72 processors for compiling on a SC072-PDS?
distcc does not require all machines to share a filesystem, have synchronized clocks, or to have the same libraries or header files installed.
distcc runs as a TCP/IP daemon, so each of the 12 nodes would need to be reachable with a separate IP address.
> The SiCortex SC072-PDS seems to be going for less than $2000 on ebay > these days.
> It strikes me that it might be a fantastic mipsel compile farm machine > [1], but I have zero experience in HPC, so have what is probably a > dumb question:
> Would there be an (obvious / easy) way to just run http://distcc.org > on each of the 12 nodes / use all 72 processors for compiling on a > SC072-PDS?
> distcc does not require all machines to share a filesystem, have > synchronized clocks, or to have the same libraries or header files > installed.
> distcc runs as a TCP/IP daemon, so each of the 12 nodes would need to > be reachable with a separate IP address.
Not a dumb question at all.
It's quite straightforward to set up distcc to run on all the nodes. Each node has its own IP address on the internal network. Normally those addresses are visible inside the box, but with a little bit of work, you can make those addresses routable from the outside as well.
You more or less just follow the distcc setup instructions; just make sure the files end up in the right places.
> On Dec 26, 2009, at 2:08 PM, Danny wrote: > > The SiCortex SC072-PDS seems to be going for less than $2000 on ebay > > these days.
> > It strikes me that it might be a fantastic mipsel compile farm machine > > [1], but I have zero experience in HPC, so have what is probably a > > dumb question:
> > Would there be an (obvious / easy) way to just runhttp://distcc.org > > on each of the 12 nodes / use all 72 processors for compiling on a > > SC072-PDS?
> > distcc does not require all machines to share a filesystem, have > > synchronized clocks, or to have the same libraries or header files > > installed.
> > distcc runs as a TCP/IP daemon, so each of the 12 nodes would need to > > be reachable with a separate IP address.
> Not a dumb question at all.
> It's quite straightforward to set up distcc to run on all the nodes. Each > node has its own IP address on the internal network. Normally those > addresses are visible inside the box, but with a little bit of work, you > can make those addresses routable from the outside as well.
Thanks a lot; I got an account on a SiCortex system and am testing this out now.
The first problem I've run into is that SiCortex does not support the o32 MIPS ABI, which Debian (and probably many other GNU/Linux distributions) use because in the past it has been "compatible with everything". However (from the SiCortex Programmer's Guide):
""" Programmers who have used other MIPS systems may be familiar with the o32 MIPS ABI. This is a 32-bit ABI and predates the availability of 64-bit registers in the MIPS architecture family. The System does not sup- port the o32 ABI. """
So at the moment I'm contacting the people who were involved with a Debian n32 port [1] (someone did performance tests and for non- scientific workloads n32 is more efficient than n64), and reading up on their published work [2].
Danny wrote: > On Dec 26 2009, 2:29 pm, Win Treese <tre...@serissa.com> wrote:
>> On Dec 26, 2009, at 2:08 PM, Danny wrote:
>>> The SiCortex SC072-PDS seems to be going for less than $2000 on ebay >>> these days.
>>> It strikes me that it might be a fantastic mipsel compile farm machine >>> [1], but I have zero experience in HPC, so have what is probably a >>> dumb question:
>>> Would there be an (obvious / easy) way to just runhttp://distcc.org >>> on each of the 12 nodes / use all 72 processors for compiling on a >>> SC072-PDS?
>>> distcc does not require all machines to share a filesystem, have >>> synchronized clocks, or to have the same libraries or header files >>> installed.
>>> distcc runs as a TCP/IP daemon, so each of the 12 nodes would need to >>> be reachable with a separate IP address.
>> Not a dumb question at all.
>> It's quite straightforward to set up distcc to run on all the nodes. Each >> node has its own IP address on the internal network. Normally those >> addresses are visible inside the box, but with a little bit of work, you >> can make those addresses routable from the outside as well.
> Thanks a lot; I got an account on a SiCortex system and am testing > this out now.
> The first problem I've run into is that SiCortex does not support the > o32 MIPS ABI, which Debian (and probably many other GNU/Linux > distributions) use because in the past it has been "compatible with > everything". However (from the SiCortex Programmer's Guide):
> """ > Programmers who have used other MIPS systems may be familiar with > the o32 MIPS ABI. This is a 32-bit ABI and predates the availability > of > 64-bit registers in the MIPS architecture family. The System does not > sup- > port the o32 ABI. > """
> So at the moment I'm contacting the people who were involved with a > Debian n32 port [1] (someone did performance tests and for non- > scientific workloads n32 is more efficient than n64), and reading up > on their published work [2].
Why not contact the people who are still maintaining the compiler?
This is not an authoritative answer, but it seems we have abi code for both mipsn32 and mips64. I have a machine I can test with, but this is something we should pursue offlist and then report the results back.
> Danny wrote: > > The first problem I've run into is that SiCortex does not support the > > o32 MIPS ABI, which Debian (and probably many other GNU/Linux > > distributions) use because in the past it has been "compatible with > > everything". However (from the SiCortex Programmer's Guide):
> > """ > > Programmers who have used other MIPS systems may be familiar with > > the o32 MIPS ABI. This is a 32-bit ABI and predates the availability > > of 64-bit registers in the MIPS architecture family. The System does > > not support the o32 ABI. > > """
> > So at the moment I'm contacting the people who were involved with a > > Debian n32 port [1] (someone did performance tests and for non- > > scientific workloads n32 is more efficient than n64), and reading up > > on their published work [2].
> Why not contact the people who are still maintaining the compiler?
> This is not an authoritative answer, but it seems we have abi code for > both mipsn32 and mips64. I have a machine I can test with, but this is > something we should pursue offlist and then report the results back.
My reading of the above was that o32 abi binaries was something that wasn't supported either at the silicon level, or at the linux level (eg the option to support the abi would have to be compiled in; for example, a.out support is sometimes not compiled in to x86 linux). So I don't see how compilers would be related to the problem.
It's possible that the compilers could produce o32 binaries that could not then run on sicortex systems but could on other mipsel systems; that may be useful, but isn't what I was talking about. As I'm interested in getting n32 into Debian anyway, whether or not this works isn't of long-term concern for me.
BTW it looks like the Debian n32 project is alive and there is an actual repo available [1], so I'll try doing a fakechroot debboostrap with that [2] and see how it works on the SiCortex.
