At the risk of seeming like I'm on a vendetta here: as I mentioned
before, I'm fine with having this in the space for the short term, and
playing around with it. However, I think it would be a tremendous
waste (both of space, and of potential funds) if we didn't eBay them
afterwards. I suspect it'll be tremendously under-utilised, and I
don't want to get into the business of billing people for power.
If anyone needs computing power like this, you can get a cc2.8xl
instance on EC2 for $2.40/hour. This is commodity stuff.
On 7 October 2012 22:31, Big Will <william.gilchri...@gmail.com> wrote:
> It would be useful for learning big data processing
Not really. The whole thrust of the "big data" thing is using large
numbers of normal-sized commodity machines to do things you'd
previously have needed the big iron for. The utility of a really large
machine is precisely that it allows one to use a large dataset without
the need for any fancy map-reduce footwork.
Thank you Jasper, this is great. I personally would like to see this hardware used in the space, even if we cannot justify leaving it powered on all the time there are plenty of projects that could utilise this hardware.
One problem with using the SCA ports to connect these servers together into a single BIOS image is that the resulting hardware platform is not SMP but NUMA. If the installed operating system just sees 64 CPU cores and 512 G of RAM then the likelihood is that performance will not scale as expected as the scheduler schedules processes to run on a CPU core that is far from the memory used. If the OS understands the NUMA nature of the platform then it should scale pretty well and it would certainly be really interesting to see.
Other options for using this hardware include running 'Zeus' as 4 separate partitions of a single x3950. With the SCA hardware physically connected it is possible to configure the BIOSes to any combination of partitions of the 4 physical hosts without physically connecting/disconnecting anything.
I think that for running VMs or using as a render farm then 4 single host nodes would work best. I would love to be able to let the BSD hackers loose on the hardware for them to test their scalability and I think it would be good for LHS to be associated with such efforts.
BTW - I have this weekend removed these 4 x3950 servers from the space to safe storage in my cellar in Clapton. There are still 2 * x3850 and 2 * x3950 in the space. I think that once the flack has calmed down from this hardware donation and a number of the other servers have been successfully auctioned on eBay then I can return them if there is enough interest and consensus among LHS members
On Sun, 7 Oct 2012, Simon wrote:
> Thank you Jasper, this is great.
> I personally would like to see this hardware used in the space, even if > we cannot justify leaving it powered on all the time there are plenty of > projects that could utilise this hardware.
> One problem with using the SCA ports to connect these servers together > into a single BIOS image is that the resulting hardware platform is not > SMP but NUMA. If the installed operating system just sees 64 CPU cores > and 512 G of RAM then the likelihood is that performance will not scale > as expected as the scheduler schedules processes to run on a CPU core > that is far from the memory used. If the OS understands the NUMA nature > of the platform then it should scale pretty well and it would certainly > be really interesting to see.
Oh yes, that reminds me, i know the guy that did this:
> Other options for using this hardware include running 'Zeus' as 4 > separate partitions of a single x3950. With the SCA hardware physically > connected it is possible to configure the BIOSes to any combination of > partitions of the 4 physical hosts without physically > connecting/disconnecting anything.
> I think that for running VMs or using as a render farm then 4 single > host nodes would work best.
> I would love to be able to let the BSD hackers loose on the hardware for > them to test their scalability and I think it would be good for LHS to > be associated with such efforts.
> BTW - I have this weekend removed these 4 x3950 servers from the space > to safe storage in my cellar in Clapton. There are still 2 * x3850 and > 2 * x3950 in the space.
> I think that once the flack has calmed down from this hardware donation > and a number of the other servers have been successfully auctioned on > eBay then I can return them if there is enough interest and consensus > among LHS members
> Simon
> On Sunday, 7 October 2012 22:31:06 UTC+1, Big Will wrote:
> It would be useful for learning big data processing
Renting compute time isn't as cool as cobbling together it ourselves to play with, even if just for the short term. I had a chance to build and play with an enormous Myrinet/Rocks powered cluster a few years ago and it was a brilliant learning experience. It isn't something you get a chance to do very often.
I totally understand the power/space issues and I think Simons idea of it happening after all the current server hubbub has died down is a good one.
On 7 Oct 2012, at 22:42, Russ Garrett <r...@garrett.co.uk> wrote:
> On 7 October 2012 21:09, Jasper Wallace <jas...@pointless.net> wrote:
>> I've started a wiki page with some ideas for what we would use Zeus for,
>> and how it would be used:
> At the risk of seeming like I'm on a vendetta here: as I mentioned
> before, I'm fine with having this in the space for the short term, and
> playing around with it. However, I think it would be a tremendous
> waste (both of space, and of potential funds) if we didn't eBay them
> afterwards. I suspect it'll be tremendously under-utilised, and I
> don't want to get into the business of billing people for power.
> If anyone needs computing power like this, you can get a cc2.8xl
> instance on EC2 for $2.40/hour. This is commodity stuff.
On 7 October 2012 22:57, Simon <skl...@gmail.com> wrote:
> Thank you Jasper, this is great.
> I personally would like to see this hardware used in the space, even if we
> cannot justify leaving it powered on all the time there are plenty of
> projects that could utilise this hardware.
Apologies for being blunt, but: I don't see any.
What's currently on the wiki page is a list of cool ideas, but with no
real plans behind them. I have yet to see any concrete reasons why we
need this hardware in the space.
This hardware is likely worth thousands of pounds on ebay. We
desperately need more cash to help with the upcoming space move. I
don't see how it's in the best interests of the space to keep hold of
this hardware for the long term, depreciating, just because some
people think it's cool.
In the end, Hackspace is about providing physical facilities which
people can't get elsewhere. Computing power is not a physical
facility, and so I don't think we have any place providing it on a
significant scale.
IIRC, when the Stratsys was going to be used, the charge for the
printing was going to include the cost of the plastic and the cost of
the power to run it.
Do the same. Put a time booking system in place, work out the power
requirements, then set it up, so when you book the machine for a run,
you pay for the power used.
The PSU's are a known capacity, so it's easy to calculate how much
juice they're using.
We already have a version of the power-monitoring software designed.
Sol had a display running off the smart meter output during the first
server auction.. I don't know how much work it'll be to adapt it to
this use.
Add in a link to IRC, where you can log how long it's been running/how
long the current run is to go.
Time booking via the wiki or a google-calender.
If the member's who want to use it for the first couple of runs set
this up and organise the maintenance schedule between them, then it'll
be another facility for the space, and another sub-group that can
teach other people how to do this.
It'll also be another selling point for new members that want access
to bigger iron. We didn't have access to this sort of hardware when i
was at uni.
On Oct 7, 11:06 pm, Tim Reynolds <t...@christwithfries.net> wrote:
> Renting compute time isn't as cool as cobbling together it ourselves to play with, even if just for the short term. I had a chance to build and play with an enormous Myrinet/Rocks powered cluster a few years ago and it was a brilliant learning experience. It isn't something you get a chance to do very often.
> I totally understand the power/space issues and I think Simons idea of it happening after all the current server hubbub has died down is a good one.
> On 7 Oct 2012, at 22:42, Russ Garrett <r...@garrett.co.uk> wrote:
> > On 7 October 2012 21:09, Jasper Wallace <jas...@pointless.net> wrote:
> >> I've started a wiki page with some ideas for what we would use Zeus for,
> >> and how it would be used:
> > At the risk of seeming like I'm on a vendetta here: as I mentioned
> > before, I'm fine with having this in the space for the short term, and
> > playing around with it. However, I think it would be a tremendous
> > waste (both of space, and of potential funds) if we didn't eBay them
> > afterwards. I suspect it'll be tremendously under-utilised, and I
> > don't want to get into the business of billing people for power.
> > If anyone needs computing power like this, you can get a cc2.8xl
> > instance on EC2 for $2.40/hour. This is commodity stuff.
Can someone clarify how big this is and where it would go?
There are a couple of projects that sound interesting but I agree with Russ
that this seems to be something that will be used once or twice and then
forgotten. Billy's idea is nice but I don't think it will work. We already
have a similar system with the laser cutter and last time I checked we were
getting a lot less that 75% of the time on it being paid for.
Beyond the 7 people on this list that have expressed any (not obviously
dis)interest is there and real call for this? The BSD stuff sounds like it
might be of interest but:
a) do we have any existing members who fall into this category
b) do they actually want to try this?
A render farm would be cool but is there anyone who wants to:
a) set the bloody thing up
b) use it
c) teach others to use it
We have a lot of computers, we a reasonable amount of computing power (for
what, I think, the majority of us want to do) and whilst as a one of "fire
it up and poke it" exercise we should definitely do it, keeping it around
seems to be a very large, expensive albatross.
*tl;dr* set it up once for a weekend of big iron fun then, unless people
can actually show that *they* (not some hypothetical 'others') have some
serious projects that they are committed to *and will carry out* that
really do need this sort of machine: sell it,
</2 uninformed cents>
S
On 8 October 2012 00:27, Bacon Zombie <baconzom...@gmail.com> wrote:
On Oct 8, 1:28 am, SamLR <sam.lindenrat...@gmail.com> wrote:
> Can someone clarify how big this is and where it would go?
> There are a couple of projects that sound interesting but I agree with Russ
> that this seems to be something that will be used once or twice and then
> forgotten. Billy's idea is nice but I don't think it will work. We already
> have a similar system with the laser cutter and last time I checked we were
> getting a lot less that 75% of the time on it being paid for.
Storage, might be an issue, but it's already been mooted that you
could install it high up on brackets in one of the toilets.
If it only gets used, once or twice, sure that's an issue, but if the
people that are interested in using it, are the people setting it up,
then what's the problem, it's their time.
If it's out of the way, and only accessed via the network, then it's
only one network socket, and we're not using all of the ones we've
got.
Book in advance, and pay in advance. It'll also mean that anyone using
it would have to be more organised about what they're doing, which is
no bad thing.
> Beyond the 7 people on this list that have expressed any (not obviously
> dis)interest is there and real call for this? The BSD stuff sounds like it
> might be of interest but:
> a) do we have any existing members who fall into this category
> b) do they actually want to try this?
> A render farm would be cool but is there anyone who wants to:
> a) set the bloody thing up
> b) use it
> c) teach others to use it
As i said, if it's their time at the space that they're using to set
this up, and it's stashed out of the way, and the power is factored
into the costs, then why worry?
> We have a lot of computers, we a reasonable amount of computing power (for
> what, I think, the majority of us want to do) and whilst as a one of "fire
> it up and poke it" exercise we should definitely do it, keeping it around
> seems to be a very large, expensive albatross.
If the people who want to use it, are actually interested in using it,
then they'll take the time to set it up and convince other members
that it's a worthwhile project, just like buying the laser-cutter or
the 3-in-1 lathe.
> *tl;dr* set it up once for a weekend of big iron fun then, unless people
> can actually show that *they* (not some hypothetical 'others') have some
> serious projects that they are committed to *and will carry out* that
> really do need this sort of machine: sell it,
If they don't get it set up then sure, flog it on ebay.
I'll happily help with installing it on the toilet ceiling. It's dead
space at the moment. We've got spare shelf-brackets and nailing a
server to a toilet ceiling won't be one of the strangest installation
jobs i've done.
If someone who knows the network can plumb it in, the rest is just
setting up the power monitoring software. and documenting it on the
wiki.
Then the people who are interested can get on with it.
And if no-one is interested in a few months, then we can flog it on
ebay.
I don't know if the limit on the number of sales per month is still
active on the hackspace ebay account, but if we sell it last, then
that should be enough time for anyone interested to get on with it.
I think Russ's point that the Space is planning to move (which costs money)
and it is worth a great deal (relatively speaking) should over-rule the
points about how it could be used. We need to look at the space a a
community, which has the best interests of the majority at heart. I have
used the laser cutter twice (but know other people use it a lot, but don't
pay for the time they are on it, which is disgraceful, and think it is a
brilliant piece of equipment).
Zeus would take up space (which is limited), especially while we are
working at getting it installed.
If people need to book a time to use it before hand I think it will be a
pink elephant.
We need to look at the big picture of the Space rather than seeing a pretty
new shiny!
Hugs,
Noko
On 8 October 2012 04:24, Morris <a.turntabl...@gmail.com> wrote:
> I think Russ's point that the Space is planning to move (which costs
> money) and it is worth a great deal (relatively speaking) should over-rule
> the points about how it could be used. We need to look at the space a a
> community, which has the best interests of the majority at heart. I have
> used the laser cutter twice (but know other people use it a lot, but don't
> pay for the time they are on it, which is disgraceful, and think it is a
> brilliant piece of equipment).
I think if people are genuninely going to use it and it's not going to be a
drain on resources (i.e. space and money, which by if it can be stored
nicely in the toilets and have its 'leccy paid for it won't be) then we
should keep it.
By your argument we should also sell the laser cutter & 3-in-1 as they're
worth a fair bit.
I'm more worried that it will get used once or twice and become a drain on
resources (by depreciating whilst taking up space or by being used and not
paid for).
In the interest of fairness I would be very interested to see how much
various other items cost in terms of electricity.
Surely my point was that this is not going to be used as much as the laser
cutter and 3-in-1? This is not yet up and running so surely it is a
different argument.
Ask the list how many people think they will genuinely use it.
Noko
On 8 October 2012 11:47, SamLR <sam.lindenrat...@gmail.com> wrote:
> I think Russ's point that the Space is planning to move (which costs
>> money) and it is worth a great deal (relatively speaking) should over-rule
>> the points about how it could be used. We need to look at the space a a
>> community, which has the best interests of the majority at heart. I have
>> used the laser cutter twice (but know other people use it a lot, but don't
>> pay for the time they are on it, which is disgraceful, and think it is a
>> brilliant piece of equipment).
> I think if people are genuninely going to use it and it's not going to be
> a drain on resources (i.e. space and money, which by if it can be stored
> nicely in the toilets and have its 'leccy paid for it won't be) then we
> should keep it.
> By your argument we should also sell the laser cutter & 3-in-1 as they're
> worth a fair bit.
> I'm more worried that it will get used once or twice and become a drain on
> resources (by depreciating whilst taking up space or by being used and not
> paid for).
> In the interest of fairness I would be very interested to see how much
> various other items cost in terms of electricity.
> wrote:
> Surely my point was that this is not going to be used as much as the laser
> cutter and 3-in-1? This is not yet up and running so surely it is a
> different argument.
> Ask the list how many people think they will genuinely use it.
> Noko
> On 8 October 2012 11:47, SamLR <sam.lindenrat...@gmail.com> wrote:
>> I think Russ's point that the Space is planning to move (which costs
>>> money) and it is worth a great deal (relatively speaking) should over-rule
>>> the points about how it could be used. We need to look at the space a a
>>> community, which has the best interests of the majority at heart. I have
>>> used the laser cutter twice (but know other people use it a lot, but don't
>>> pay for the time they are on it, which is disgraceful, and think it is a
>>> brilliant piece of equipment).
>> I think if people are genuninely going to use it and it's not going to be
>> a drain on resources (i.e. space and money, which by if it can be stored
>> nicely in the toilets and have its 'leccy paid for it won't be) then we
>> should keep it.
>> By your argument we should also sell the laser cutter & 3-in-1 as they're
>> worth a fair bit.
>> I'm more worried that it will get used once or twice and become a drain
>> on resources (by depreciating whilst taking up space or by being used and
>> not paid for).
>> In the interest of fairness I would be very interested to see how much
>> various other items cost in terms of electricity.
As somebody previously mentioned, the cost of a 88 core/60gb EC2 instance
is $2.40 (~£1.60) per hour. So it;s not like the cost or availability of
such a machine is prohibitive (unlike say access to a Laser Cutter or
3-in-one).
Not having 'Zeus' would also means not having the maintenance/management
effort required (I'd imagine with such a system there's going to be a fair
amount of tweaking required on a project by project basis depending on
which resources are being pushed)
Finally, I was under the impression that the idea of 'big data' projects,
from a technology standpoint, is supposed to be that you can distribute
workload (and reduce bottlenecks) for improved performance, as long as you
can develop the project on a less able machine, you should be able to
execute it in an EC2 environment.
On Mon, Oct 8, 2012 at 1:07 PM, David Murphy <murphy.da...@gmail.com> wrote:
> few people do big data projects which can't be handled on a modern gaming
> rig.
> I certainly see the point. but it's worth a try for a couple of months.
> perhaps someone will do something really interesting.
> On Mon, Oct 8, 2012 at 12:16 PM, Clare Greenhalgh <
> claregreenha...@gmail.com> wrote:
>> Surely my point was that this is not going to be used as much as the
>> laser cutter and 3-in-1? This is not yet up and running so surely it is a
>> different argument.
>> Ask the list how many people think they will genuinely use it.
>> Noko
>> On 8 October 2012 11:47, SamLR <sam.lindenrat...@gmail.com> wrote:
>>> I think Russ's point that the Space is planning to move (which costs
>>>> money) and it is worth a great deal (relatively speaking) should over-rule
>>>> the points about how it could be used. We need to look at the space a a
>>>> community, which has the best interests of the majority at heart. I have
>>>> used the laser cutter twice (but know other people use it a lot, but don't
>>>> pay for the time they are on it, which is disgraceful, and think it is a
>>>> brilliant piece of equipment).
>>> I think if people are genuninely going to use it and it's not going to
>>> be a drain on resources (i.e. space and money, which by if it can be stored
>>> nicely in the toilets and have its 'leccy paid for it won't be) then we
>>> should keep it.
>>> By your argument we should also sell the laser cutter & 3-in-1 as
>>> they're worth a fair bit.
>>> I'm more worried that it will get used once or twice and become a drain
>>> on resources (by depreciating whilst taking up space or by being used and
>>> not paid for).
>>> In the interest of fairness I would be very interested to see how much
>>> various other items cost in terms of electricity.
I originally spoke with Simon and Jasper about this and their thoughts
were that it should stay and we should at least play with it. There
are some cool ideas and I think that is also what the space is about.
I agreed with them that we should at least plug it in, get some hard
figures and then decide.
That said, why re-invent the wheel? Billing for time, looking at power
requirements... EC2 has that covered. I initially wanted a render farm
setup but I quickly realised thats all covered with EC2 3DSMax and
Blender instances anyway.
I think that a lot of our problems will be solved by a move. Its a
subtle and elegant solution and if this raises some capital, then so
much the better. I now agree that yes, we'd be wasting our time.
That said, its muggins here who has to shift the bugger as Im likely
to be driving it to storage ;)
B
On 8 October 2012 13:07, David Murphy <murphy.da...@gmail.com> wrote:
> few people do big data projects which can't be handled on a modern gaming
> rig.
> I certainly see the point. but it's worth a try for a couple of months.
> perhaps someone will do something really interesting.
> On Mon, Oct 8, 2012 at 12:16 PM, Clare Greenhalgh
> <claregreenha...@gmail.com> wrote:
>> Surely my point was that this is not going to be used as much as the laser
>> cutter and 3-in-1? This is not yet up and running so surely it is a
>> different argument.
>> Ask the list how many people think they will genuinely use it.
>> Noko
>> On 8 October 2012 11:47, SamLR <sam.lindenrat...@gmail.com> wrote:
>>>> I think Russ's point that the Space is planning to move (which costs
>>>> money) and it is worth a great deal (relatively speaking) should over-rule
>>>> the points about how it could be used. We need to look at the space a a
>>>> community, which has the best interests of the majority at heart. I have
>>>> used the laser cutter twice (but know other people use it a lot, but don't
>>>> pay for the time they are on it, which is disgraceful, and think it is a
>>>> brilliant piece of equipment).
>>> I think if people are genuninely going to use it and it's not going to be
>>> a drain on resources (i.e. space and money, which by if it can be stored
>>> nicely in the toilets and have its 'leccy paid for it won't be) then we
>>> should keep it.
>>> By your argument we should also sell the laser cutter & 3-in-1 as they're
>>> worth a fair bit.
>>> I'm more worried that it will get used once or twice and become a drain
>>> on resources (by depreciating whilst taking up space or by being used and
>>> not paid for).
>>> In the interest of fairness I would be very interested to see how much
>>> various other items cost in terms of electricity.
> I originally spoke with Simon and Jasper about this and their thoughts
> were that it should stay and we should at least play with it. There
> are some cool ideas and I think that is also what the space is about.
> I agreed with them that we should at least plug it in, get some hard
> figures and then decide.
> That said, why re-invent the wheel? Billing for time, looking at power
> requirements... EC2 has that covered. I initially wanted a render farm
> setup but I quickly realised thats all covered with EC2 3DSMax and
> Blender instances anyway.
> I think that a lot of our problems will be solved by a move. Its a
> subtle and elegant solution and if this raises some capital, then so
> much the better. I now agree that yes, we'd be wasting our time.
> That said, its muggins here who has to shift the bugger as Im likely
> to be driving it to storage ;)
> B
> On 8 October 2012 13:07, David Murphy <murphy.da...@gmail.com> wrote:
> > few people do big data projects which can't be handled on a modern gaming
> > rig.
> > I certainly see the point. but it's worth a try for a couple of months.
> > perhaps someone will do something really interesting.
> > On Mon, Oct 8, 2012 at 12:16 PM, Clare Greenhalgh
> > <claregreenha...@gmail.com> wrote:
> >> Surely my point was that this is not going to be used as much as the
> laser
> >> cutter and 3-in-1? This is not yet up and running so surely it is a
> >> different argument.
> >> Ask the list how many people think they will genuinely use it.
> >> Noko
> >> On 8 October 2012 11:47, SamLR <sam.lindenrat...@gmail.com> wrote:
> >>>> I think Russ's point that the Space is planning to move (which costs
> >>>> money) and it is worth a great deal (relatively speaking) should
> over-rule
> >>>> the points about how it could be used. We need to look at the space a
> a
> >>>> community, which has the best interests of the majority at heart. I
> have
> >>>> used the laser cutter twice (but know other people use it a lot, but
> don't
> >>>> pay for the time they are on it, which is disgraceful, and think it
> is a
> >>>> brilliant piece of equipment).
> >>> I think if people are genuninely going to use it and it's not going to
> be
> >>> a drain on resources (i.e. space and money, which by if it can be
> stored
> >>> nicely in the toilets and have its 'leccy paid for it won't be) then we
> >>> should keep it.
> >>> By your argument we should also sell the laser cutter & 3-in-1 as
> they're
> >>> worth a fair bit.
> >>> I'm more worried that it will get used once or twice and become a drain
> >>> on resources (by depreciating whilst taking up space or by being used
> and
> >>> not paid for).
> >>> In the interest of fairness I would be very interested to see how much
> >>> various other items cost in terms of electricity.
Very interested in the Internet simulation aspect and modelling scenarios of lots of networked computers. I am preparing some material with some international friends from #cryptoparty to make a case for this.
> On Oct 8, 1:28 am, SamLR <sam.lindenrat...@gmail.com> wrote:
> > Can someone clarify how big this is and where it would go?
> > There are a couple of projects that sound interesting but I agree with Russ
> > that this seems to be something that will be used once or twice and then
> > forgotten. Billy's idea is nice but I don't think it will work. We already
> > have a similar system with the laser cutter and last time I checked we were
> > getting a lot less that 75% of the time on it being paid for.
> Storage, might be an issue, but it's already been mooted that you
> could install it high up on brackets in one of the toilets.
No, the machine that may go in one of the toilets is 'Borg', the babbage replacement and vm server.
Zeus will be 4 4U machines, so a stack about 3 foot high. I think it would be too heavy to wall mount safely, and without a rack to put them in the only option is stacking them on the floor somewhere.
>> On Oct 8, 1:28 am, SamLR <sam.lindenrat...@gmail.com> wrote:
>> > Can someone clarify how big this is and where it would go?
>> > There are a couple of projects that sound interesting but I agree with Russ
>> > that this seems to be something that will be used once or twice and then
>> > forgotten. Billy's idea is nice but I don't think it will work. We already
>> > have a similar system with the laser cutter and last time I checked we were
>> > getting a lot less that 75% of the time on it being paid for.
>> Storage, might be an issue, but it's already been mooted that you
>> could install it high up on brackets in one of the toilets.
> No, the machine that may go in one of the toilets is 'Borg', the babbage
> replacement and vm server.
> Zeus will be 4 4U machines, so a stack about 3 foot high. I think it would
> be too heavy to wall mount safely, and without a rack to put them in the
> only option is stacking them on the floor somewhere.
If it will only be used for short periods of time and not permanently
on then it could be that it has a home when it's off and set up
wherever there is space available when it is needed. Of course as long
as the 'off location' is not inconvenient and it is not left in the
'on location' for too long.
