Install Team

[Ken Starks]

Tom King will lead this group. We will be doing some things
differently this year. We will employ some kvm switch operation in
combination to free-standing machine installs. Each station will be
equipped with a combination of USB key and CD media. This will reduce
the power problems we experienced in 2009. We will be using one
standard distro this year instead of several. We can discuss the
logistics and previous problems encountered in this tread.

[HidekiAI]

Hello Tom,

My name is Hideki "Tony" Ikeda. If there are anything I can do for
preparations, please let me know...

Can you or anybody who was doing install previous year talk about
software issues such as what went right and wrong last year? As well
as what to expect for software installations and time spent on
diagnostics (i.e simple as adding "noapic" as a kernel option on boot,
etc) - I'm sure you'd not want us to futz around for long period of
time to make it work (i.e. give up after 15 minutes and move on to
next box)

Also, please specify patitionings when installing (i.e. how much
dedicate to swap (i.e. rules such as 2xRAM), /boot, and /, a dedicated
partition for /var? /home? use ext2 or ext3 - or even ext4?). Do we
pre-setup the network as fixed I.P. or DHCP? Pre-configured
iptables? Empty resolv.conf or with some default nameserver?

thanks!

[dond...@reglue.org]

simple as adding "noapic" as a kernel option on boot

Most of them were - e.g. default vga.

A quick reference sheet on the install line parameters would help.

You'll probably use default disk partitioning - except when 2 40G hard
drives are being used because others aren't available. (Would we want to
use LVM in that case?)

> Do we
> pre-setup the network as fixed I.P. or DHCP?

There going out to random kids - so DHCP.


Pre-configured
> iptables?

Nice thought -- block a lot of those pesky Microsoft ports that are so
worm ripe - but you don't know where they're going and you won't be able
to change after the fact.

Empty resolv.conf or with some default nameserver?

Interesting thought. Possibly encourage OpenDNS (or some non-telecom DNS)?

[helios]

We've done about 300 installs this year so far and the Mint 8 (based on Ubuntu 9.10) runs extremely well on ext4.  On 40 gig drives we do a 1 gig swap, anything above that our default is 2 gigs of swap.  It saves a bunch of time in the long run to have a bit of extra swap, especially when we are dealing with machines that will only have 1 gig of memory.

We also use a firefox extension that blocks nasty websites and such but I set that up when I deliver the computers to the individual kids.

--
A child's exposure to technology should never be predicated on an ability to pay for it.

[Alan Jones]

Hi Ken and all,

On Mon, May 3, 2010 at 7:03 PM, helios <hel...@fixedbylinux.com> wrote:
> On 40 gig drives we do a 1 gig
> swap, anything above that our default is 2 gigs of swap.

It's probably worthwhile to make sure there's always at least as much as ram.
Maybe a smidge more. Suspend to disk in linux dumps ram to swap. So
having at least as much as ram means strange things won't happen if they
hit suspend to disk.

Though I imagine in most cases you'd hit that with those current numbers.

Cheers,

Alan.

[FelixTheCat]

Last year was a HUGE learning experience. We won't even attempt PXE
installs but will be using USB keys and boot CDs for those computers
that don't boot to USB from BIOS. The current thought is to have two
stations hooked up to a single KVM switch, and eight nodes hooked up
to install from the KVM switch. Each station will be responsible for
four nodes.

We'll also have an extra KVM switch in case we figure we can afford
some more space, people, and such to install more nodes at once.

For those picky computers that won't install using defaults, we plan
on having one additional KVM with only two nodes connected. This is
reserved for trouble installs only and we'll make sure anyone manning
that station is very well versed in boot options. Anything that simply
refuses to boot/install will be set aside and hacked on in the
future.

Beyond that, we won't be having any kind of networking and won't be
running updates unless we decide later it'd be preferable. At this
point, the only reasons to have a network would be installing
proprietary video drivers and updating security issues. I'll leave
that up to everyone else if we want to pursue that.

So, having put all that out at once, let me know if you still wish to
work with installs. If so, what level, whether it be regular installs
or trouble installs.

Thanks!

[dond...@reglue.org]

I would like to work on trouble installs (or regular installs).

> Last year was a HUGE learning experience. We won't even attempt PXE
> installs but will be using USB keys and boot CDs for those computers
> that don't boot to USB from BIOS. The current thought is to have two
> stations hooked up to a single KVM switch, and eight nodes hooked up
> to install from the KVM switch. Each station will be responsible for
> four nodes.

Only 8 installs going simultaneously?

Isn't Andi bringing 11 flat screens? How many tables will there be for
installs? The KVMs may add cable awkwardness and delay the notice of a
crashed/failed/ waiting install message.

Some KVMs may certainly be very helpful but it might not be pertinent to
plan for installs using exclusively KVM.

As far as space consumption 4 computers will fit with monitors and
keyboards on each table.

As many people as there will be available KVM switches might be a bit
superfluous. It seems to be an unnecessary limiting factor.

Will there be a computer available / space / network computer availability
for quick google searches in troubleshooting?

I am not advocating the use of PXE boot but I do feel that the PXE boot
idea has been tossed aside bit quickly. I believe the problem last year
was that it had not been taken into consideration that not all NICs PXE
boot - a big stack of rominator CDs prepared in advance could have helped
with that.

I believe clear expectations, a clear flow, and reducing unnecessary
complexity can help greatly. The switch to USB boot and CDs that support
USB boot will provide some stream lining that will help smooth things out.

[Paul Griffiths]

I agree.  We lost a huge amount of time last year because of the KVMs, with machines sitting idle waiting for single-threaded access to the console.

PXE boot also is nice, and useful in many cases, but I think for our purposes here, USB sticks (and enough of them) should more than suffice.  With enough USB sticks we can literally set up as many install stations as we have table space, monitors (11+?) and power to handle.  I think I heard Ken say that keyboards and mice were in abundant supply.

I'm not familiar with rominator, but it sounds similar to plpbt -- if a USB equipped machine can't boot from USB, simply boot from a CD and then point to the USB to continue.  Whatever works.  I can certainly bring a few pre-built plpbt CDs with me for those boxes that might need a jump-start.

We also need to have a plan to accommodate machines with no USB at all, and we'll need a few full install CDs made from the same .iso that the USB sticks are built from.

Finally, anything that will not boot from a CD could be handled similarly by booting from a floppy and then pointing to a CD to continue -- but do we really want to go there?

griff

_______
Paul (griff) Griffiths
512/241-1644

[Alan Jones]

Hi All,

Would it be worthwhile looking at having a tower with a bunch of hot swap bays
(ideally using ide/sata directly - not via usb) that could be used to
partition and
dump an image onto the hard disks directly?

A generic config would probably be robust enough to handle boot and basic
graphics support to allow getting the rest done and the transfer
should be a bunch
faster than copying from cd/usb. It means you could have a potentially bootable
machine ready before you've even plugged it into a display and keyboard allowing
for less display use time per machine and, hopefully, higher throughput.

Just a thought. It may also be considered too much overhead for an uncertain
benefit.

Cheers,

Alan.

[Thomas King]

Well, it appears that, if Andi was already planning on bringing that many flat screens for the installations, plans were already made on how the workflow and setup is going to be.

I'm going to bow out, then, and let y'all take it. I was asked about a week and a half ago to lead the installations but I'm woefully behind and don't think the place has the power to handle so many monitors hooked up to each computer, plus space is really going to be a challenge even having one person man one or two stations at once. You'll have faster installs with the USB keys but there will still be a bit of downtime for the person manning each station, hence using one station for four installs. Last year we had one station for six installs and that was a horrible bottleneck, agreed.

I appreciate being asked, though, but hope it all goes well. :) I've got a ton of irons in the fire and not enough time to switch gears with planning the installations out differently.

Godspeed.
Tom King

[Andy Krell]

Don't forget the Intake and Repair teams will need monitors as well.
Last year we managed 4 stations just for repair, I see that growing
this year. We'll need to make VERY effective use of the power and
space available to us to accommodate stations for all the teams. Some
stations may transition between teams depending on where we bottleneck
during the day. I need to do a count of available LCD screens I have
on hand, a dozen is my goal.

I get the allure of a PXE server but see the following drawbacks:
- Additional power draw for server, switches
- Additional wiring mess, less physical flexibility (gotta be close to
switch)
- Far from universal boot support, rominator CDs are added complexity
- Frustrating experience last year

CDs and USB seem fundamentally simpler to me, and I find that
appealing.

KVM upsides:
- Less power draw from reduced number of monitors
- Less install labor needed for smooth installs

KVM downsides:
- Wiring mess, less physical flexibility
- Bottlenecks installs to one operator

If we're going to have a dozen skilled installed crew I think we'd go
faster without the KVMs. Given the provenance of many of the machines
I'd hate to assume installs will tend towards the straightforward.

[sgue...@austin.rr.com]

IMHO we should not see any machines w/o both a CD drive and a USB port. Whether it'll boot from the USB.. well that's another story.
---- Paul Griffiths <gr...@joseph-bolton.com> wrote:

[sgue...@austin.rr.com]

I don't think these are all for installs.

[sk...@skipsjunk.net]

I have a couple of 'bench' LCDs if we need them.  I think they are 19".

[Don Davis]

USB sticks (and enough of them) should more than suffice. With enough USB
> sticks we can literally set up as many install stations as we have table space,
> monitors (11+?) and power to handle. I think I heard Ken say that keyboards and
> mice were in abundant supply.

It is important to stay aware of power concerns and how things might be
done without tripping the power.

I can certainly bring a few pre-built
> plpbt CDs with me for those boxes that might need a jump-start.

Could you bring about 20?

> We also need to have a plan to accommodate machines with no USB at all, and
> we'll need a few full install CDs made from the same .iso that the USB sticks
> are built from.

The install will be a bit larger than a CD - more than 2 gigs. What are
the chances that a machine within specs won't have USB?

> Finally, anything that will not boot from a CD could be handled similarly by
> booting from a floppy and then pointing to a CD to continue -- but do we really
> want to go there?

Doubtful - all the recommended specs place an acceptable machine a bit
outside of the boot from floppy to CD range.







> griff
> _______
> *Paul (griff) Griffiths*

[sgue...@austin.rr.com]

20! What the heck for?

[Don Davis]

sgue...@austin.rr.com wrote:
> 20! What the heck for?

Ease of access. They're easy to burn. Keep 4 or more at a station and
nobody has to fish. No need to pass them around, wait, or look for them.
Having 20 can't hurt and it is certainly more than enough. The effort of
burning 20 isn't much more than burning 10 and with only 10 things could
go wrong/slow.

(In chemistry, when planning a chemical reaction you use as little as
possible of the expensive chemical and dump on the cheap chemical to
maximize usage of the expensive chemical. I'm trying to remember the
terms - limiting reagent/ excess reagent?
I also look at it in terms of game theory - what advantage do we have by
burning only a few compared to the cost/risk of burning extra.)

[helios]

Once the ISO is created, I've agreed to burn the media.  I don't mind doing it, I believe the iso is being worked on now.
s

[RW]

I've got two lcd's here at work that we are getting rid of. I'll bring them if it helps. 

[helios]

Getting rid of as in donating?  We could use them after the event if they are up for donation.

On Wed, May 5, 2010 at 11:07 AM, RW <thedisenfranc...@gmail.com> wrote:

I've got two lcd's here at work that we are getting rid of. I'll bring them if it helps. 

--

[RW]

Yes.

On May 5, 2010 12:24 PM, "helios" <hel...@fixedbylinux.com> wrote:

Getting rid of as in donating?  We could use them after the event if they are up for donation.

On Wed, May 5, 2010 at 11:07 AM, RW <thedisenfranc...@gmail.com> wrote:
>

> I've got two...

--

[helios]

That will work buddy...since we cannot use CRT's anymore for individual kids installs, those are extremely valuable to us.  Thanks muches and please pass along my thanks to your boss.  If he needs a receipt for taxes let me know.

[RW]

No prob. Ill also be bringing a toshiba satellite (turion64,2gb) and another p4 dell. As we decomission more hardware I'll make sure it all gets donated.

On May 5, 2010 12:29 PM, "helios" <hel...@fixedbylinux.com> wrote:

That will work buddy...since we cannot use CRT's anymore for individual kids installs, those are extremely valuable to us.  Thanks muches and please pass along my thanks to your boss.  If he needs a receipt for taxes let me know.

On Wed, May 5, 2010 at 11:26 AM, RW <thedisenfranc...@gmail.com> wrote:
>
> Yes.
>>

>> O...

[Andy Krell]

The LCDs are meant to spread amongst all the teams, we'll need 4 or 5
for repair. Rather have too many than too few, and we can certainly
set them aside if the power draw or space becomes an issue. Based on
figures Ken provided LCD draw is about 1/2 that of a CRT, so I think
we'll be able to have more workstations than last year. Plus we'll be
using ALL of the circuits in the place this time. :)

I can bring 11 LCDs without wreaking undue havoc in the workplace.
(Spares and unused workstations)

When the team leads get together next time we can figure out a total
required monitor count.

On May 4, 11:22 pm, <sguen...@austin.rr.com> wrote:
> I don't think these are all for installs.

> ---- Thomas King <thomas.k...@gmail.com> wrote:
>
> > Well, it appears that, if Andi was already planning on bringing that many
> > flat screens for the installations, plans were already made on how the
> > workflow and setup is going to be.
>
> > I'm going to bow out, then, and let y'all take it. I was asked about a week
> > and a half ago to lead the installations but I'm woefully behind and don't
> > think the place has the power to handle so many monitors hooked up to each
> > computer, plus space is really going to be a challenge even having one
> > person man one or two stations at once. You'll have faster installs with the
> > USB keys but there will still be a bit of downtime for the person manning
> > each station, hence using one station for four installs. Last year we had
> > one station for six installs and that was a horrible bottleneck, agreed.
>
> > I appreciate being asked, though, but hope it all goes well. :) I've got a
> > ton of irons in the fire and not enough time to switch gears with planning
> > the installations out differently.
>
> > Godspeed.
> > Tom King
>

[Paul Griffiths]

On 5/5/2010 6:26 AM, Don Davis wrote:

USB sticks (and enough of them) should more than suffice.  With enough USB

sticks we can literally set up as many install stations as we have table space, monitors (11+?) and power to handle.  I think I heard Ken say that keyboards and mice were in abundant supply.

It is important to stay aware of power concerns and how things might be done without tripping the power.

Your point is well taken.  Very much agree.  This year I understand we will be operating off multiple circuits, rather than just one or two.  We will also be using LCDs rather than the CRTs we used last year.  And I think someone at the Triumph planning session offered to bring a couple of those Kill-A-Watt power monitoring devices so that we can monitor the current drain on the circuit and tell what we're pulling.  I also have one that I could bring if there's consensus that they would be useful.

One other thing to consider.  Someone put out a call for multiple "100 foot" extension cords, "better too long than not".  This is not necessarily the case.  Distance from an outlet is only one factor.  We need to also consider the current drop across the length of the wire itself.  These cables will likely (hopefully) be 12 awg or at least 14 awg and the wire itself will eat some amount of current.  Finally, and I hate to be a worry wart, but for fire safety reasons we must be very careful about daisy-chaining extension cords or power strips.  It's the (old) wiring back inside the walls that gets hot, not the wiring out on the floor that you can see.  Hopefully, as a last resort, the breakers really do trip before we cause that kind of problem.

 I can certainly bring a few pre-built

plpbt CDs with me for those boxes that might need a jump-start.

Could you bring about 20?

I can't imagine that we would need 20, but I can certainly make up however many we could reasonably use.

We also need to have a plan to accommodate machines with no USB at all, and we'll need a few full install CDs made from the same .iso that the USB sticks are built from.

The install will be a bit larger than a CD - more than 2 gigs. What are the chances that a machine within specs won't have USB?

OK, then we can strike this from the list.

Finally, anything that will not boot from a CD could be handled similarly by booting from a floppy and then pointing to a CD to continue -- but do we really want to go there?

Doubtful - all the recommended specs place an acceptable machine a bit outside of the boot from floppy to CD range.

And so we can strike this one too.  I like taking things off the list.

Can someone point me to a document that tells me the minimum machine specs that we can expect to see?  I think I must have missed seeing it somehow.

[helios]

The way it is supposed to work is that the install team will not have to worry about problem machines.  Intake is going to qualify the machines that ultimately sit in front of you.  That means if it won't even boot from a CD, then it goes to repair to have the problem diagnosed/fixed and taken to install, parts replaced or salvaged.  We spent way too much time last year on fancy install schemes on failing machines.  If it doesn't install to cd or usb, it gets taken over to Alex and his crew.  Ultimately if one of our kids gets a machine that won't boot to cd but has an install on it, it's not going to be much good to him or her without working media.  If you have a machine in front of you, it has already either been qualified or has been repaired and tested.  If you get a machine that will not boot to CD, someone didn't do their job.

The specs for machines acceptable for use are 2 gig + P4 processors or similar AMD chips.  The peripherals should be added to or repaired long before it gets to you.  That's the plan anyway.

[helios]

Don Davis has agreed to head the Install Team. If you have any
questions, suggestions or comments about the Install process, please
contact Don...He's your guy.