I've got a surprise for you!

23 views
Skip to first unread message

Trent Nelson

unread,
Jan 26, 2009, 6:32:46 PM1/26/09
to python-c...@python.org, bob.go...@hp.com, ti...@idyll.org, glynn....@sun.com, jim.w...@sun.com, ted....@sun.com, Frank.Wi...@sun.com, hjan...@microsoft.com, garr...@microsoft.com, pu...@cse.msu.edu, sra...@microsoft.com, cno...@microsoft.com, di...@exchange.microsoft.com, Jimmy.S...@microsoft.com, dfu...@microsoft.com, stoc...@cse.msu.edu, pitc...@cse.msu.edu, tne...@onresolve.com, ma...@canonical.com, exa...@divmod.com, gy...@divmod.com, lhaw...@google.com, do...@ubuntu.com, fuzz...@voidspace.org.uk, oubi...@divmod.com, jacob.ka...@gmail.com, p...@python.org, cli...@cse.msu.edu
Catchy subject eh? Unless, of course, you've seen Forgetting Sarah
Marshall, in which case, it's probably a tad disturbing.

Nevertheless, I do have a surprise for everyone. I spent a lot of
time early last year trying to keep the buildbots green, especially
the x64 Windows ones. This was quite enjoyable, at least initially.

By the time I got back to London after PyCon last year, the buildbot
honeymoon was wearing off. As there's usually only one person that
has access to a given buildbot, and that person is rarely you, it
can be a right pain in the ass trying to debug problems you can't
reproduce on platforms you don't have access to (especially if all
you've got is an error message that doesn't make sense).

I was convinced there must be a better way.

Around mid-April, the buildbot on my FreeBSD 6.x box I had recently
set up kept failing on on setitimer tests. I received an email from
a chap named Guilherme Polo who had seen the buildbot test failures
and wanted to assist. I was swamped with client work at the time; I
wasn't able to run any of the test scripts he sent me, but, hey, he
seemed like a nice chap, so I thought what the hell, and just gave
him an account on the box and checked out a copy of trunk in his
home directory.

Now, mind you, this was before Guilherme was a committer; I didn't
know the guy from a bar of soap. For all I knew, he could have been
using the shell account to launch a massive DDoS or phishing scam.

However, no more than five minutes after I created his account, he'd
diagnosed the problem, replicated the behaviour in a C program, and,
with a bit of googling, figured out what was wrong and proposed a
fix.

And that's when it hit me. Buildbots are fine when everything is
running smoothly, but nothing compares to actually having access to
a system when you're trying to debug something.

So, I thought to myself, why not buy a couple of clunky old boxes
off eBay and donate them to the PSF, such that all developers had
access to them? I dropped a note to Guido and Neal, they put me
in touch with Titus, who had just accepted a position at Michigan
State, and, well...

Ten months, seven trips to MSU, six blown fuses and about $60,000
later, I'm proud to introduce you all to Snakebite: The Open Network!
A network of around 37-ish servers of all different shapes and sizes,
spread over three sites, specifically geared towards the needs of
open source projects like Python.

Every CPython, Jython, IronPython and PyPy committer will have access
to every development server on the network. I've also extended the
offer to prominent Python projects like Django and Twisted.

Eventually, I'll invite other open source projects to participate
(Apache, Subversion, MySQL, Postgres, etc), but the network is my
gift to All Things Python, first and foremost, so Python projects
will always get preferential treatment.

Support for the initiative so far has been nothing short of sublime.

Microsoft jumped on board and provided unlimited MSDN licenses in
less time than it took me to write them an e-mail asking for stuff.

I sent HP an e-mail asking if they could spare a Tru64 license, and
maybe 2GB of RAM for an extremely crappy Itanium box I bought off
eBay. They saw my Tru64 license request and raised with media and
licenses to the latest version of HP-UX.

Unfortunately, it was too much trouble for them to try and source
2GB of RAM for the Itanium I bought. So, instead, they shipped two
massive quad Itanium 2 RX-5670s, chock full of 73GB 15k disks and
no less than 78GB of RAM between the two servers; 32GB in one and
46GB in the other. Well then.

(I'd hate to think what would have turned up had I asked them for
two quad Itanium monsters.)

Sun, Google and Canonical have also expressed a lot of interest in
the project -- I stopped asking for hardware a while back though as
we've literally run out of space to host it all.

The website is live, but the content is a bit sparse at the moment,
excluding the poorly worded front page and the reasonably accurate
network page:
http://www.snakebite.org
http://www.snakebite.org/network

It'll probably be a few weeks before you can start logging in and
doing stuff. The HPCC/CSE server room at MSU is about to have walls
knocked in and ramps built in order to accommodate a giant PDU that
has been sitting outside it for about six months; the Snakebite rack
is going to get shuffled around a bit so I figure there's not much
point going live before that's taken care of.

Other than that, I'm just happy to get this off my chest, ten months
is a freakin' long time to try and keep something like this a secret
;-)

Thanks to everyone who has contributed so far. Titus, Bill, Adam,
Kelly, George: Snakebite wouldn't be anything more than a twinkle
in my eye if it weren't for the support MSU has thrown behind the
project, thank you for all your efforts to date. Hank, Garrett and
Sam: having the support of Microsoft from very early on has been a
huge boost and the MSDN licenses have already been invaluable. Bob,
well, what can I say, there was a period there where every e-mail
thread between us seemed to result in something being shipped from
HP to MSU; thanks to you and HP's open source labs.

And last but not least, thanks to Guido and all the Python committers
for their tireless efforts to date. Although the sheer elegance of
the language is what initially attracted me to Python, it was the
developer community that made me want to stick around.

Snakebite is my gift to you!


Trent.

Trent Nelson

unread,
Jan 27, 2009, 3:01:38 PM1/27/09
to snakebi...@googlegroups.com, python-c...@python.org, bob.go...@hp.com, ti...@idyll.org, glynn....@sun.com, jim.w...@sun.com, ted....@sun.com, Frank.Wi...@sun.com, hjan...@microsoft.com, garr...@microsoft.com, pu...@cse.msu.edu, sra...@microsoft.com, cno...@microsoft.com, di...@exchange.microsoft.com, Jimmy.S...@microsoft.com, dfu...@microsoft.com, stoc...@cse.msu.edu, pitc...@cse.msu.edu, tne...@onresolve.com, ma...@canonical.com, exa...@divmod.com, gy...@divmod.com, lhaw...@google.com, do...@ubuntu.com, fuzz...@voidspace.org.uk, oubi...@divmod.com, jacob.ka...@gmail.com, p...@python.org, cli...@cse.msu.edu
I've just set up a mailing list for those that want to carry on with
discussions; this CC list is getting a bit unwieldy. Subscription
URL: http://groups.google.com/group/snakebite-list. E-mail address
is snakebi...@googlegroups.com.

I'll be sending the rest of my replies there after this e-mail.

Trent.

Frank Niessink

unread,
Jan 27, 2009, 3:07:51 PM1/27/09
to snakebite
Hi Trent,

On Jan 27, 12:32 am, Trent Nelson <python-committers-
l...@trentnelson.com> wrote:
>     And that's when it hit me.  Buildbots are fine when everything is
>     running smoothly, but nothing compares to actually having access to
>     a system when you're trying to debug something.

Any plans to open this up for open source projects developing with
python?

I'm asking on behalf of the Task Coach project (http://
www.taskcoach.org). Task Coach is an open source task/todo manager,
developed with python and wxPython. It has close to 3000 automated
unittests, roughly 60000 translation tests and a small number of other
automated tests (integration, release, deployment). It is available
for Windows, Linux and Mac OS X. One of our developers has set up
buildbots, but his machines are not always available. I'd love to have
buildbots running in a more stable manner with more different
platforms available.

Keeping my fingers crossed, Frank

Frank Wierzbicki

unread,
Jan 26, 2009, 6:52:22 PM1/26/09
to trent....@snakebite.org, python-c...@python.org, bob.go...@hp.com, ti...@idyll.org, Glynn....@sun.com, jim.w...@sun.com, Ted....@sun.com, hjan...@microsoft.com, garr...@microsoft.com, pu...@cse.msu.edu, sra...@microsoft.com, cno...@microsoft.com, di...@exchange.microsoft.com, Jimmy.S...@microsoft.com, dfu...@microsoft.com, stoc...@cse.msu.edu, pitc...@cse.msu.edu, tne...@onresolve.com, ma...@canonical.com, exa...@divmod.com, gy...@divmod.com, lhaw...@google.com, do...@ubuntu.com, fuzz...@voidspace.org.uk, oubi...@divmod.com, jacob.ka...@gmail.com, p...@python.org, cli...@cse.msu.edu

On Jan 26, 2009, at 6:32 PM, Trent Nelson wrote:
> Snakebite is my gift to you!
Wow, this will be an amazing resource. Congratulations on pulling all
of this together, and thanks. Can I mention this to the other Jython
committers or would you prefer I wait on that?

-Frank

Brett Cannon

unread,
Jan 26, 2009, 6:56:04 PM1/26/09
to trent....@snakebite.org, python-c...@python.org, Jimmy.S...@microsoft.com, do...@ubuntu.com, ti...@idyll.org, pu...@cse.msu.edu, hjan...@microsoft.com, lhaw...@google.com, garr...@microsoft.com, fuzz...@voidspace.org.uk, exa...@divmod.com, cno...@microsoft.com, bob.go...@hp.com, glynn....@sun.com, sra...@microsoft.com, pitc...@cse.msu.edu, ma...@canonical.com, di...@exchange.microsoft.com, oubi...@divmod.com, jim.w...@sun.com, Frank.Wi...@sun.com, cli...@cse.msu.edu, dfu...@microsoft.com, jacob.ka...@gmail.com, gy...@divmod.com, p...@python.org, tne...@onresolve.com, stoc...@cse.msu.edu, ted....@sun.com
[SNIP - Trent finally gets to announce Snakebite publicly]

Thanks for spear-heading this, Trent, and to everyone who helped along
the way! This is an amazing project. It makes me actually want to try
to fix a bug just to give it a go. This is definitely going to help
keep Python ahead of the pack when it comes to cross-platform support
and code quality.

I only have two questions at this point. One, is it cool to blog about
this? And two, are there plans to have buildbots running on the
machines at all so we know exactly which ones are triggering failures?
Because it would be beyond cool to have a massive buildbot page where
if I see something red on it I can just SSH directly into that machine
and get the thing green.

-Brett

M.-A. Lemburg

unread,
Jan 27, 2009, 3:30:02 AM1/27/09
to trent....@snakebite.org
Hi Trent,

that's an amazing project you've started rolling there !

Just curious: would there be more rackspace available to host other
architectures, like e.g. IBM machines ? AIX appears to be missing from
the list of supported OSes on:

http://www.snakebite.org/network

Regards,
--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, Jan 27 2009)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/

> _______________________________________________
> python-committers mailing list
> python-c...@python.org
> http://mail.python.org/mailman/listinfo/python-committers

Alexandre Vassalotti

unread,
Jan 26, 2009, 9:12:05 PM1/26/09
to trent....@snakebite.org, Jimmy.S...@microsoft.com, python-c...@python.org, do...@ubuntu.com, ti...@idyll.org, pu...@cse.msu.edu, hjan...@microsoft.com, jim.w...@sun.com, garr...@microsoft.com, fuzz...@voidspace.org.uk, exa...@divmod.com, cno...@microsoft.com, bob.go...@hp.com, glynn....@sun.com, sra...@microsoft.com, pitc...@cse.msu.edu, ma...@canonical.com, di...@exchange.microsoft.com, oubi...@divmod.com, lhaw...@google.com, stoc...@cse.msu.edu, cli...@cse.msu.edu, dfu...@microsoft.com, jacob.ka...@gmail.com, gy...@divmod.com, p...@python.org, tne...@onresolve.com, Frank.Wi...@sun.com, ted....@sun.com
[SNIP: Trent announces his awesome secret project]

There is only one word I can say: wow! And to everyone who was
involved, thank you!

This massive buildbot network is really going to help us toward making
Python a top-class platform in term of portability. I am sure that
having SSH access to the buildbots will encourage everyone to fix bugs
quickly.

Finally, I have one question. Will these buildbots be open for general
development work? It would be great to have access to them for
creating new cross-platform libraries for Python.

-- Alexandre

Alex Martelli

unread,
Jan 27, 2009, 11:20:55 AM1/27/09
to Alexandre Vassalotti, trent....@snakebite.org, Jimmy.S...@microsoft.com, python-c...@python.org, do...@ubuntu.com, ti...@idyll.org, pu...@cse.msu.edu, hjan...@microsoft.com, jim.w...@sun.com, garr...@microsoft.com, fuzz...@voidspace.org.uk, exa...@divmod.com, cno...@microsoft.com, bob.go...@hp.com, glynn....@sun.com, sra...@microsoft.com, pitc...@cse.msu.edu, ma...@canonical.com, di...@exchange.microsoft.com, oubi...@divmod.com, lhaw...@google.com, stoc...@cse.msu.edu, cli...@cse.msu.edu, dfu...@microsoft.com, jacob.ka...@gmail.com, gy...@divmod.com, p...@python.org, tne...@onresolve.com, Frank.Wi...@sun.com, ted....@sun.com
On Mon, Jan 26, 2009 at 6:12 PM, Alexandre Vassalotti
<alex...@peadrop.com> wrote:
> [SNIP: Trent announces his awesome secret project]
>
> There is only one word I can say: wow! And to everyone who was
> involved, thank you!
>
> This massive buildbot network is really going to help us toward making
> Python a top-class platform in term of portability. I am sure that
> having SSH access to the buildbots will encourage everyone to fix bugs
> quickly.
>
> Finally, I have one question. Will these buildbots be open for general
> development work? It would be great to have access to them for
> creating new cross-platform libraries for Python.

Yep, I for example would love to use this resource to ensure that gmpy
builds and runs well on more platforms than the usual
mac/windows/linux set I have easy personal access to, and I suspect
most other maintainers of third party Python add-ons feel similarly.
I'm sure the line must be drawn somewhere, however, as there will
never be enough computing resources in the "buildbot cloud" to support
the thousands of Python add-ons that are not part of the Python
distribution itself. A criterion such as "only 3rd party add ons
whose maintainers are also Python committers" (even though it would
favor gmpy) might be a tad arbitrary.


Alex

C. Titus Brown

unread,
Jan 27, 2009, 11:36:41 AM1/27/09
to Alex Martelli, Alexandre Vassalotti, trent....@snakebite.org, Jimmy.S...@microsoft.com, python-c...@python.org, do...@ubuntu.com, ti...@idyll.org, pu...@cse.msu.edu, hjan...@microsoft.com, jim.w...@sun.com, garr...@microsoft.com, fuzz...@voidspace.org.uk, exa...@divmod.com, cno...@microsoft.com, bob.go...@hp.com, glynn....@sun.com, sra...@microsoft.com, pitc...@cse.msu.edu, ma...@canonical.com, di...@exchange.microsoft.com, oubi...@divmod.com, lhaw...@google.com, stoc...@cse.msu.edu, cli...@cse.msu.edu, dfu...@microsoft.com, jacob.ka...@gmail.com, gy...@divmod.com, p...@python.org, tne...@onresolve.com, Frank.Wi...@sun.com, ted....@sun.com
On Tue, Jan 27, 2009 at 08:20:55AM -0800, Alex Martelli wrote:
-> On Mon, Jan 26, 2009 at 6:12 PM, Alexandre Vassalotti
-> <alex...@peadrop.com> wrote:
-> > [SNIP: Trent announces his awesome secret project]
-> >
-> > There is only one word I can say: wow! And to everyone who was
-> > involved, thank you!
-> >
-> > This massive buildbot network is really going to help us toward making
-> > Python a top-class platform in term of portability. I am sure that
-> > having SSH access to the buildbots will encourage everyone to fix bugs
-> > quickly.
-> >
-> > Finally, I have one question. Will these buildbots be open for general
-> > development work? It would be great to have access to them for
-> > creating new cross-platform libraries for Python.
->
-> Yep, I for example would love to use this resource to ensure that gmpy
-> builds and runs well on more platforms than the usual
-> mac/windows/linux set I have easy personal access to, and I suspect
-> most other maintainers of third party Python add-ons feel similarly.
-> I'm sure the line must be drawn somewhere, however, as there will
-> never be enough computing resources in the "buildbot cloud" to support
-> the thousands of Python add-ons that are not part of the Python
-> distribution itself. A criterion such as "only 3rd party add ons
-> whose maintainers are also Python committers" (even though it would
-> favor gmpy) might be a tad arbitrary.

Hi all,

<delurk>

I don't expect any problem with people using this for whatever they
want. If it becomes massively popular, well, then we'll just have to
ramp up the resources...!

Right now, though, I'm more interested in recruiting "good citizens",
people who can reasonably be expected to not kvetch too much as we
develop out the infrastructure. I don't see any problem with extending
the invitation to any long-term member of the Python community, and
eventually anyone who writes a polite e-mail.

That having been said, I also don't have any problem with limiting
access somewhat arbitrarily to people we "like". Gotta start somewhere!

On that front, incidentally, I hope to make the machines available to
GSoC and GHOP students, and I will be involving MSU undergrads in their
care and maintenance. I am also working to develop a more flexible and
simple set of continuous build software; more on that at PyCon.

cheers,
--titus
--
C. Titus Brown, c...@msu.edu

Steve Holden

unread,
Jan 27, 2009, 12:05:20 PM1/27/09
to ti...@idyll.org, Alex Martelli, Jimmy.S...@microsoft.com, python-c...@python.org, do...@ubuntu.com, pu...@cse.msu.edu, hjan...@microsoft.com, lhaw...@google.com, garr...@microsoft.com, fuzz...@voidspace.org.uk, exa...@divmod.com, cno...@microsoft.com, bob.go...@hp.com, glynn....@sun.com, sra...@microsoft.com, pitc...@cse.msu.edu, ma...@canonical.com, Alexandre Vassalotti, di...@exchange.microsoft.com, oubi...@divmod.com, jim.w...@sun.com, Frank.Wi...@sun.com, cli...@cse.msu.edu, dfu...@microsoft.com, jacob.ka...@gmail.com, gy...@divmod.com, p...@python.org, tne...@onresolve.com, stoc...@cse.msu.edu, ted....@sun.com, trent....@snakebite.org
C. Titus Brown wrote:
> On Tue, Jan 27, 2009 at 08:20:55AM -0800, Alex Martelli wrote:
> -> On Mon, Jan 26, 2009 at 6:12 PM, Alexandre Vassalotti
[...]

> -> > Finally, I have one question. Will these buildbots be open for general
> -> > development work? It would be great to have access to them for
> -> > creating new cross-platform libraries for Python.
> ->
> -> Yep, I for example would love to use this resource to ensure that gmpy
[understandable special pleading]

>
> Hi all,
>
> <delurk>
>
> I don't expect any problem with people using this for whatever they
> want. If it becomes massively popular, well, then we'll just have to
> ramp up the resources...!
>
That's the spirit! Demand is evidence of the need to supply!

> Right now, though, I'm more interested in recruiting "good citizens",
> people who can reasonably be expected to not kvetch too much as we
> develop out the infrastructure. I don't see any problem with extending
> the invitation to any long-term member of the Python community, and
> eventually anyone who writes a polite e-mail.
>

So that will eliminate about 25% of the community right there ;-)

> That having been said, I also don't have any problem with limiting
> access somewhat arbitrarily to people we "like". Gotta start somewhere!
>
> On that front, incidentally, I hope to make the machines available to
> GSoC and GHOP students, and I will be involving MSU undergrads in their
> care and maintenance. I am also working to develop a more flexible and
> simple set of continuous build software; more on that at PyCon.
>

I'm looking forward to that.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

C. Titus Brown

unread,
Jan 27, 2009, 3:38:37 PM1/27/09
to snakebi...@googlegroups.com
On Mon, Jan 26, 2009 at 03:56:04PM -0800, Brett Cannon wrote:
-> I only have two questions at this point. One, is it cool to blog about
-> this? And two, are there plans to have buildbots running on the
-> machines at all so we know exactly which ones are triggering failures?
-> Because it would be beyond cool to have a massive buildbot page where
-> if I see something red on it I can just SSH directly into that machine
-> and get the thing green.

I'll speak for Trent here...

Yes, and yes.

Brett Cannon

unread,
Jan 27, 2009, 5:20:50 PM1/27/09
to snakebi...@googlegroups.com

Done and great!

-Brett

Trent Nelson

unread,
Jan 27, 2009, 6:47:10 PM1/27/09
to Brett Cannon, snakebi...@googlegroups.com
On Mon, Jan 26, 2009 at 03:56:04PM -0800, Brett Cannon wrote:

> Because it would be beyond cool to have a massive buildbot page where
> if I see something red on it I can just SSH directly into that machine
> and get the thing green.

Ohhhhh Brett, so innocent ;-)

If you think that's beyond cool, I think you've got a pretty good
chance of going into a full blown seizure when you see what we've
got in mind for the future.

> -Brett

Trent.

Trent Nelson

unread,
Jan 27, 2009, 7:16:09 PM1/27/09
to Frank Wierzbicki, snakebi...@googlegroups.com

Sure, feel free to let the other committers know. Once I've done a
bit of emergency tidying up of the website so it looks half present-
able, I'll forward on the initial e-mail I sent last night to the
python-(dev|users|announce) lists and let that serve as the official
announcement.

Feel free to blog/twitter/whatever about Snakebite in the meantime.

> -Frank

Trent.

Brett Cannon

unread,
Jan 27, 2009, 8:08:51 PM1/27/09
to snakebi...@googlegroups.com
On Tue, Jan 27, 2009 at 15:47, Trent Nelson <trent....@snakebite.org> wrote:
>
> On Mon, Jan 26, 2009 at 03:56:04PM -0800, Brett Cannon wrote:
>
>> Because it would be beyond cool to have a massive buildbot page where
>> if I see something red on it I can just SSH directly into that machine
>> and get the thing green.
>
> Ohhhhh Brett, so innocent ;-)
>

Yeah, I know, I have some things to learn. =)

> If you think that's beyond cool, I think you've got a pretty good
> chance of going into a full blown seizure when you see what we've
> got in mind for the future.

Great! This will definitely help with my New Year's resolution of
spending two hours a week on python-dev community work which includes
working on flaky tests so people don't have red herrings to chase
after.

-Brett

Trent Nelson

unread,
Jan 27, 2009, 8:56:30 PM1/27/09
to M.-A. Lemburg, snakebi...@googlegroups.com
On Tue, Jan 27, 2009 at 09:30:02AM +0100, M.-A. Lemburg wrote:
> Hi Trent,
>
> that's an amazing project you've started rolling there !
>
> Just curious: would there be more rackspace available to host other
> architectures, like e.g. IBM machines ? AIX appears to be missing from
> the list of supported OSes on:
>
> http://www.snakebite.org/network

Well spotted, there is indeed a noticeable absence of AIX coverage.
Right at this very moment, we actually do have an AIX box; I managed
to pick up a 2nd (or 3rd/4th/5th, who knows with eBay) hand Intelli-
Station 275 9114 that I put AIX 6 on.

However, I'm not entirely sure how it happened, but that box is in
Titus's lab at MSU in East Lansing, and I seem to have the hard
drive here in my apartment in London ;-) We'll look at rectifying
that down the track.

For the record, for a company as large as IBM that spends so much
money on open source (or at least spends a lot of money trying to
inform people they're big on open source), their website, as usual,
is absolutely useless. I don't know how many links on their open
source home page I traversed, but not once did I find any contact
details or info or anything about someone to contact to discuss
a relationship between IBM and Snakebite.

(As opposed to say, HP, Microsoft and Sun, who all have open source
websites that make it very easy to get in touch with people who can
make decisions.)

If anyone knows of IBM contacts within their open source division;
please let me know.


Trent.

illume

unread,
Jan 27, 2009, 11:33:27 PM1/27/09
to snakebite
Hi,

sounds like an awesome project!

The pygame.org project would love to have access to this snakebite
project :)


Please email me details if it would be ok for us to use the machines.
We only have around 8 committers... but we could probably get by with
1-2 of us having access to the machines so we can set up our "The
Spectacularly Adequate Automated Pygame Build Page" build bots on
there.

We'll probably keep using our current machines, as we release binaries
from the build bots too... but these machines would be great for
testing/compiling code!


cheers,



On Jan 27, 10:32 am, Trent Nelson <python-committers-

M.-A. Lemburg

unread,
Jan 28, 2009, 11:47:59 AM1/28/09
to Trent Nelson, snakebi...@googlegroups.com
Hi Trent,

I've started a request into IBM to search for a contact regarding
these matters. I'll keep you informed in private email if anything
results from this.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, Jan 28 2009)

M.-A. Lemburg

unread,
Jan 28, 2009, 11:51:30 AM1/28/09
to snakebi...@googlegroups.com, Jimmy.S...@microsoft.com, python-c...@python.org, do...@ubuntu.com, ti...@idyll.org, pu...@cse.msu.edu, hjan...@microsoft.com, lhaw...@google.com, garr...@microsoft.com, fuzz...@voidspace.org.uk, exa...@divmod.com, cno...@microsoft.com, bob.go...@hp.com, glynn....@sun.com, sra...@microsoft.com, pitc...@cse.msu.edu, ma...@canonical.com, di...@exchange.microsoft.com, oubi...@divmod.com, jim.w...@sun.com, Frank.Wi...@sun.com, cli...@cse.msu.edu, dfu...@microsoft.com, jacob.ka...@gmail.com, gy...@divmod.com, p...@python.org, tne...@onresolve.com, stoc...@cse.msu.edu, ted....@sun.com

Would it be possible to have such a mailing list setup on python.org ?
Or perhaps have Mailman running on snakebite.org ?

Thanks,

Barry Warsaw

unread,
Jan 28, 2009, 1:29:21 PM1/28/09
to M.-A. Lemburg, snakebi...@googlegroups.com, Jimmy.S...@microsoft.com, python-c...@python.org, do...@ubuntu.com, ti...@idyll.org, pu...@cse.msu.edu, hjan...@microsoft.com, jim.w...@sun.com, garr...@microsoft.com, fuzz...@voidspace.org.uk, exa...@divmod.com, cno...@microsoft.com, bob.go...@hp.com, glynn....@sun.com, sra...@microsoft.com, pitc...@cse.msu.edu, ma...@canonical.com, di...@exchange.microsoft.com, oubi...@divmod.com, lhaw...@google.com, stoc...@cse.msu.edu, cli...@cse.msu.edu, dfu...@microsoft.com, jacob.ka...@gmail.com, gy...@divmod.com, p...@python.org, tne...@onresolve.com, Frank.Wi...@sun.com, ted....@sun.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 28, 2009, at 11:51 AM, M.-A. Lemburg wrote:

> On 2009-01-27 21:01, Trent Nelson wrote:
>> I've just set up a mailing list for those that want to carry on
>> with
>> discussions; this CC list is getting a bit unwieldy. Subscription
>> URL: http://groups.google.com/group/snakebite-list. E-mail address
>> is snakebi...@googlegroups.com.
>>
>> I'll be sending the rest of my replies there after this e-mail.
>
> Would it be possible to have such a mailing list setup on python.org ?
> Or perhaps have Mailman running on snakebite.org ?

+1 for either. Send a message to postm...@python.org if you want
the former. It should definitely be approved.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSYCkAXEjvBPtnXfVAQLlxgP9F/14phSrD+HXsOLcgaBaymORuILxFpFp
AXVH7o1tebi7oOXamB/2dMJaUkVAz7Sf8rtnv4tP0l/Z/1kHJFZrRZQaLTS3KxYh
6ooBVHkzokPGtmOS5Ms8ijuMiI3e3OBHL90/gkVf3O3G1+ELM3CTeqp5HgHD6/J/
DY7nYkvJfS4=
=6wKo
-----END PGP SIGNATURE-----

Dirkjan Ochtman

unread,
Jan 28, 2009, 4:44:57 PM1/28/09
to snakebite
On Jan 27, 5:36 pm, "C. Titus Brown" <c...@msu.edu> wrote:
> Right now, though, I'm more interested in recruiting "good citizens",
> people who can reasonably be expected to not kvetch too much as we
> develop out the infrastructure.  I don't see any problem with extending
> the invitation to any long-term member of the Python community, and
> eventually anyone who writes a polite e-mail.

I've been running a buildbot setup to run the tests for Mercurial; it
would be really nice if I could expand the setup by adding platforms
through Snakebite. Like you mention, debugging problems on build
slaves you don't have access to can be kind of a bitch, and it would
be nice if I could go in and try stuff... We would love to take
advantage of your setup.

So please let me know when you're admitting other (Python-based)
projects!

Cheers, and thanks for everything,

Dirkjan

Trent Nelson

unread,
Jan 29, 2009, 2:31:15 PM1/29/09
to snakebi...@googlegroups.com
Hi Dirkjan,

On Wed, Jan 28, 2009 at 01:44:57PM -0800, Dirkjan Ochtman wrote:


>
> On Jan 27, 5:36?pm, "C. Titus Brown" <c...@msu.edu> wrote:
> > Right now, though, I'm more interested in recruiting "good citizens",
> > people who can reasonably be expected to not kvetch too much as we

> > develop out the infrastructure. ?I don't see any problem with extending


> > the invitation to any long-term member of the Python community, and
> > eventually anyone who writes a polite e-mail.
>
> I've been running a buildbot setup to run the tests for Mercurial; it
> would be really nice if I could expand the setup by adding platforms
> through Snakebite. Like you mention, debugging problems on build
> slaves you don't have access to can be kind of a bitch, and it would
> be nice if I could go in and try stuff... We would love to take
> advantage of your setup.
>
> So please let me know when you're admitting other (Python-based)
> projects!

To follow on from Titus's sentiments, I think what we'll do over the
next couple of months is give priority to projects with members that
can help out with a lot of the things that need doing when setting
up a network of this size.

Trent.

illume

unread,
Jan 29, 2009, 11:08:02 PM1/29/09
to snakebite
Awesome.


We've been working towards a distributed testing system that you might
be interested in.

It's kind of simple really...

The test results can optionally be submitted to a central repository
by anyone who has the package installed.

eg.
>>> import pygame.test
>>> pygame.test.run()
>>> pygame.test.submit()

That will just upload the results of the tests to a central place.

Of course this requires the tests to be installed with the package.

Then you can take advantage of millions, thousands, or hundreds of
people using your package for submitting test results. Not as good as
a compile farm, but should provide far greater coverage of different
types of hardware.


Status of project?

DONE 1. the "The Spectacularly Adequate Automated Pygame Build Page"
code was extended ( http://thorbrian.com/pygame/builds.php ) to turn
it into a website that can take submissions from random sources. He
also refactored the testing system to make test results in a better
format.
DONE 2. tests and test runner are included with pygame.
DONE 3. testing system was made more resiliant and useful -- using
multiple processes with test run-time timeouts ( so one segfault can't
stuff up all the tests ) -- and timing of tests as benchmarks.
4. write a submit function which urllibs up the test result dictionary
to the central repository.
5. release version of package(pygame 1.9) with everything included.
6. wait for test results to start piling in!
7. write reporting code for all the test results.

Anyway... thought it was an interesting idea that could be applied to
snakebite (possibly as a central repository for test results) and for
implementations of python, and different python packages. It's not a
compile farm exactly -- but has similar goals.


ps. the tinypy.org (python in 64K of code) project would also love to
use snakebite too :)

Trent Nelson

unread,
Feb 2, 2009, 2:40:18 AM2/2/09
to snakebi...@googlegroups.com, M.-A. Lemburg, Jimmy.S...@microsoft.com, python-c...@python.org, do...@ubuntu.com, ti...@idyll.org, pu...@cse.msu.edu, hjan...@microsoft.com, jim.w...@sun.com, garr...@microsoft.com, fuzz...@voidspace.org.uk, exa...@divmod.com, cno...@microsoft.com, bob.go...@hp.com, glynn....@sun.com, sra...@microsoft.com, pitc...@cse.msu.edu, ma...@canonical.com, di...@exchange.microsoft.com, oubi...@divmod.com, lhaw...@google.com, stoc...@cse.msu.edu, cli...@cse.msu.edu, dfu...@microsoft.com, jacob.ka...@gmail.com, gy...@divmod.com, p...@python.org, tne...@onresolve.com, Frank.Wi...@sun.com, ted....@sun.com
On Wed, Jan 28, 2009 at 01:29:21PM -0500, Barry Warsaw wrote:
> On Jan 28, 2009, at 11:51 AM, M.-A. Lemburg wrote:
>
> > On 2009-01-27 21:01, Trent Nelson wrote:
> >> I've just set up a mailing list for those that want to carry on
> >> with
> >> discussions; this CC list is getting a bit unwieldy. Subscription
> >> URL: http://groups.google.com/group/snakebite-list. E-mail address
> >> is snakebi...@googlegroups.com.
> >>
> >> I'll be sending the rest of my replies there after this e-mail.
> >
> > Would it be possible to have such a mailing list setup on python.org ?
> > Or perhaps have Mailman running on snakebite.org ?
>
> +1 for either. Send a message to postm...@python.org if you want
> the former. It should definitely be approved.

I'll follow up with the interested parties in private.

Trent.

seanlynch

unread,
Feb 12, 2009, 10:03:01 AM2/12/09
to snakebite


On Jan 26, 5:32 pm, Trent Nelson <python-committers-
l...@trentnelson.com> wrote:
...
>     Snakebite is my gift to you!
>
>         Trent.

Trent,

This is probably the best idea since sourceforge!

Trent Nelson

unread,
Feb 12, 2009, 11:12:48 AM2/12/09
to snakebi...@googlegroups.com
On Thu, Feb 12, 2009 at 07:03:01AM -0800, seanlynch wrote:
>
>
>
> On Jan 26, 5:32?pm, Trent Nelson <python-committers-
> l...@trentnelson.com> wrote:
> ...
> > ? ? Snakebite is my gift to you!
> >
> > ? ? ? ? Trent.

>
> Trent,
>
> This is probably the best idea since sourceforge!

Haha, is that a compliment or an insult? ;-)

Trent.

Steve Holden

unread,
Feb 12, 2009, 11:14:35 AM2/12/09
to snakebi...@googlegroups.com
I should treat it as a Dire Warning (tm) if I were you. The road to
hell, etc.

regards
Steve


C. Titus Brown

unread,
Feb 12, 2009, 11:16:24 AM2/12/09
to snakebi...@googlegroups.com
On Thu, Feb 12, 2009 at 11:14:35AM -0500, Steve Holden wrote:
->
-> Trent Nelson wrote:
-> > On Thu, Feb 12, 2009 at 07:03:01AM -0800, seanlynch wrote:
-> >
-> >>
-> >> On Jan 26, 5:32?pm, Trent Nelson <python-committers-
-> >> l...@trentnelson.com> wrote:
-> >> ...
-> >>
-> >>> ? ? Snakebite is my gift to you!
-> >>>
-> >>> ? ? ? ? Trent.
-> >>>
-> >> Trent,
-> >>
-> >> This is probably the best idea since sourceforge!
-> >>
-> >
-> > Haha, is that a compliment or an insult? ;-)
-> >
-> I should treat it as a Dire Warning (tm) if I were you. The road to
-> hell, etc.

That's OK -- I'll "take care" of things if they go that far...

(Anyone see those shotgun incendiary rounds they used in Burn Notice in
the 1st season? Fantastic way to shut down a computer, methinks.)

Trent Nelson

unread,
Feb 12, 2009, 11:26:33 AM2/12/09
to snakebi...@googlegroups.com
On Thu, Feb 12, 2009 at 08:16:24AM -0800, C. Titus Brown wrote:
>
> On Thu, Feb 12, 2009 at 11:14:35AM -0500, Steve Holden wrote:
> ->
> -> Trent Nelson wrote:
> -> > On Thu, Feb 12, 2009 at 07:03:01AM -0800, seanlynch wrote:
> -> >>>
> -> >> Trent,
> -> >>
> -> >> This is probably the best idea since sourceforge!
> -> >>
> -> >
> -> > Haha, is that a compliment or an insult? ;-)
> -> >
> -> I should treat it as a Dire Warning (tm) if I were you. The road to
> -> hell, etc.
>
> That's OK -- I'll "take care" of things if they go that far...
>
> (Anyone see those shotgun incendiary rounds they used in Burn Notice in
> the 1st season? Fantastic way to shut down a computer, methinks.)

I'd google it, but alas, having just tried to download something
from an Advertis^WSourceForge mirror, my browser has decided to
call it quits and go its own way. Shame, really, I'll miss the
lil' fella'.


Sean Lynch

unread,
Feb 12, 2009, 11:33:58 AM2/12/09
to snakebi...@googlegroups.com
Compliment.

I know sourceforge is crufty now, but it was revolutionary when it
started and gave a lot of small projects a leg up. It spurred a round
of collaboration.

It might be full of text editors and more text editors, and "I'm
working on this project for class, it's a text editor" today.

When it started it gave developers something they didn't have easy
access to in one centralized place.

snakebite should allow app testing on multiple platforms. A need for
today's developers that is not easy to come by in one place.

Trent Nelson

unread,
Feb 12, 2009, 12:23:17 PM2/12/09
to snakebi...@googlegroups.com
On Thu, Feb 12, 2009 at 10:33:58AM -0600, Sean Lynch wrote:
>
> On Thu, Feb 12, 2009 at 10:26 AM, Trent Nelson
> <trent....@snakebite.org> wrote:
> >> -> > Haha, is that a compliment or an insult? ;-)

> Compliment.


>
> I know sourceforge is crufty now, but it was revolutionary when it
> started and gave a lot of small projects a leg up. It spurred a round
> of collaboration.
>
> It might be full of text editors and more text editors, and "I'm
> working on this project for class, it's a text editor" today.
>
> When it started it gave developers something they didn't have easy
> access to in one centralized place.
>
> snakebite should allow app testing on multiple platforms. A need for
> today's developers that is not easy to come by in one place.

Glad I pushed for clarity ;-) That's an excellent summation of one
of the key goals: giving smart, motivated people access to Cool
Stuff For Free (tm).

The 'all in one place' argument is a good one. Programmes like HP's
Test Drive, OpenSUSE's buildout thingy, Sun's OpenSolaris farm are
all fantastic in principle, but they're all aimed at the individual
level. Everyone has to go in and spend time setting up a dev. env.
from scratch (and maybe not even one that'll persist when you log
out or your time expires). That's a time consuming process for one
person, let alone an entire team. And if you have to do that every
time you log in, gah.

Snakebite's aimed at the project level. A project is brought onto
Snakebite; individuals aren't. As part of your project's induction,
we want you to tell us everything you need (or want) in order to
develop your software optimally.

Need access to SQL Server 2000, 2005 and 2008, Oracle 8-10 and DB2
8-9? Great, leave it with us and we'll endeavour to sort that out
for you. Got a couple of developers in your project that want to
get their hands on the latest NIC|GPU|<whatever> card, but you have
had no luck getting donations from the vendor? No problem, let us
leverage Snakebite to contact the vendor on your behalf and try and
arrange some donations. It's a win-win for you and them. ('Them',
in this context, refers to companies that see the value in getting
their new hardware into the hands of very clever people, who, with
even just a little bit of support, will crank out some high quality
drivers for their preferred platform. What company wouldn't spare
a couple of units here or there (negligible cost) in exchange for
increased platform coverage? It doesn't really matter, they won't
be around for long enough, as the vendors who *do* get it will eat
them up. Snakebite reduces the need for these vendors to coordin-
ate communication and donations to an infinite number of individu-
als. They just deal with us, and we farm out their contributions
to the highest bid^W^W^W anyone that wants it.)

Or maybe, just maybe, you need access to as many different Linux
platforms as possible? No problem, we'll drop Microsoft a note
for you and see if we can leverage some of their OSL resources.

(Bet you didn't know that Microsoft have a crème de la crème Open
Source Lab in MA somewhere that has over 300 different Linux plat-
forms/configurations that they use for Windows/Linux interop dev,
and have been beating down the door to see where they can offer
help to Snakebite. Those little rascals! They're the enemy of
Linux! How dare they show such support for Open Source!)

Nothing like ending a rant with a troll-provoking sarcastic Micro-
soft bashing ;-) Emphasis on 'sarcastic'; actual Microsoft bashing
is just so 1999. (I just did it again!)

Enjoy.

Trent.

illume

unread,
Feb 13, 2009, 1:51:25 AM2/13/09
to snakebite
yes, the sourceforge compile farm was pretty good at the start... but
didn't really scale to heaps of projects.

It's of course closed down last I heard.


On Feb 13, 2:03 am, seanlynch <sean.seanly...@gmail.com> wrote:

C. Titus Brown

unread,
Feb 13, 2009, 8:38:41 PM2/13/09
to snakebi...@googlegroups.com
On Thu, Feb 12, 2009 at 10:51:25PM -0800, illume wrote:
-> yes, the sourceforge compile farm was pretty good at the start... but
-> didn't really scale to heaps of projects.
->
-> It's of course closed down last I heard.

With snakebite, we're hoping to keep things small and focused enough to
be of real use to a few select Open Source communities and projects. Or
at least *I* am ;)

--titus

Reply all
Reply to author
Forward
0 new messages