Django's testing infrastructure

[Eric Holscher]

Hey everyone,

During the sprints, I worked to set up a hudson instance for Django. This is hopefully going to be the way that we are going to go forward for now with doing continuous integration for Django. I have a pretty good setup going currently, and want make it really fantastic. At this point in time, what we really need now is some more hardware to be able to run tests on.

The current setup is hosted at: http://hudson.djangoproject.com/

Currently, I have tests running on the following architectures:

Django trunk:

Solaris:
Python 2.4-2.5
Databases: sqlite, postgres, mysql

Ubuntu:
Python 2.5-2.6
Databases: sqlite, postgres, mysql

Django 1.1.X:

Solaris:
Python 2.5
Databases: sqlite, postgres

This gives us pretty good coverage currently, but the whole point of doing CI is that we catch bugs on other platforms we wouldn't normally run on.

What we need
===========

So I'm looking for people who can offer other boxes that they would like to see tested. Currently the big ones we aren't covering are Windows boxes, and the Oracle backends. There are lots of other permutations that we could be testing against (Different python runtimes being a good example).

However, we don't want to get in a situation where boxes that people have set up just disappear. So, I'm only currently looking for machines that people would be able to dedicate to the effort. We would require a django-testing user account on the box, with our SSH key on it. There would also be a team of trusted users, who would have access to this key and thus your machine.

We want the build farm to be stable and useful, and in the past we have had too much trouble having machines just disappear.

Requirements
===========

Currently the hudson requirements seem to be about <1GB of disk space, with 512MB of ram. I'm also looking into some pony build/barn based alternatives that would knock the memory requirements down pretty substantially. However, for the current 1.2 release it looks like hudson is how we're going to make it going forward.

Note that for $20/mo a 512MB machine can be run on Rackspace cloud, so another way that we might be able to get this going is to be able to have donations to the DSF, and have them get some dedicated rackspace boxes. However, for now, I'm hoping that we can cobble together enough stuff to get 1.2 tested really well.

Feedback
==========

If you have any thoughts on CI, or any advice, I would love to hear it. I'm trying to make our Continuous Integration setup really kick ass, so feedback is necessary to do this. I have some notes and ideas that I have been recording while setting things up over at the pycon etherpad:

http://pyconpads.net/django-testing

Let me know if you have any thoughts, questions, or concerns.

Cheers,
Eric

[Sean O'Connor]

A platform we probably should get into the mix is a CentOS/RHEL 5 box.  I suspect a significant portion of the community is stuck on such boxes and given the ancient versions of everything available under RHEL I'm sure that there are things which will break there and not in a developer's standard environment.

I personally don't have a suitable RHEL box laying around but this seems like a good candidate for either another volunteer or DSF funds.

____________________________
Sean O'Connor
http://seanoc.com

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Tobias McNulty]

I think this is a great move and will be an awesome resource for the community.  I'm working on getting some CPU time at the OSU OSL and/or from Caktus to contribute to the effort.

Cheers,

Tobias

--
Tobias McNulty
Caktus Consulting Group, LLC
P.O. Box 1454
Carrboro, NC 27510
(919) 951-0052
http://www.caktusgroup.com

[Russell Keith-Magee]

On Fri, Feb 26, 2010 at 11:54 AM, Eric Holscher <er...@ericholscher.com> wrote:
> Hey everyone,
>
> During the sprints, I worked to set up a hudson instance for Django. This is
> hopefully going to be the way that we are going to go forward for now with
> doing continuous integration for Django. I have a pretty good setup going
> currently, and want make it really fantastic. At this point in time, what we
> really need now is some more hardware to be able to run tests on.

Great work Eric!

> So I'm looking for people who can offer other boxes that they would like to
> see tested. Currently the big ones we aren't covering are Windows boxes, and
> the Oracle backends. There are lots of other permutations that we could be
> testing against (Different python runtimes being a good example).

Two others to put on this list of unusual platforms that need testing
(in case they aren't on your radar already):

* MySQL has two configurations that require testing: MyISAM and InnoDB.

* We also need testing for GIS configurations.

Russ %-)

[Stephen Wolff]

We'd like to offer a CentOS 5 machine with Python 2.6 / MySQL 5.0
(InnoDB and MyISAM) for testing (with Lighttpd 1.4.23 - if that is used
in any tests). We'll try and set it up over the next week.

Stephen

[Jirka Vejrazka]

Hi Eric,

great activity, thanks!

During the EuroDjango spring in Prague, I set up a Ubuntu + Oracle
10g test machine (buildbot) under Jacob's supervision. Shortly after,
Jacob has set up a mailing list for buildbots. However, there were no
followups from either side and my test machine slowly died as I did
poor job maintaining it.

If there is some traction, I'd be more than happy to
resurrect/reinstall the machine and give you an access to it.

Cheers

Jirka

[simonjwoolf]

Hi,

We would be happy to contribute space on one of our staging dedicated
servers.
Our stack is:

CentOS 5
MySQL 5.0 (using InnoDB as standard, but you are welcome to have a
MyISAM database as well)
Python 2.6
Lighttpd 1.4.23

Let me know if this is useful, and perhaps contact me privately to
discuss details of getting it set up.

Simon Woolf
Loopo

On Feb 26, 4:08 am, "Sean O'Connor" <sean.b.ocon...@gmail.com> wrote:
> A platform we probably should get into the mix is a CentOS/RHEL 5 box.  I
> suspect a significant portion of the community is stuck on such boxes and
> given the ancient versions of everything available under RHEL I'm sure that
> there are things which will break there and not in a developer's standard
> environment.
>
> I personally don't have a suitable RHEL box laying around but this seems
> like a good candidate for either another volunteer or DSF funds.
>
> ____________________________

> Sean O'Connorhttp://seanoc.com

> > django-develop...@googlegroups.com<django-developers%2Bunsu...@googlegroups.com>

[Cramet Matthieu]

Wouldn't it be interesting to build some Virtual Machines that people could grab and deploy directly in the cloud ? It might interest people willing to contribute hardware but not having much time to dedicate at maintaining it.

I personnaly have a spare Desktop with a Quad Core and 4G of RAM that could run at leat 4 VMs.

My 2 cents :-D

Matthieu

To unsubscribe from this group, send email to django-develop...@googlegroups.com.

[Jacob Kaplan-Moss]

On Fri, Feb 26, 2010 at 10:32 AM, Cramet Matthieu <m.cr...@gmail.com> wrote:
> Wouldn't it be interesting to build some Virtual Machines that people could
> grab and deploy directly in the cloud ? It might interest people willing to
> contribute hardware but not having much time to dedicate at maintaining it.
> I personnaly have a spare Desktop with a Quad Core and 4G of RAM that could
> run at leat 4 VMs.

If you (or anyone else) would be willing to develop some VMs --
VMWare, VirtualBox, or EC2 images would probably be best -- I have
space/bandwidth to host 'em for download. I think it's a fantastic
idea. Especially EC2 instances, actually: if someone does the grunt
work, I think Hudson can boot and manage them directly.

Jacob

[Steven Elliott Jr]

I have about 4 apple Xserves with quad cores and 16 GBs or RAM sitting
in my server room at work. I will see if I can use them for this
purpose. I don't think it will be a problem since I'm the CIO and am
pretty much left alone to do what i want with our hardware. I'll post
back next week after I get the ok from the board.

Sent from my iPhone

On Feb 26, 2010, at 11:43 AM, Jacob Kaplan-Moss <ja...@jacobian.org>
wrote:

> --
> You received this message because you are subscribed to the Google
> Groups "Django developers" group.
> To post to this group, send email to django-d...@googlegroups.com
> .
> To unsubscribe from this group, send email to django-develop...@googlegroups.com

[Christopher Petrilli]

I can toss in a Core i7 w/12GB for testing as well, if someone can
give me the VMs in either VirtualBox or Linux KVM format. I think this
is a brilliant idea, and would even be willing to contribute some
money to cover EC2/etc, if that's what it takes.

Chris

--
| Chris Petrilli
| petr...@amber.org

[Mikhail Korobov]

That's great news, thanks!

A very minor issue: web server returns 'Content-Type:text/
html;charset=ISO-8859-1' header for this page: http://hudson.djangoproject.com/monitor/?
but the actual page encoding is utf-8 so there are strange symbols
instead of translated strings.

[Eric Holscher]

Awesome response!

Thanks for the offering of support everyone. Luckily, with hudson, it's pretty simple to get this all set up. We basically just need to figure out what the infrastructure looks like.

I don't know if it's going to make more sense to try and set up dedicated Database servers, or try and set up each client to have the necessary databases running on each one.

Currently, all I need for a slave machine is a box with a publically accessable IP, with an account with the Django Testmaster's key on it. Hudson will take care of everything from there, at least on the Django side.

Hudson will ssh in and pull down Java if necessary, then run itself as a client. It seems we will need to have some kind of standard setup for the databases, with either local connections being let through without auth, or with a standard username and password (maybe django/django), so that we can do the database operations.

If anyone has much experience here, I'll probably be doing some benchmarking to figure out what kind of app to database server ratio we need, and what makes sense in that regard. 

I'll give this some more thought this weekend, and try and email everyone who has offered support. I know there is a django-buildbots mailing list that never got used last time we did this, i dunno if we want to keep the conversation here and more public, or put it over there and keep the archives around for setting stuff up.

Thanks again for all the offers, it's really great, now we just need to figure out how to scale this up without using lots of human hours in the process.

Cheers,

Eric

--

You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

--
Eric Holscher
Web Developer at The World Company in Lawrence, Ks
http://ericholscher.com

[Stephen Wolff]

btw - Simon and I work together - so the identical specs were actually one machine... anyhow, we're really pleased to be involved, and to see how this all goes. We've been toying with CI for a while - including trying out the Trac Bitten plugin (http://bitten.edgewall.org/) - which could be another possibility which would tie in with Django's Trac instance. I don't think it's quite as mature as Hudson though.

Stephen

[Chris Hasenpflug]

On Feb 26, 10:43 am, Jacob Kaplan-Moss <ja...@jacobian.org> wrote:
> If you (or anyone else) would be willing to develop some VMs --
> VMWare, VirtualBox, or EC2 images would probably be best -- I have
> space/bandwidth to host 'em for download. I think it's a fantastic
> idea. Especially EC2 instances, actually: if someone does the grunt
> work, I think Hudson can boot and manage them directly.

If EC2 is a direction we'd like to go (I think it makes a lot of
sense), I'd be happy to invest some time in building images. But
don't want it to be for not if the plan is really to stay with always
on machines.

[Cramet Matthieu]

I don't know i had your message right Chris but i think the purpose is to keep those machines running for a __long__ time. BUT having VMs ready would allow anyone to offer a new machine/slice in the case that someone finally decide to end his participation.

About VMs formats, I am not an expert but i previously used the ones from Turnkey that can be imported into VMware or Virtualbox and even uploaded to EC2 or any other cloud service i think.

@Eric : Can you a create a Wiki page somewhere to list the missing architectures ?

[Jakub Uniejewski]

Great work!

When it comes to hardware, I was kind of convinced that Django have
been already using the snakebite network to access various platforms
for builds testing (http://djangoninja.com/post/73799680/django-on-
snakebite)

Isn't their network compatible? (more about SnakeBite: http://www.snakebite.org/)

Jakub