[mozbase] RIP autobot@k0s.org

[Jeff Hammel]

Hitherto, I had gotten away with running autobot on my rackspace hosted
k0s.org. However, the python on k0s.org is 2.6, not 2.7 as is now
(implicitly) required, so all builds fail with e.g.

"""

======================================================================
ERROR: Test installing webapps into a profile that has no prior webapps
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jhammel/mozilla/src/autobot/bot/slave/mozbase/build/env/src/mozbase/mozprofile/tests/test_webapps.py", line 112, in test_install_webapps
webapps = WebappCollection(self.profile, apps=self.manifest_path_1)
File "/home/jhammel/mozilla/src/autobot/bot/slave/mozbase/build/env/src/mozbase/mozprofile/mozprofile/webapps.py", line 124, in __init__
self.extend(self.read_json(app))
File "/home/jhammel/mozilla/src/autobot/bot/slave/mozbase/build/env/src/mozbase/mozprofile/mozprofile/webapps.py", line 278, in read_json
ret.append(Webapp(**d))
TypeError: __init__() keywords must be strings

======================================================================
ERROR: Tests installing webapps when the webapps directory already exists
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jhammel/mozilla/src/autobot/bot/slave/mozbase/build/env/src/mozbase/mozprofile/tests/test_webapps.py", line 159, in test_install_webapps_preexisting
manifest_json_2 = WebappCollection.read_json(self.manifest_path_2)
File "/home/jhammel/mozilla/src/autobot/bot/slave/mozbase/build/env/src/mozbase/mozprofile/mozprofile/webapps.py", line 278, in read_json
ret.append(Webapp(**d))
TypeError: __init__() keywords must be strings

======================================================================
ERROR: Tests WebappCollection.read_json
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jhammel/mozilla/src/autobot/bot/slave/mozbase/build/env/src/mozbase/mozprofile/tests/test_webapps.py", line 34, in test_read_json_manifest
manifest_json_1 = WebappCollection.read_json(self.manifest_path_1)
File "/home/jhammel/mozilla/src/autobot/bot/slave/mozbase/build/env/src/mozbase/mozprofile/mozprofile/webapps.py", line 278, in read_json
ret.append(Webapp(**d))
TypeError: __init__() keywords must be strings

======================================================================
ERROR: Tests the methods of the WebappCollection object
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jhammel/mozilla/src/autobot/bot/slave/mozbase/build/env/src/mozbase/mozprofile/tests/test_webapps.py", line 83, in test_webapp_collection
self.assertIsInstance(webapps[0], Webapp)
AttributeError: 'WebappTest' object has no attribute 'assertIsInstance'

----------------------------------------------------------------------
Ran 99 tests in 123.810s

FAILED (errors=4)
program finished with exit code 1
elapsedTime=125.600304
"""

For this reason I have deactivated aut...@k0s.org.

Sadly, the obvious solution of upgrading python on that machine is not
trivial. k0s.org is on ubuntu 9.10. While I would love to and should
upgrade to 12.x (or 13.04 soon enough), being a remote server I have
hesitated to do this (for 4 years) since there is a large class of
problems whereby if the update goes wrong, the machine could effectively
become non-responsive and my class of support is such that I can ask for
reboots or OS reimaging. I don't have time to do any of this right
now. If someone wished to, I am more than happy to provide credentials.

So what are the alternatives?

We can go without mozbase CI. I for one think that this is a bad idea
especially as we're in the midst of getting more of our infrastructure
and test harnesses on mozbase. OTOH, if no one cares about CI than me,
then that POV is probably inaccurate.

If so desired, :jgriffin has volunteered with getting a modern ubuntu
machine set up internal to our cluster if we wanted to put the existing
autobot infrastructure there. In addition to fixing the python problem,
this would also allow autobot to send mail on failed builds to relevant
parties (it actually does this already, but ABICT I am the only one that
gets them do to the way that SMTP is configured on k0s.org -- that is,
badly -- so most mail services will consider them spam). We would need
to ensure that the public waterfall remained public, and could
conceivably have a firewalled private waterfall where builds could be
forced, if desired. In addition, others could contribute code and
maintenance if desired and make it a mozbase-team asset vs something
that just jeff runs.

Alternatively, if someone wanted to spearhead the initiative (ouch!),
jenkins could be set up for mozbase CI. I have no interest or love of
jenkins work so I am personally uninterested in doing this. However, if
someone(s) else wanted to take this on, I really don't care about what
platform we use for CI as long as I don't have to care about it (that
is, there's no "extra" stuff to do for CI). Personally, I consider
buildbot better architected and easier to do complex things,
particularly having but the autobot layer over it for my purposes here.
But again, if someone else wants to own this, then I'm happy to give it
over.

So for now, we are CI-free, so please be careful. I'll talk with
Jonathan following his return from Las Vegas (unless he bets the mozbase
code in a poker game) and we can finalize whatever decision then. I'm
more than happy to talk about the architectural fine points of buildbot
vs jenkins but think that that should probably occur off-list, though
feel free to say I'm wrong.

Jeff

[William Lachance]

On 03/04/2013 12:48 PM, Jeff Hammel wrote:

> Alternatively, if someone wanted to spearhead the initiative (ouch!),
> jenkins could be set up for mozbase CI. I have no interest or love of
> jenkins work so I am personally uninterested in doing this. However, if
> someone(s) else wanted to take this on, I really don't care about what
> platform we use for CI as long as I don't have to care about it (that
> is, there's no "extra" stuff to do for CI). Personally, I consider
> buildbot better architected and easier to do complex things,
> particularly having but the autobot layer over it for my purposes here.
> But again, if someone else wants to own this, then I'm happy to give it
> over.

An alternative which requires little maintenance might be to use
travis-ci, which only requires adding a .yml file to the repo and
setting up a github service hook. I filed a bug to investigate using this:

https://bugzilla.mozilla.org/show_bug.cgi?id=847692

You can see an example of it in action here:

https://travis-ci.org/wlach/mozbase

The main downside compared to autobot seems to be the lack of working
irc notification to irc.mozilla.org, but hopefully that will be fixed at
some point (or maybe we could update face to handle a web service hook
from travis?)

Will