Moving to a consistent, modern Python

[Gregory Szorc]

According to bug 724191 [1], our build infrastructure is currently
running the following Python versions:

* Talos - 2.4
* Tests - 2.5.1
* Builds - 2.7.1
* Buildbot - 2.5.1

I find it frustrating that when making improvements to the build and
test systems that I have to target both old and inconsistent versions of
Python. It is so frustrating that it sometimes makes me reconsider
making improvements to these components because I don't want to deal
with the headache!

AFAICT, there is nothing *technical* holding us back from upgrading to
2.6 or 2.7. The Windows mozilla-build environment now ships with 2.7. OS
X 10.7 ships with 2.7.1 (although 2.6 might be the default), and nearly
every *NIX distro ships with 2.6 and/or 2.7. Needless to say, things
seem to just work on 2.6 and 2.7. Instead, the barrier seems to be human
resources and prioritization. With this post, I'm hoping to raise
awareness and get this on the radar.

There is also the question of what to upgrade to. In my ideal world I'd
say move ALL THE THINGS to 2.7 because that is the best 2.x version
available (I think it's safe to say we don't want to deal with 3.x at
this time). But, I can sympathize with people on 2.6 who may suddenly
experience a breakage because someone checked in something requiring 2.7.

So, when can we modernize the build and testing infrastructure and what
should we consolidate to?

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=724191

Gregory Szorc
g...@mozilla.com

[Boris Zbarsky]

On 2/27/12 8:09 PM, Gregory Szorc wrote:
> AFAICT, there is nothing *technical* holding us back from upgrading to
> 2.6 or 2.7. The Windows mozilla-build environment now ships with 2.7. OS
> X 10.7 ships with 2.7.1

What about 10.6? Note that we can do some things on 10.6 that we cannot
(yet, perhaps) do on 10.7, or anywhere else for that matter, as regards
profiling, so there is strong incentive to be able to compile on 10.6.

-Boris

[Justin Lebar]

You can get Python 2.7 via macports, no? We already require macports
for other things...

Not to disregard the pain of requiring everyone to install python 2.7...

> _______________________________________________
> dev-planning mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-planning

[Boris Zbarsky]

On 2/27/12 8:23 PM, Justin Lebar wrote:
> You can get Python 2.7 via macports, no? We already require macports
> for other things...
>
> Not to disregard the pain of requiring everyone to install python 2.7...

As long as it's available, it's not a huge deal, actually. Upgrading
python via macports is a bit of a pain, but doable.

-Boris

[Jason Duell]

On Feb 27, 5:09 pm, Gregory Szorc <g...@mozilla.com> wrote:
>
> AFAICT, there is nothing *technical* holding us back from upgrading to
> 2.6 or 2.7.

One dumb little thing: our websockets' test server used to require
python 2.5. It's fixed on trunk/aurora/beta, but we still need 2.5
for ESR (the fix could be ported there: it's just test infrastructure,
not part of the build).

Jason

[Dave Mandelin]

On Feb 27, 5:09 pm, Gregory Szorc <g...@mozilla.com> wrote:

> In my ideal world I'd say move ALL THE THINGS to 2.7 because
> that is the best 2.x version available (I think it's safe to say we don't
> want to deal with 3.x at this time).

+1 to that! It sounds like there haven't been any serious objections
yet, so let's do it.

Dave

[Justin Dolske]

Even if we only end up moving "SOME OF THE THINGS! _o/", that still
seems virtuous. Win-win.

Justin

[Clint Talbert]

Do you happen to remember the bug number for this fix?

I had tried to hunt it down when I filed the tracking bug, but was
unable to turn it up.

Thanks,
Clint

[Kyle Huey]

On Tue, Feb 28, 2012 at 6:28 PM, Clint Talbert <ctal...@mozilla.com> wrote:

> Do you happen to remember the bug number for this fix?
>

688768.

- Kyle

[Clint Talbert]

We are. The slowest and scariest part of this is moving Talos, because
it is crucial to our systems, has arcane dependencies, involves updating
1000 test slaves, and isn't something you can easily do piecemeal. That
said, my team and several folks from releng are already working on
migrating it. I'm expecting it to be finished in Q2 of this year.

The correctness test systems - mochitest, reftest, xpcshell etc should
all work out of the box right now with 2.7. If anyone has spare cycles
to verify that assertion and file any bugs they find, please do. Ensure
the bugs you file block 724191 so we can ensure they get addressed.

Thanks,
Clint