Recipe to upgrade to Python 3.x on Ubuntu 12.04 and 14.04
Mike Dewhirst
Python 2.7 because unicode on 2.7 is doing my head in. This is ugly I
know but that's how it is.
Staging server is Ubuntu 12.04on local hardwarewith Python 2.7 and it
runs Apache2 and mod_wsgi. It serves Django, Subversion, Buildbot, Trac
and Samba (local fileserver for Windows PCs) all of which need to keep
running after the upgrade. I can take them all down temporarily when
required.
Production server is Ubuntu 14.04 with Python 2.7 and runs Apache2 and
mod_wsgi. It is dedicated to serving Django alone. It is a VM in a
private cloud.
At this stage my fallback plan is to build a new staging server on
Ubuntu 16.04 on local hardware and reinstall Subversion, Buildbot, Trac
and Samba. That's a lot of tasks I don't have the time/brainspace to
tackle just yet.
Does anyone have any suggested strategies or advice? The sole objective
is to retire Python 2.7. There are no Django third party app
requirements for Python 2.7. Haven't looked at Buildbot or Trac yet.
Thanks folks
Mike
James Schneider
On Sep 6, 2016 10:44 PM, "Mike Dewhirst" <mi...@dewhirst.com.au> wrote:
>
> I'm looking for a step-by-step system admin recipe to get away from Python 2.7 because unicode on 2.7 is doing my head in. This is ugly I know but that's how it is.
>
> Staging server is Ubuntu 12.04on local hardwarewith Python 2.7 and it runs Apache2 and mod_wsgi. It serves Django, Subversion, Buildbot, Trac and Samba (local fileserver for Windows PCs) all of which need to keep running after the upgrade. I can take them all down temporarily when required.
>
> Production server is Ubuntu 14.04 with Python 2.7 and runs Apache2 and mod_wsgi. It is dedicated to serving Django alone. It is a VM in a private cloud.
>
Argh...different platform OS versions for staging and production!
I suppose that isn't the worst thing as it relates to Python 2.7, since things should stay synced.
Python 3 development is moving much faster, so you do have different 3.X versions available between Ubuntu LTS releases. This is problematic with your current deployment, especially since a few changes in 3.2 broke code compatibility as it relates to Unicode strings, ironically.
The only way to keep the Python versions the same would be to compile a specific version yourself on both boxes (although someone may provide a back port).The real 'gotcha' here is WSGI/Apache. Mod_wsgi has to be compiled against the version of Python that you plan to use. Obviously it can also be compiled in along with python itself, but now we've started down a long unmanageable road of security updates and custom integration. This is why package management was invented; so that I don't have to think about keeping things up to date, and just rely on people smarter than me to provide those updates in an automated process. Well...I still have to tell the system to update, but everything else is automated.
I'm only referring to the base python (and the root pip3) installation here. Pip3 and virtualenv should be utilized for everything else.
> At this stage my fallback plan is to build a new staging server on Ubuntu 16.04 on local hardware and reinstall Subversion, Buildbot, Trac and Samba. That's a lot of tasks I don't have the time/brainspace to tackle just yet.
>
If a solid back port for a common version of python (and mod_wsgi) between staging and prod is not available, I'd go this route, and upgrade everything to the same version.
> Does anyone have any suggested strategies or advice? The sole objective is to retire Python 2.7. There are no Django third party app requirements for Python 2.7. Haven't looked at Buildbot or Trac yet.
>
Getting your internal tools like Trac migrated should be a lower priority than your application code.
Unless there is some whiz-bang feature only available for Py3, you'll probably be on 2.7 for those for a while just because of the complexity integration pieces you mentioned. That may be problematic if the same WSGI processes handle all of these integration apps.
It seems like folks are reasonably happy with Py 3.4, but 3.5 has a bunch of async improvements that Django is or will soon be taking advantage of for Channels.
You may also want to consider building in a configuration manager such as Ansible or Salt stack. Once set up, you can deploy multiple staging and prod servers with a couple commands. With the right planning, that could potentially work across major Ubuntu versions as an upgrade path.
Probably won't save you time up front, but it definitely will next time this comes around, with the added bonus of a near instantaneous recovery of a dev/prod system build with little need for backups beyond your custom app code and the play books for your config management.
Otherwise, I wouldn't push too hard to get off of 2.7 just for the sake of moving to 3.X. Phase it out where it makes sense, or halt development for a week while the infrastructure is upgraded/rebuilt end-to-end.
Backup, backup, backup...
-James
Mike Dewhirst
>
> On Sep 6, 2016 10:44 PM, "Mike Dewhirst" <mi...@dewhirst.com.au
> >
> > I'm looking for a step-by-step system admin recipe to get away from
> Python 2.7 because unicode on 2.7 is doing my head in. This is ugly I
> know but that's how it is.
> >
> > Staging server is Ubuntu 12.04on local hardwarewith Python 2.7 and
> it runs Apache2 and mod_wsgi. It serves Django, Subversion, Buildbot,
> Trac and Samba (local fileserver for Windows PCs) all of which need to
> keep running after the upgrade. I can take them all down temporarily
> when required.
> >
> > Production server is Ubuntu 14.04 with Python 2.7 and runs Apache2
> and mod_wsgi. It is dedicated to serving Django alone. It is a VM in a
> private cloud.
> >
>
> Argh...different platform OS versions for staging and production!
>
hindsight. By the time we went into production Ubuntu had released 14.04
and we didn't have any pressing incentive to move staging forward. We
still have until mid-next year before support evaporates.
> I suppose that isn't the worst thing as it relates to Python 2.7,
> since things should stay synced.
>
> Python 3 development is moving much faster, so you do have different
> 3.X versions available between Ubuntu LTS releases. This is
> problematic with your current deployment, especially since a few
> changes in 3.2 broke code compatibility as it relates to Unicode
> strings, ironically.
>
any current Ubuntu system. And having done some research on this I was
expecting advice to move in that direction. I would impose 3.5 on all
systems if we go there.
> The only way to keep the Python versions the same would be to compile
> a specific version yourself on both boxes (although someone may
> provide a back port).The real 'gotcha' here is WSGI/Apache. Mod_wsgi
> has to be compiled against the version of Python that you plan to use.
> Obviously it can also be compiled in along with python itself, but now
> we've started down a long unmanageable road of security updates and
> custom integration. This is why package management was invented; so
> that I don't have to think about keeping things up to date, and just
> rely on people smarter than me to provide those updates in an
> automated process. Well...I still have to tell the system to update,
> but everything else is automated.
>
bits and pieces but it isn't something I have deliberately done before.
I have totally relied on package management to date. Again, honesty
compels me to say it is daunting.
> I'm only referring to the base python (and the root pip3) installation
> here. Pip3 and virtualenv should be utilized for everything else.
>
> > At this stage my fallback plan is to build a new staging server on
> Ubuntu 16.04 on local hardware and reinstall Subversion, Buildbot,
> Trac and Samba. That's a lot of tasks I don't have the time/brainspace
> to tackle just yet.
> >
>
> If a solid back port for a common version of python (and mod_wsgi)
> between staging and prod is not available, I'd go this route, and
> upgrade everything to the same version.
>
OS and infrastructure instead of the Django side of things. Might leave
some services on the old machine and just move the staging server to 16.04.
> > Does anyone have any suggested strategies or advice? The sole
> objective is to retire Python 2.7. There are no Django third party app
> requirements for Python 2.7. Haven't looked at Buildbot or Trac yet.
> >
>
> Getting your internal tools like Trac migrated should be a lower
> priority than your application code.
>
> Unless there is some whiz-bang feature only available for Py3, you'll
> probably be on 2.7 for those for a while just because of the
> complexity integration pieces you mentioned. That may be problematic
> if the same WSGI processes handle all of these integration apps.
>
suffice.
> It seems like folks are reasonably happy with Py 3.4, but 3.5 has a
> bunch of async improvements that Django is or will soon be taking
> advantage of for Channels.
>
development is done with 3.5 and all testing is on 2.7.
> You may also want to consider building in a configuration manager such
> as Ansible or Salt stack. Once set up, you can deploy multiple staging
> and prod servers with a couple commands. With the right planning, that
> could potentially work across major Ubuntu versions as an upgrade path.
>
> Probably won't save you time up front, but it definitely will next
> time this comes around, with the added bonus of a near instantaneous
> recovery of a dev/prod system build with little need for backups
> beyond your custom app code and the play books for your config management.
>
> Otherwise, I wouldn't push too hard to get off of 2.7 just for the
> sake of moving to 3.X. Phase it out where it makes sense, or halt
> development for a week while the infrastructure is upgraded/rebuilt
> end-to-end.
>
> Backup, backup, backup...
>
Thank you James. Much appreciated.
Mike
> -James
>
> --
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-users...@googlegroups.com
> <mailto:django-users...@googlegroups.com>.
> To post to this group, send email to django...@googlegroups.com
> <mailto:django...@googlegroups.com>.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CA%2Be%2BciUNRN663pkvuLr_beVtZWQovPrTFx%2Bg8a9FrpMAfYjtUg%40mail.gmail.com
> <https://groups.google.com/d/msgid/django-users/CA%2Be%2BciUNRN663pkvuLr_beVtZWQovPrTFx%2Bg8a9FrpMAfYjtUg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.
Erik Cederstrand
> Den 7. sep. 2016 kl. 10.49 skrev James Schneider <jrschn...@gmail.com>:
>
> You may also want to consider building in a configuration manager such as Ansible or Salt stack. Once set up, you can deploy multiple staging and prod servers with a couple commands. With the right planning, that could potentially work across major Ubuntu versions as an upgrade path.
>
> Probably won't save you time up front, but it definitely will next time this comes around, with the added bonus of a near instantaneous recovery of a dev/prod system build with little need for backups beyond your custom app code and the play books for your config management.
Erik
Mike Dewhirst
install either. I have a single bash script called install.host and
that's where I put all such OS edits. I run the script to make the
install and then comment out that line afterwards. It also does the
security upgrades from the Ubuntu repos. I then take that script to a
new (next) machine, uncomment as appropriate and let it rip.
That has worked for me so far but now I want to use Python 3.5 and
probably virtualenv (which I have previously only used on dev machines)
and Apache2.4 (previously Apache 2.2) and mod_wsgi etc etc.
Bottom line is I think I have to invest a bit of effort in a new Ubuntu
16.04 machine. But I would prefer a lovely easy recipe to bring Python
3.5 to both my existing servers being 12.04 staging and 14.04 production.
Thanks Erik
Mike
>
> Erik
>