Arches moving to Python 3

[Alexei Peters]

The Arches Team is excited to annouce several technology improvements that are coming to Arches v5 (to be released in January 2020) and are now live on the master branch.

The main update is that Arches is now compatible with Python 3.7.  The version of Python that Arches currently runs on (2.7) will be retired at the end of the year (https://pythonclock.org/) and the latest version of Arches will not be able to run on version 2.7 any more. 

While we were at it we also upgraded django to version 2.2, Elasticsearch to 7.4, Postgres to 11.5, and finally Postgis to 2.5

We've also eliminated TileStache as a dependency and now power our .mvt tile data directly out of Postgres/Postgis.

These are all much needed updates and guarentee that Arches will live on a solid foundation for the foreseeable future.  These updates also bring with them some performance improvements as well!

What this means, though, is that the latest on master won't work in your current virtual environments.  You will need to create a new virtual environment based on Python 3.7.  You'll also need to install Postgres 11.5 and Elasticsearch 7.4

Once those dependencies are installed and running and your new virtual environment is set up you should be able to point to master in your Arches clone and run Arches as usual.  Any packages will have to be reloaded to ensure that data is properly loaded into the database and Elasticsearch.

Keep in mind that any packages/custom components that you happen to be developing or have developed won't work with the latest until also made python 3 compatible.

Finally the team is currenlty hard at work implmenting Celery (http://www.celeryproject.org/) as a Task manager that Arches will be able to leverage for such long running tasks as import/export and to support the mobile application.

We're super excited about this long awaited transition and hope you are too!

The Arches Team.

[Adam Cox]

Thanks for the update Alexei! All of this sounds like great progress.

Adam

--
You received this message because you are subscribed to the Google Groups "Arches Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to arches-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/arches-dev/c04bd56e-970a-4b76-9e51-15db4e55d3b8%40googlegroups.com.

[Survey Wessex]

Hi Alexei,

I installed an Ubuntu 18.04 development server based on the updated dependencies: Python 3.7, Django 2.2, Elasticsearch 7.4, Postgres 11.5 and Postgis 2.5.

Two observations:

1. Elasticsearch is running as a system service (systemctl) and not as a daemon from a dedicated folder in /Projects/Elasticsearch7.4/. Is that correct?

2. Assuming 1. is correct, how do I treat the wsgi daemon process in the virtual host file when serving arches through apache?

Just changing 'python2.7' to 'python3.7' doesn't seem to work:

WSGIDaemonProcess sslarc2 python-path=/home/ubuntu/Projects/arc2:/home/ubuntu/Projects/ENV/lib/python3.7/site-packages
WSGIScriptAlias / /home/ubuntu/Projects/arc2/arc2/wsgi.py process-group=sslarc2

At the moment I can't serve Arches through apache. Any ideas how to fix this?

Jens

[Ryan Anderson]

Hi Jens,

Glad you are trying to host Arches v5 with Apache. 

I’m not sure about #1, but for #2 have you upgraded your apache-modwsgi module? I think you need to install libapache2-mod-wsgi-py3.

We’ve also run into a few errors hosting with apache using python3.7, if you still can’t get it to work you might try creating a new virtual env with python 3.6.

Good luck.

Ryan

--
You received this message because you are subscribed to the Google Groups "Arches Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to arches-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/arches-dev/0f8cd311-9d9a-4660-b5d1-197eb8f5a0fa%40googlegroups.com.

[Alexei Peters]

Hi Jens,

I'm happy to hear that you've successfully upgraded all of your dependencies.

Unfortunately, I'm not super familiar with setting up Arches for Apache. I'll let my other colleagues speak to the WSGI issues that you are having.

I do think that #1 is correct though.  Elasticsearch is proxied through Arches and as long as the port that ES typically uses (9200) is visible from Arches then this should work.

Cheers,

Alexei

Director of Web Development - Farallon Geographics, Inc. - 971.227.3173

--
You received this message because you are subscribed to a topic in the Google Groups "Arches Development" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/arches-dev/y4hyhyUyRvc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to arches-dev+...@googlegroups.com.

[Survey Wessex]

Hi Ryan,

I installed libapache2-mod-wsgi-py3 and did a2dismod/enmod.

Still getting errors like:

(from apache error log)

File "/home/ubuntu/Projects/ENV/lib/python3.7/site-packages/django/contrib/gis/db/backends/postgis/base.py", line 2, in <module> from django.db.backends.postgresql.base import (
File "/home/ubuntu/Projects/ENV/lib/python3.7/site-packages/django/db/backends/postgresql/base.py", line 24, in <module>
raise ImproperlyConfigured("Error loading psycopg2 module: %s" % e)
django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 module: No module named 'psycopg2._psycopg'

Does that match your observations?

I'll try Python 3.6 tomorrow.

Jens

[Survey Wessex]

Hi Ryan,

 

eventually I managed to serve Arches 5 through Apache!

I took your advice and used python3.6 and created a new virtual env.

In addition to that it is also necessary to change the syntax for the wsgi-mod daemon process in the virtual host file:

based on

https://django.readthedocs.io/en/2.2.x/howto/deployment/wsgi/modwsgi.html

and

https://www.digitalocean.com/community/tutorials/how-to-serve-django-applications-with-apache-and-mod_wsgi-on-debian-8

it should be:

 WSGIDaemonProcess wessex python-home=/home/ubuntu/Projects/ENV python-path=/home/ubuntu/Projects/wessex
 WSGIProcessGroup wessex
 WSGIScriptAlias / /home/ubuntu/Projects/wessex/wessex/wsgi.py

The syntax previously used for python 2.7 does not work any more.

Jens

 

UPDATE

Just realised you guys released v5.0.0 today. Well done, great job!

You might want to update the documentation including the 'Further Reference' box based on the above

https://arches.readthedocs.io/en/stable/serving-arches-with-apache/