Proposed contribution to ease Arches installation

[Shea Newton]

Hi, Arches developers. 

I'm posting with a proposed contribution that has the potential to ease Arches installation (e.g. for local development instances) on the "big three" platforms: Linux, Windows, and MacOS.

The purpose of all this is to gauge any community interest. If there is some, we'd like to work with you to make the proposal below a reality, committing to a definite support lifetime so Arches users will be on safe ground relying on it.

Basically, we (ActiveState) would like to provide pre-assembled, downloadable runtimes for Arches for all the major operating systems and distributions. These runtimes would include all of the Arches dependencies, including Python2 itself for long-term v4 release support, and Python3 for v5. Our runtimes would be fully open source -- that is, they'd just include the underlying licensing of Arches and its dependencies and we'd update them regularly to keep up with Arches development and with security vulnerability fixes for underlying dependencies.

If this sounds like it might be useful, read on.

The bigger picture is that the ActiveState Team is starting an initiative to provide redistributable runtimes for open source applications that face portability or environmental compatibility issues -- such as Python2 lifetime issues, Windows deployment dependencies, etc.

ActiveState has a long history of providing runtime environments. Our best-known product is probably still Perl for Windows, which we've distributed for over 20 years. Relatedly, a lot of our enterprise business is in making application-specific runtimes: e.g., stock Perl/Python/etc + all your application's custom dependencies, built and compatibility-tested for Windows and other platforms.

 

This new initiative expands on that. We're trying to help key open source projects that have particularly complex dependencies and/or upstream expiration issues. By taking care of the boring part of getting Arches up and running -- and keeping it secure -- we can free users, developers, and support vendors focusing on the things that matter, like configuring data models, features, contributing code, customizing look-and-feel, etc.

How it would work:

The runtimes would be assembled by ActiveState Platform, the same build infrastructure we use for all our products. The downloads would then be available from a standard place on ActiveState Platform, accessible manually and via the API. The downloads page at https://www.archesproject.org/downloads/ can link to them there. They can also be copied and hosted directly at archesproject.org -- we don't want to supplant existing download locations or namespaces, of course, the runtimes are freely redistributable.

We've looked at the Arches code and at the dependencies list at https://arches.readthedocs.io/en/stable/requirements-and-dependencies/ and we are confident about the technical side of making these runtimes.

We've also read the "Guidelines for Commercial Entities" at https://www.archesproject.org/code-of-conduct/, and agree with everything there. We believe our proposal is fully in accordance with those guidelines, but we welcome any feedback of course.

We’d love to hear your thoughts on this thread. Replying with a simple "Sounds great!" (or "No, our use of Arches wouldn't benefit from this") is fine, but if you have specific ideas or questions about things you'd like to be sure the runtimes address, we want to hear them. If you do have a confidential question feel free to email me directly: shean {AT} activestate.com.

tldr; We’d love to know what you’d need to be happy citing us in your ‘Software Dependencies’ documentation as an alternative to otherwise manually configuring and installing Arches dependencies :) 

Regards,

Shea Newton

Developer Advocate

ActiveState

[Adam Cox]

Hi Shea,

As a community member and service provider (though somewhat on hiatus at the moment) I think this sounds like a good addition to the project. The installation process has always been a bit onerous, but certain advancements here and there, especially work that folks have done with Docker, have been helpful to many people. It seems like ActiveState would be another great addition to the deployment options. You were clear about this, but to reiterate: ActiveState would be presented as an additional option, the existence of which would not prevent other distributions from being created in the future. It occurs to me even now that simply better presentation and documentation of the Docker installation would also be beneficial.

I would be happy to help if I can. You may want to wait a few months until Arches 5 is released (others on this forum are more involved in that process than I) because it will include a few large dependency upgrades: Python 3, Elasticsearch 7, etc. So, stay tuned for that (I don't believe those new technologies are in our "latest" docs yet). One other note: This ticket in the docs repo gives some more recent information about how elasticsearch works with Arches. To sum it up, Arches no longer requires tweaks to the ES config file, and will work fine with the default ES configuration (this should make your work easier).

Adam

[Dennis Wuthrich]

Hi Shea,

My quick take is that your proposed contribution sounds like a good idea.  I agree with Adam Cox that Arches can be challenging to deploy, and having an option that streamlines the process would benefit the community.  One thing to be aware of: we're doing a lot of development on Arches (with a move to Python 3.7, Django 2.2, and Postgres 11 currently underway) so you may want to decide if you'll be supporting multiple versions of Arches.

As for your tldr; question: I think the community could update Arches' software dependencies section to include ActiveState as an alternative once you've decided that you'll be offering runtimes for Arches and community members have had a chance to test drive them.

In any case, thanks for reaching out to the community and educating us on the possibility of using ActiveState to deploy Arches.  I really appreciate your helpful and respectful post.

Cheers,

Dennis

[Shea Newton]

Hey Adam,

Thank you for your feedback, and for the Elasticsearch pointer :)

I'm happy to hear you're amenable to our proposal. And happy that you reiterated your condition above, it bore repeating. We're on the same page. 

I'll look forward to the chance to reach out with more questions or if I run into issues moving forward. 

Shea

[Shea Newton]

Hi Dennis, 

Thanks for your thoughts, I'm glad to hear them. And glad you're also amenable to our proposal.

What I'd like to do now is to start working to get a runtime together for Arches v4. We'd like to support multiple versions of Arches, and hopefully anything I learn along the way can help guide me when it comes to the forthcoming dependency updates you and Adam have mentioned. 

When we're in a place to give community members a chance to test drive a runtime or two, do you have a preference on how I proceed? I'd be happy to continue the conversation here. I can also see some potential in creating an issue or PR on the Arches repo but I'll defer that decision to you and the other members here. 

Shea 

[Shea Newton]

Hello Arches Community!

It's been awhile since our last conversations and I thought I'd provide an update on where ActiveState's at regarding something helpful y'all can test drive. We're nearing a position to provide a portion of this project's dependencies pre-built (the items in the requirements.txt and requirements_dex.txt) but we'd like to get the out-of-box experience totally right and we're still figuring out some of those detains. The out-of-box experience I'm thinking of includes Postgres, GIS, Elastisearch, and OpenJDK (which comes along with the Elastisearch dependency). Those bits have inspired some rethinking and infrastructure changes that we hope will also provide value to others moving forward. 

Thank you for your awesome work, for your patience, and congrats on an awesome 5.0 release!!

- Shea