Kubernetes Support

[Phil Weir]

Hello all,

 

We have been trialling the Arches 5.0 docker setup with Kubernetes Helm charts, and made decent progress. Is anybody else actively looking at this themselves, and/or interested in collaborating on this?

At present, we are focusing on static Helm charts, but would be working towards an Arches Kubernetes Operator that could go on https://operatorhub.io/ , to simplify provisioning and maintenance for Kubernetes-focused teams, such as ourselves.

Keen to avoid any duplication, and to build something that we can keep aligned with the overall project direction, so will keep testing and post back with repos - it would be helpful to gauge interest too, so if there is work in progress elsewhere, or this just piques curiosity, please shout out :)

All the best

Phil Weir

This email was sent from Flax & Teal Limited, a company registered in NI, with number NI617545

at Farset Labs, Weavers Court, Linfield Industrial Estate, Linfield Road, Belfast, BT12 5GH

Flax & Teal is a member of the Avata Industries business syndicate

[Phil Weir]

On Thursday, July 9, 2020 at 8:36:45 PM UTC+1 Phil Weir wrote:

We have been trialling the Arches 5.0 docker setup with Kubernetes Helm charts, and made decent progress. Is anybody else actively looking at this themselves, and/or interested in collaborating on this?

At present, we are focusing on static Helm charts, but would be working towards an Arches Kubernetes Operator that could go on https://operatorhub.io/ , to simplify provisioning and maintenance for Kubernetes-focused teams, such as ourselves.

<snip>

 

To follow up on this and the discussion from the general mailing list: a few thoughts on design considerations for Kubernetes deployment of Arches:

Questions to be answered:

      - the Python project code-base needs to be pre-built into a per-Arches-instance docker image - could this be a standard, e.g., Gitlab CI file for doing this (which users can modify for other CI)? This would mean that when a project repo is created it'll automatically build a usable image.

      - how do we best handle static assets? (we have a current approach that broadly seems to work for this, but optionally pushing to a CDN would be nicer)

      - is any state (e.g. session state) managed outside of the obvious locations - postgres and couchdb? (the Arches 5 docker-compose looks like that's a no)

      - what maintenance tasks/jobs need run periodically - either manually or automatically? What one-off jobs/commands should be accommodated?

      - how should back-ups be approached? Should this be a Kubernetes design point, or is this overextending scope, provided clear direction is provided?

      - what default storage types, sizes make sense? E.g. should we default to S3/minio/Blob storage for uploads?

      - initialization - what options should be at the infrastructure level, how do we differentiate essential first-run steps (for any deployment) from optional set-up that may need to be customized

      - what are the essential-to-parameterize aspects for quick set-up e.g. initial username

      - how are upgrades to the underlying project or to the Helm chart handled?

None of those are blockers to getting a rough testable base - I've got one running, using out-of-the-box Arches 5, on our internal test cluster (prototype code: https://github.com/flaxandteal/helm-arches - still a couple fixes needed to be properly testable, and a redeploy to check it still works here)

In terms of usability and uptake, I would suggest tidying up that reference chart, so it is as simple and quick to pick up and play with as possible for someone basically familiar with Kubernetes, even it doesn't describe a full recipe for a production-ready system. At the same time, we can then grow a parallel fork with any the relevant features for enterprise-level deployment as we progressively incorporate them - e.g. linking with common Kubernetes plugins for additional network security, scalable Kubernetes-side storage, third-party integration, Kubernetes level access controls and network/process isolation, for instance. Charts that I've seen attempt both in one codebase can be quite hard to get to grips with and try out locally or on a dev cluster.

[Phil Weir]

Happy almost New Year! Nothing too exciting to mention, but a brief update on the Kubernetes Helm charts - we have linked the unofficial alpha version of this on artifacthub.io, which is now the Helm team's centrepoint for package discovery. There is still a bit to do to make it easy to test, but this is a step closer to being able to launch and scale Arches instances onto an existing k8s (kubernetes) cluster in a few commands. To be continued... :)

All the best,

Phil

--
You received this message because you are subscribed to the Google Groups "Arches Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to arches-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/arches-dev/eb1a02e1-e0dc-401d-8069-48f95dba7a36n%40googlegroups.com.