Testing

[Didrik]

Hello fellow Alavetelians. Here in Sweden at handlingar.se we are currently trying to structure our workflow and development pipeline. If you have any tests you'd like to share or any recommendations regarding testing, hosting and development pipelines please let us know.

Best regards,

Didrik Gauffin

Open Knowledge Sweden

[Roz Townsend]

Bump. We just had a conversation about this again yesterday during our DevOps meeting at Handlingar. We were specifically talking about testing, but any insights or recommendations would be helpful.

Thanks,

-Roz

[Gareth Rees]

We use GitHub Actions to run the tests for the core Alaveteli codebase [1] and WhatDoTheyKnow's theme [2]. Interestingly 37signals (makers of Rails) have recently moved CI to local development machines [3]. I wouldn't necessarily recommend that yet as Alaveteli's test suite isn't as fine tuned as theirs is likely to be. Practically speaking, we write some code locally (Graeme uses a Nix environment, I use Docker) and run tests related to the code change. For example, if I'm working on the User model I might run the user model tests, user controller tests, etc, but not tests for Announcement. Its a judgement call. Then the full test suite is run once a pull request is opened.

Do you have more specific questions or decisions you're trying to make?

In terms of hosting/deployment, we run VMs on big bare metal servers. We deploy WDTK on develop – the bleeding edge, so to speak – so once code is merged, someone runs our deployment command to update the site. We try to deploy things fairly soon after they're merged so we don't have a big backlog of things to release in one go. That way, if something breaks we have fewer changes to assess. For the Alavetelis we host, we deploy them off stable release tags.

Its worth keeping an eye out at what the wider Rails community is doing. They're making big improvements in deployment with default setups for Docker [4], Kamal [5] and Thruster [6]. Can't be sure how compatible Alaveteli will be for this in the immediate future, , but our long term view is to get as closely aligned to the default Rails environment as we can.

Best,

Gareth

[1] https://github.com/mysociety/alaveteli/blob/develop/.github/workflows/ci.yml

[2] https://github.com/mysociety/whatdotheyknow-theme/blob/master/.github/workflows/ci.yml
[3] https://world.hey.com/dhh/we-re-moving-continuous-integration-back-to-developer-machines-3ac6c611

[4] https://github.com/rails/rails/pull/46762

[5] https://github.com/rails/rails/pull/51798

[6] https://github.com/rails/rails/pull/51793

[Laurent Savaëte]

At MaDada.fr, we are pretty poor with tests. We have added a few in our theme, mostly to validate some of the tricky parts of our customisations (appeals and such). For the rest, I trust alaveteli to work. We use tagged versions of it, but deploy our code on each commit to specific branches (dev deploys to staging, master to our production server). We use stock alaveteli and do not fork it, instead we patch the desired code through our theme.

Our deployment is done through ansible from gitlab CI, so there is no reliance on any particular dev machine setup. We deploy to a staging server then to production. The first one sort of serves as a testing ground, but obviously does not replace a proper automated test suite, there is definitely room for improvement there.

I develop on a VM on my laptop that is built through the same ansible script, so in theory I have a full replica of production on my machine, and test with a mix of manual runs and automated tests. This is something I would like to improve. Version upgrades of alaveteli are always a bit tricky.

Servers are 2 VPS boxes provided by gandi, nothing special about them.

I've looked a bit at Nix for deploying the system, still getting familiar with it, but I'm very impressed with what it can do. If I were starting from scratch, I would go with nix instead of ansible, for the additional reliability and speed. It cannot get stuck half way like ansible, leaving you with a broken server. I would also do that over a mess of docker containers which add complexity we don't really need at our scale.

Overall, it works well, continuous deployment means we catch problems quickly when they appear. As deployment is essentially free (git push), there is no hesitation in doing it as long as the code we push won't knowingly break anything.

Laurent

--
You received this message because you are subscribed to the Google Groups "Alaveteli Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to alaveteli-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/alaveteli-dev/8df077e8-e465-40fb-b856-1023e311cb54n%40googlegroups.com.

[Oliver Lineham]

Quite a late reply, but I thought I'd describe our hosting setup since we are quite different.

We have run fully containerised since 2015 (not using the Dockerfile/scripts from Alaveteli, or the install scripts). Everything is in separate containers. We have also run Sidekiq+Redis since 2015 for periodic jobs - not cron. Unicorn instead of Passenger.

Postfix writes inbound mail to an S3 bucket. A Sidekiq job reads it from S3, ingests it into Alaveteli, and moves the raw email to another bucket for safekeeping. Decoupling inbound mail reception from the processing has proved quite useful for continuing to receive mail while Alaveteli is down, and to be able to move an erroring raw email out of the way temporarily or reprocess it when ready without dropping any mail.

In terms of testing, I'm happy to leave Rails unit tests to Alaveteli developers who know Rails better than me. I'm more concerned with full system integration tests which is something I need to work on. I can at least stand up a completely new suite of containers for a test environment identical to prod in just a few seconds, which is nice.

All that said, it's a fork and not maintained. I am currently working on a full re-containerisation which is less challenging to keep current with Alaveteli releases. Puma not Unicorn. Still no cron jobs or alert daemon, instead sidekiq-cron and ActiveJobs. These start in milliseconds (as Rails is already loaded in Sidekiq), instead of the 10 seconds start up time of a rake task which has to load Rails every time.

If this is something people are interested in I'll be happy to share the work soon when it's a bit more presentable, as time allows.

Oliver

[Laurent Savaëte]

Hi Oliver,

You might want to reach out to the Swedish team at handlingar.se as I believe they are working on a similar containerised setup (I don't have details). Ping me in private if you don't have their contact, I'm not sure they are following this group.

Laurent

To view this discussion on the web visit https://groups.google.com/d/msgid/alaveteli-dev/CAOPSvnAmKZAZei2Z%2B%2B0%3D4VX2qtScTsYt6ggpo%3DCa13MYp0rSUQ%40mail.gmail.com.