Testing Docker images, Github Releases

[Brent Moran]

Context:

I woke up this morning to find that the current prod docker image was
not able to successfully run via our `install.sh` script. The problem
was that static files aren't being served properly.

Problems:

- The docker image was published to the `latest` tag, but couldn't run
via the `install.sh` script.
- The `master` branch of the repo has (for a while; multiple days)
been such that building a prod docker image using the repo would
result in an image that can't run via the `install.sh` script.

This Proposal aims to solve those problems, and kick off the
discussion of how we want to handle releasing 'production' code and
artifacts.

Tasks for making it possible to test our images:

- Set up a separate `mathesar-git-nightly` repo on docker hub.
- Modify `install.sh` to be able to use that repo, or a local image
rather than `mathesar-prod` when given a specific parameter.
- Establish an understanding that any non-trivial branch being merged
into master must first be built as a docker image, and then smoke
tested by using `install.sh` to run it as if it were the prod image.
- The build of the test image must be in a fresh repo pulled
directly from github, checked out to the appropriate branch, in a
pristine docker environment.
- Once per day, someone (maybe me) should build a completely fresh
pull of the repo in a pristine environment and push the resulting
image to `mathesar-git-nightly`. This image should be pulled and run
for a double-check.
- For now, we could do this manually, but we should make a GH action
for it ASAP (the pristine Docker environment requirement is a pain).

Doing the above should endure that any Docker build produced from the
master branch will work as the `mathesar-prod` image.

Proposed Alpha Release Process

We need a process for releasing GH versions and associated artifacts
in a safe(ish) way. We'll take the above as a given, i.e., we know (or
are relatively sure) that a docker image built from master works.

1. Determine a commit into master that will be the base of the release.
2. Compose the release notes, and get the prose written up.
3. Build the code for that commit into a docker image; push to
`mathesar-git-nightly:latest`, and pull/run it to ensure it works.
4. Tag the same image (on your local machine) for
`mathesar/mathesar-prod:X.Y.Z`, and `mathesar/mathesar-prod:latest`,
and push that.
5. Publish the release on Github.

We should also figure out if we want any human bureaucracy involved,
and who those humans should be.

[Dominykas Mostauskis]

A few thoughts:

- There's a complication that you can elucidate for me hopefully: the service layer image can depend on a specific version of our caddy image and maybe other images. Do we need to take additional steps to account for this? The naive way to look at this is that if we want a certain caddy image to be used with a certain service layer image, we just tag both as `latest`, and Watchtower will update them to that. But what if there's a reason to run a non-latest version of the service layer? It might require a non-latest caddy image. Would you then have the user edit their `docker-compose.yml` by hand to use specific versions of containers, having manually looked up which version of which dependency image is necessary?

- I can volunteer for doing and testing nightly builds; I think it makes sense for you to not do it, since you have more responsibilities already;

- When it comes to nightly builds and testing, to what extent do you think we can automate? Building, pushing, provisioning, and running a few HTTP requests seems trivial.

[Kriti Godey]

We discussed further and decided on next steps at today's meeting. https://wiki.mathesar.org/en/meeting-notes/2023-02/2023-02-20-launch-check-in#testing-docker-images-github-releases