Build from scratch specificity

[Brent Moran]

Hello,

This emails serves to reopen a discussion on our stance w.r.t. the specificity of the build from scratch instructions.

My opinion is that over-specifying for instructions like that (for example, instructing the admin about how to install a specific version of Python) is a mistake. Some reasons are:

• It won't be portable to any but a very small set of systems (even between version of the same distro)
•  In some distros (mine) the instructions can change even without updating the distro version
• It puts us in the position of maintaining things we have no control over (links to python tarballs, for example)
• For the intended audience of that installation method, over-specifying hinders their understanding of the actual requirements. For example, having `postgresql` hidden in a big `apt install` blob obscures what version they'll actually get, and also which versions are compatible.
• The intended audience of this method probably has specific requirements (e.g., they have a PostgreSQL server already running on their local machine). Any overspecification on our part then gives them questions about why we specified this-or-that command and whether the results will conflict with their setup (for Postgres, do we rely on the Debian  pg_ctlcluster parts? It's hard to tell, but if we're specifying how you install PG on Debian, a cautious admin may assume we do)

My opinion is that the central difficulty experienced by Kriti during the build from scratch QA was due to 2 big problems (aside from a small bug that wouldn't be blocking, and an omission from the example environment variables, which are updated in the docs PR):

• The difficulty of getting a specific version of Python running on a given machine, and getting a virtual environment working with that version (this was most of the problem)
• Distraction from over-specification of how to install things. All the links to various pieces proved to be red herrings, making things more rather than less difficult. Using the standard package manager on Debian would have sufficed for all dependencies except python 3.9 (or 3.10).

I think if we make any changes to the docs, it should be to reduce the overspecification, and emphasize the absolutely crucial nature of making sure you are running the correct version of Python.

Brent

[Pavish Kumar Ramani Gopal]

I'm in agreement with Brent on not over-specifying instructions for all the reasons mentioned in his mail.

I would like to suggest a couple improvements, which would make this installation method way easier. We don't necessarily have to do them as part of this release.

• Remove caddy from this installation method.
• Since this is an advanced type of installation, I think it should purely focus on Mathesar.
• We can have an additional section which recommends having a reverse proxy and provides caddy as an example.
• Build static frontend files, host them somewhere, and download them during this installation process, instead of having to ask the user to download Node and npm and compile them

[Sean Colsen]

Build static frontend files

We have this issue for that: Allow people to install Mathesar from source without NodeJS

​

[Kriti Godey]

Quick notes from the meeting today about this topic + some additional notes from me:

From the meeting

• We should have a generic “install from scratch” page for all the reasons Brent described
• We should also have a separate “guide” page for how to set up a cloud Mathesar server with more precise instructions for a specific platform (e.g. starting with “set up a Debian 12 VM instance on GCP, AWS, Azure”)
• We are not doing any of this for 0.1.4 (which is due to be released tomorrow), we’ll discuss this more and make changes after the release is out.

My notes

• Here’s some installation instructions that we could look at as examples
• Synapse has per-platform instructions on their main install page, this could be an alternative to the "guide" page, but is probably too much work for us right now.
• Wiki.js has a "Ubuntu LTS" set up guide, which is what I followed when setting up our (now defunct) installation. It still uses Docker, and we won't be using Docker, but I think it's still a good example because it specifies an OS.
• I think reducing installation complexity is also key here
  
• I agree with Pavish that we need to prioritize building and publishing static files for each release so we don't have Node / NPM as a dependency in this process
• I also think we really need to prioritize getting Mathesar working with recent Python versions so that we don't have to have the complexity of people having to use non-default Python. This wasn't an issue with the previous Debian stable version, but bookworm (Debian 12) ships with 3.11.