Desktop Mathesar

[Sean Colsen]

I saw this post come through via Syften and wanted to highlight it.

This person has relational data and wants a way to store and manage it locally without installing anything complex. I wish I could recommend Mathesar for this, but I don’t think we’re actually a good fit.

In my opinion this use case is quite common. I want a product like this! And as I’ve been talking to other people about Mathesar over the past year and a half, my sense is that there is a huge gap in the market here.

From a feature standpoint, Mathesar is quite close to supporting this person’s use case. We’d just need to improve the editing experience for large text fields. That’s easy work. But from an installation standpoint, I feel like we’re quite far from being a viable suggestion for this person. Mathesar’s architecture makes it difficult to install. This person wants a desktop app, not a web app.

I’ve brought up this issue before, and I understand we’re not focusing on this direction for our product currently. I just wanted to highlight it again because I think this reddit post articulates the use case quite nicely.

I’ve been thinking more about desktop Mathesar and I’m a little curious how we would build a desktop app if we wanted to, hypothetically speaking. I don’t mean to distract people. But if anyone wants to chime in with a quick thought about the direction you might head, I’d be very interested to hear it. Specifically, I wonder if anyone might actually see a somewhat easy path towards building a desktop version of Mathesar.

There are plenty of tools that would make the front end portion of a desktop app relatively easy. For example Tauri and Wails both produce single file executable applications that utilize the system webview. I think these are really neat! Imagine being able to tell our users to just download and run Mathesar! But neither of them support Python backends. If our backend were in TypeScript, Rust, or Go, we could potentially head that direction without much friction. Is there a tool like that for Python though? I see pywebview, but my sense is that it still requires the user’s OS to have a python version compatible with the application. That makes me wary, given the difficulty I recently had in attempting to install Mathesar outside of Docker. Sorting out python 3.9 vs python 3.10 was a major headache. But maybe there would be an easy path heading that direction somehow.

To be clear: I’m not suggesting we pursue desktop Mathesar any time soon. I’m mostly just tossing this thought out there for others to keep in mind, especially as we consider back end architectural changes. And I’m also inviting ideas from others about any easy paths that you think might be present.

​

[Dominykas Mostauskis]

Can we just have a local HTTP server and have the "desktop frontend" connect to that? And maybe have the "desktop frontend" be able to connect to a remote backend too.

[Kriti Godey]

That's my thought too.

[Sean Colsen]

Dom and Kriti,

I’m a little confused. I don’t want to delve into a lengthy discussion for such a tangential topic, but I’m curious to better understand your thinking and see if I’m missing something.

When you say “local HTTP server”, I picture a Django app which serves our REST API. Is that what you’re talking about? If so, installing it would basically be as complex and technically demanding as installing Mathesar currently. Right? Or am I missing something? This sort of “desktop frontend” offers very little value in my opinion. It might give users a pretty icon in their window switcher, but it doesn’t seem like it would help the user in the post I linked to because it would still require technical expertise to install/manage/upgrade.

What I’m after here is a way to let non-technical people run Mathesar without relying on any technical people. A full desktop app does that pretty well because users can install it easily, open it easily (without needing to launch a back end HTTP server), close it easily, and upgrade it easily (e.g. Tauri’s built in updater).

With all the effort we’re putting into making Mathesar usable for non-technical people, I wish non-technical people could use it without relying on any technical people to manage it. I think that would do wonders for adoption!

​

[Kriti Godey]

If so, installing it would basically be as complex and technically demanding as installing Mathesar currently. Right?

No, I was thinking about a full-fledged desktop app that would set up a local HTTP server behind the scenes, but from the user's perspective, it would function just like any other desktop app. No installation instructions needed – just download and install just like any other app. But when you opened the app, it could launch a local HTTP server that runs our backend.

I don't know enough about this to suggest a particular technology, but it seems like this approach would involve the least amount of work.

[Sean Colsen]

Kriti,

a full-fledged desktop app that would set up a local HTTP server behind the scenes

Ok, this makes more sense. Thanks for clarifying.

Within the above description, I can imagine two different approaches:

• (A) A desktop app to manage a local Mathesar web server. It has some UI to start/stop the server. Maybe also some UI to add/remove PostgreSQL servers, akin to our MATHESAR_DATABASES env variable. Then the user opens a web browser to actually use Mathesar.

• (B) A desktop app which provides a fully self-contained UI for the entire Mathesar application. No need to use a separate web browser application. The user never knows anything about an HTTP server. The architecture within the app includes an HTTP server (backend) and a webview rendering layer (frontend).

Kriti, when you say “but from the user’s perspective, it would function just like any other desktop app,” I gather you’re picturing (B). Is that right? Approach (B) is exactly what Tauri and Wails are designed to do which is why it would be relatively easy to implement if our backend were already in TypeScript, Rust, or Go. I could paraphrase my original message in this thread by saying: I’ve seen some tech that would make path (B) easy if our stack were different. Has anyone come across similar tech that would make path (B) easy with our current stack?

To be clear, I’ve not used either Tauri or Wails, but I’ve been following Tauri closely for the past couple years with great excitement. Compare its stars to Wails. Lots of other people excited too! I have used Capacitor which works in a similar manner, albeit for mobile apps only (with JS backends), and it was super easy. (Tauri plans to support mobile apps soon too.)

The problem is that neither Tauri nor Wails support server-side logic in Python. Tauri says Python binding is on its roadmap, but the last time I looked into it, I got the impression that it was a long way off. I know that the PyO3 library lets you call Python code from Rust, so there could be a path towards preserving some of our Python business logic while calling it from a thin layer written in Rust, but I doubt we’d be able to just drop a Django app in there and have it work. I’d be keen to experiment with this stuff at some point though!

​

[Kriti Godey]

Here's a project we could experiment with at some point: https://github.com/pyinstaller/pyinstaller (supports Django). There's also https://www.py2exe.org/. I only looked for 2 min, but I think we can find options when we get around to working on this.

[Brent Moran]

As a user, the server-client app installation that I've found most 'desktop app' like is Syncthing, and it's not even close. I've installed it on MacOS and multiple linuxes with nary a hiccup. They let you use their original web UI through the browser or (if you install it) an Electron front end for a more desktop app feel. Users who follow the main path for installation don't even know about those details, though. They just double-click their way to success. Starting and updating Syncthing also feel 'native' wherever I've used it. As noted by Sean, that's enabled by a pretty different stack than ours (the back end is in golang).

To be honest, the server that we'd have the most trouble changing in the back end is the PostgreSQL server (given our current feature set). We can't easily change that to, e.g., SQLite (at least for storing user data), since they lack a bunch of features that we'd have to re-create (e.g., most types, and enforcing types on columns). And extending SQLite is more technical and requires more access to the underlying machine than extending PostgreSQL (or indeed any server-based RDBMS). *If* someone's familiar with a different DB solution that has (most of) the features we need and doesn't require running a server, I'd be interested in hearing about it. I'm not aware of one, though. Thus, even if we exchange Django for golang, the problem is still not solved because getting PostgreSQL installed on various platforms is still always going to feel less 'native'. I was daydreaming the other day about trying to fork SQLite to make PostgreSQLite (I.e., a serverless RDBMS with feature parity with Postgres). That would be a major endeavor of its own, though. I'm not aware of any desktop apps that have a secret PostgreSQL server running in the background, but I could do some research about it. Maybe we'd learn something there.

@Sean Colsen would it be accurate to say that the main pains you want to solve with a more 'desktop-app-like' architecture are installation, setup, and upgrading? If so, I think we may have more success more quickly by reworking versions of our installation, upgrading and setup flows that are specifically tailored to a user who wants to install Mathesar on the same machine they're physically using to use Mathesar. I have many thoughts about how to improve that, but most are already known to the group (and this email is getting too long already).

[Sean Colsen]

Thanks for this email, Brent. You’ve given me some more to ponder. Good point about the Postgres server being an even tighter constraint here. This is one reason why I’d love for us to support SQLite (but that’s a whole other topic).

I’m not aware of any desktop apps that have a secret PostgreSQL server running in the background

Right. Me neither. There is postgres-wasm but that’s still pretty far from what we’d need, and it’s crazy to read about the technical hurdles they had to overcome just to build that.

the main pains you want to solve with a more ‘desktop-app-like’ architecture are installation, setup, and upgrading?

Basically yes. But I’ll expand on that a bit. The higher-level goal I’m suggesting is that non-technical people should be able to use Mathesar without relying on any technical people. Towards the end, a non-technical user would need to be able to install, launch, close, upgrade, and uninstall Mathesar without ever messing with the terminal. They’d also need to be able to backup and restore their Mathesar data easily. Perhaps I could sum it all up by saying: “non-technical people should be able to administer Mathesar.” A desktop app which reads/writes a SQLite file is an example of a type of app which would meet all those requirements.

I think we may have more success more quickly by reworking versions of our installation, upgrading and setup flows

I have a hard time picturing improvements to our flows that would make Mathesar easy enough to non-technical people to administer. Part of what’s tricky here is that we kind of need all the pieces to be in place, like links in a chain, in order for the whole thing to work. If people can install Mathesar but can’t figure out how to start it, then they’re out of luck. If we could figure this out though (even with our current architecture), I think it would be huge.

​

[Kriti Godey]

One of the things on our roadmap is a hosted version of Mathesar that people can sign up for and use. This would be one way for non-technical users to use Mathesar without any installation needed.

I don't think this would be a replacement for a desktop version, a desktop version would have many advantages (works offline, no need to share data with a third party, etc.) but a hosted version would help non-technical users use Mathesar easily.