On the topic of a Sandstorm Debian/Ubuntu package

[Asheesh Laroia]

Hey all,

Quick question/poll:

For prospective Sandstorm users (or real Sandstorm users) who would prefer a *.deb package, here are some options I can think of:

* You can apt-get install sandstorm; it lives in Debian experimental

* You can apt-get install sandstorm; you download the package by adding a repo to your sources.list that lives at https://apt.sandstorm.io/

* You can apt-get install sandstorm-installer; you get the install.sh and when you run it (as /usr/sbin/sandstorm-install.sh), it downloads the main Sandstorm binary from us; the Debian package lives in Debian experimental.

* You can apt-get install sandstorm-installer; you get the install.sh and when you run it (as /usr/sbin/sandstorm-install.sh), it downloads the main Sandstorm binary from us; you download the package by adding a repo to your sources.list that lives at https://apt.sandstorm.io/

I am strongly considering doing the last one of these because it seems very easy to me. I guess my real question is - is that something that you would use? Would you (dear reader) be willing to use such a thing?

(The first two are harder to implement because I want auto-updates to be handled and I want there to be an easy way to get sandcats stuff too.)

Yours truly,

Asheesh.

[Greg]

Hey Asheesh,

I don’t really care how you do it, what I think matters most is:

1. That it be secure.

2. That it be easy.

To do (1) you need to have a securely pinned public key that’s used to verify downloaded content (that's signed by the private key).

The last option mentioned may not fit the “be secure” requirement, as I’m guessing its security rests on the security of HTTPS (not very good).

Some searching finds tutorials that might be helpful for creating debian package:

https://www.amon.cx/blog/simple-guide-to-debian-packages/

Along with more traditional sources:

https://wiki.debian.org/Packaging

You guys are building a very important platform that has a high probability of taking off and being used across thousands if not millions of servers.

You should bite the bullet and pick the most secure option available to you.

Cheers,

Greg Slepak

--
You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kenton Varda]

Hi Greg,

In fact, our install script and auto-updater both check signatures on the things they download, in addition to using HTTPS. So if you have a verified copy of the installer script, then the rest of the process is "secure" to your standards. :)

FWIW we have written some instructions for bootstrapping trust in the installer script through PGP web of trust:

https://docs.sandstorm.io/en/latest/install/#option-3-pgp-verified-install

Of course, these instructions would be different if you're installing from a deb package using any of Asheesh's options, but I'm not sure there is any "easy" option which achieves the same level of security. Most users will probably rely on HTTPS somewhere in the process.

-Kenton

[Alec Berryman]

Instead of leaving sandstorm in experimental, a common practice is to upload it to unstable and open a bug at "serious" priority that says "don't migrate to testing", which means it will not ever go into the stable release: e.g., blocking rust from going to testing.

If you did have a Debian package, I would expect updates to happen through new packages, not from another mechanism; that's what Google Chrome does.

I would not be surprised if your Debian package sponsor asked the package to depend on the system node and mongo instead of bundling copies.

Packages like Google Chrome are built for stable and unstable; I have not yet run into an issue with Sandstorm where package versions or libc versions mattered, but could imagine it happening.

(I'm a former Debian package maintainer, but haven't done any of that in at least 5 years)

[Kevin Reid]

On Oct 21, 2015, at 0:18, Asheesh Laroia <ash...@sandstorm.io> wrote:

> For prospective Sandstorm users (or real Sandstorm users) who would prefer a *.deb package, here are some options I can think of:

For context: I am not particularly a Debian user. I am a “I want everything I install to be through some sort of package manager” user.

> * You can apt-get install sandstorm; it lives in Debian experimental
>
> * You can apt-get install sandstorm; you download the package by adding a repo to your sources.list that lives at https://apt.sandstorm.io/

Neutral between these two choices. Do the second if it gets me faster updates.

> * You can apt-get install sandstorm-installer; you get the install.sh and when you run it (as /usr/sbin/sandstorm-install.sh), it downloads the main Sandstorm binary from us; the Debian package lives in Debian experimental.
>
> * You can apt-get install sandstorm-installer; you get the install.sh and when you run it (as /usr/sbin/sandstorm-install.sh), it downloads the main Sandstorm binary from us; you download the package by adding a repo to your sources.list that lives at https://apt.sandstorm.io/

This is Wrong™ because now there are (non-user-data) files on my system that are not managed by my package manager. It is not better than the status quo in the way I care about.

I don't necessarily insist that the package manager be apt-get/dpkg. What is important (in priority order) is:

(1) It has an accurate “uninstall” operation.
(2) It is independent of the software being installed: it cannot make a Sandstorm-specific mistake.
(3) If I want to, I can take apart the package and see the effects it will have on my system configuration. The software being installed should refrain from making any configuration changes.

> I am strongly considering doing the last one of these because it seems very easy to me. I guess my real question is - is that something that you would use? Would you (dear reader) be willing to use such a thing?

I would use it over the Infamous Curl Process because there are _additional_ layers of validation, but it doesn't contain the Benefits Of Being Packaged.

--
Kevin Reid <http://switchb.org/kpreid/>

[Jack Singleton]

> * You can apt-get install sandstorm; it lives in Debian experimental

I would personally like to see Sandstorm in Debian stable one day, with security updates coming through Debian security repos and possibly newer versions in backports…

So if this is a step in that direction, it would be my #1 choice.

BUT, I do recognize that is a lot of work. (and agree that separating out updates like that is not what you want to be spending your time on right now)

So with that in mind any of the below are fine by me!

Jack

On Oct 21, 2015, at 12:18 AM, Asheesh Laroia <ash...@sandstorm.io> wrote:

[Kenton Varda]

I think for this conversation to be useful it's important that people first understand how Sandstorm's installer actually works.

Here's an excerpt from something I wrote about this that never got published:

"""

### Sandstorm installs into its own container

The first thing to understand about Sandstorm's installer is that it sets itself up in a container under `/opt/sandstorm` (or wherever you choose). There, it downloads a bundle that contains the entire Sandstorm server and all of its dependencies -- from Javascript to libc.

The main `sandstorm` binary is statically-linked, meaning it does not need any libraries from your system. When you execute it, it sets up and enters a Linux mount namespace (commonly known as a "container", similar to chroot) within which only Sandstorm's own files are visible.

This self-containerization means:

- Sandstorm uses exactly the dependency versions against which it was tested, not the ones present on your system. This means that once Sandstorm has been tested on one machine, we can have a high degree of confidence that no differences between distros will break it. Only hardware and kernel differences matter, and both tend to have highly stable interfaces. This is important, because otherwise testing on every Linux distro would be extremely time-consuming.

- Sandstorm does not need to install anything into your system nor make assumptions about it. You could be running a homebrew distro that uses static libraries and doesn't follow FHS, and Sandstorm would run fine.

- Uninstalling Sandstorm is a matter of deleting `/opt/sandstorm`.

### Sandstorm automatically updates

Sandstorm knows how to update itself. If you allow it, Sandstorm will do so automatically every 24 hours, otherwise you can invoke it manually by running `sandstorm update`.

Updates are downloaded over HTTPS and additionally are cryptographically signed.

When an update is ready, Sandstorm kills the old server and starts the new one within a space of a second or two. Connected users do not lose their work and probably wouldn't even notice, except that Sandstorm explicitly notifies them so that they can refresh to update client-side scripts.

With the new update running, Sandstorm deletes the old version.

Thus, Sandstorm stays updated without you lifting a finger.

"""

Unfortunately doing Debian-style packages the "right way" would require basically abandoning the above strategy and instead declaring dependencies like anyone else. This would be hugely invasive to our code and a big maintenance and testing headache. Meanwhile there would ultimately be very little benefit from such a change -- indeed I would argue that it would make Sandstorm worse overall. So, we are not likely to do that.

Thus the options are:

- Ship a Debian package that contains just the installer script.

- Ship a Debian package that contains the contents of the self-container, placed under /opt/sandstorm/sandstorm-$VERSION, with its own copy of libc and everything.

Neither option complies with Debian rules, thus neither is likely to make its way into Debian proper. (I'm not sure we'd want that anyway, given the enormous difference in our release cycle lengths.)

The latter option may sound cleaner, but note that it implies giving up the Sandstorm auto-updater and relying on using apt to fetch updates. I really like that out updater knows how to "hot-update" Sandstorm without interrupting live users, which means it can reasonably by scheduled to happen automatically without admin intervention, which means we can reasonably expect than any update we push is live everywhere in 24 hours. I'm not sure how feasible it is to match that with apt. My sense is that this is not how Debian is normally used. Could we schedule update checks as a daily cron job? Can we make Sandstorm's update hand-off work in this model? (Note that it probably requires that there be a brief period in which both the old and new packages are installed.)

-Kenton

[Alec Berryman]

On Wed, Oct 21, 2015 at 12:50 PM, Kenton Varda <ken...@sandstorm.io> wrote:

I think for this conversation to be useful it's important that people first understand how Sandstorm's installer actually works.

Here's an excerpt from something I wrote about this that never got published:

"""

### Sandstorm installs into its own container

The first thing to understand about Sandstorm's installer is that it sets itself up in a container under `/opt/sandstorm` (or wherever you choose). There, it downloads a bundle that contains the entire Sandstorm server and all of its dependencies -- from Javascript to libc.

The main `sandstorm` binary is statically-linked, meaning it does not need any libraries from your system. When you execute it, it sets up and enters a Linux mount namespace (commonly known as a "container", similar to chroot) within which only Sandstorm's own files are visible.

This self-containerization means:

- Sandstorm uses exactly the dependency versions against which it was tested, not the ones present on your system. This means that once Sandstorm has been tested on one machine, we can have a high degree of confidence that no differences between distros will break it. Only hardware and kernel differences matter, and both tend to have highly stable interfaces. This is important, because otherwise testing on every Linux distro would be extremely time-consuming.

- Sandstorm does not need to install anything into your system nor make assumptions about it. You could be running a homebrew distro that uses static libraries and doesn't follow FHS, and Sandstorm would run fine.

- Uninstalling Sandstorm is a matter of deleting `/opt/sandstorm`.

### Sandstorm automatically updates

Sandstorm knows how to update itself. If you allow it, Sandstorm will do so automatically every 24 hours, otherwise you can invoke it manually by running `sandstorm update`.

Updates are downloaded over HTTPS and additionally are cryptographically signed.

When an update is ready, Sandstorm kills the old server and starts the new one within a space of a second or two. Connected users do not lose their work and probably wouldn't even notice, except that Sandstorm explicitly notifies them so that they can refresh to update client-side scripts.

With the new update running, Sandstorm deletes the old version.

Thus, Sandstorm stays updated without you lifting a finger.

"""

Unfortunately doing Debian-style packages the "right way" would require basically abandoning the above strategy and instead declaring dependencies like anyone else. This would be hugely invasive to our code and a big maintenance and testing headache. Meanwhile there would ultimately be very little benefit from such a change -- indeed I would argue that it would make Sandstorm worse overall. So, we are not likely to do that.

Thus the options are:

- Ship a Debian package that contains just the installer script.

- Ship a Debian package that contains the contents of the self-container, placed under /opt/sandstorm/sandstorm-$VERSION, with its own copy of libc and everything.

Neither option complies with Debian rules, thus neither is likely to make its way into Debian proper. (I'm not sure we'd want that anyway, given the enormous difference in our release cycle lengths.)

The latter option may sound cleaner, but note that it implies giving up the Sandstorm auto-updater and relying on using apt to fetch updates. I really like that out updater knows how to "hot-update" Sandstorm without interrupting live users, which means it can reasonably by scheduled to happen automatically without admin intervention, which means we can reasonably expect than any update we push is live everywhere in 24 hours. I'm not sure how feasible it is to match that with apt. My sense is that this is not how Debian is normally used. Could we schedule update checks as a daily cron job? Can we make Sandstorm's update hand-off work in this model? (Note that it probably requires that there be a brief period in which both the old and new packages are installed.)

Other packages, like Google Chrome and Dropbox, do not automatically schedule updates, although I suppose you could; Google Chrome does install a cron job to make sure that their custom repositories stay in /etc/apt/sources.list.  The typical Debian auto-update mechanism is unattended-upgrades, which doesn't have anything that you could hook into besides asking people to edit a Perl-like configuration file.

I was also worried about the uninstall story when I first looked at the Sandstorm install docs - "curl install.sh | bash" usually means "dump a bunch of crap in my system", and I ended up trying to use Docker, which led to a subpar experience (no auto-update, difficulties installing).  Perhaps an uninstall how-to near the install instructions, or a "why no Debian package" like above, would remove most of the need for Debian packaging - I would have been convinced by your explanation.

[Kenton Varda]

On Wed, Oct 21, 2015 at 4:55 PM, Alec Berryman <al...@thened.net> wrote:

Other packages, like Google Chrome and Dropbox, do not automatically schedule updates, although I suppose you could; Google Chrome does install a cron job to make sure that their custom repositories stay in /etc/apt/sources.list.  The typical Debian auto-update mechanism is unattended-upgrades, which doesn't have anything that you could hook into besides asking people to edit a Perl-like configuration file.

As a Chrome user I really wish it did auto-update. I'm too lazy to do it myself as often as I should... :/

 

I was also worried about the uninstall story when I first looked at the Sandstorm install docs - "curl install.sh | bash" usually means "dump a bunch of crap in my system", and I ended up trying to use Docker, which led to a subpar experience (no auto-update, difficulties installing).  Perhaps an uninstall how-to near the install instructions, or a "why no Debian package" like above, would remove most of the need for Debian packaging - I would have been convinced by your explanation.

Thanks, that's useful feedback!

Actually the text I copied was slightly a lie: We do add symlinks "sandstorm" and "spk" to /usr/local/bin, and we possibly install an initscript (or systemd equivalent). We should probably have a "sandstorm uninstall" command which carefully undoes these things.

-Kenton

[samro...@transitionnetwork.org]

Hi All

> * You can apt-get install sandstorm; it lives in Debian experimental

What are the chances of it getting into unstable?

I think this could be a really good synergy for the http://freedomboxfoundation.org/

They only install packages that are in unstable (Sid). It seems like if they could install Sandstorm then they could pick up a bunch of useful packages from you, and you might get some help from some of their dev's on packages like Diaspora.

Thanks

Sam

[Kenton Varda]

Hi Sam,

On Wed, Feb 3, 2016 at 1:33 PM, <samro...@transitionnetwork.org> wrote:

Hi All

> * You can apt-get install sandstorm; it lives in Debian experimental

What are the chances of it getting into unstable?

Unfortunately, probably not good since as discussed earlier in this thread the package flagrantly violates Debian guidelines (downloading a separate binary bundle, installing it in /opt/sandstorm, supplying its own libc, etc.), but complying with the guidelines would be a large engineering effort. :/

I think this could be a really good synergy for the http://freedomboxfoundation.org/

They only install packages that are in unstable (Sid). It seems like if they could install Sandstorm then they could pick up a bunch of useful packages from you, and you might get some help from some of their dev's on packages like Diaspora.

Maybe they could make an exception for Sandstorm?

-Kenton

[Asheesh Laroia]

FWIW Freedombox appears to be excited about ARM (or cross-platform in general), and Sandstorm only works on x86-64, since app packages contained x86-64 binaries and since MongoDB isn't supported (as far as I know) on ARM and Sandstorm's UI uses MongoDB.

On the whole, the goals of Sandstorm and of Freedombox are *extremely* well-aligned! I would *love* to see someone in the Freedombox world try running Sandstorm on a small one-board x86-64 computer and see what they think. Maybe that could be you, Sam!

If there's enough enthusiasm for Sandstorm in the Freedombox world, then creating a buildd network and building vagrant-spk packages from source could possibly work. I'd be interested to find out how much enthusiasm there is for that.

--

[andreas...@gmail.com]

On Thursday, October 22, 2015 at 1:55:56 AM UTC+2, Alec Berryman wrote:
> I was also worried about the uninstall story when I first looked at the Sandstorm install docs - "curl install.sh | bash" usually means "dump a bunch of crap in my system", and I ended up trying to use Docker, which led to a subpar experience (no auto-update, difficulties installing).  Perhaps an uninstall how-to near the install instructions, or a "why no Debian package" like above, would remove most of the need for Debian packaging - I would have been convinced by your explanation.

I guess the uninstall instructions are:

* Remove /opt/sandstorm
* Remove sandstorm user
* Remove sandstorm group
* Remove /etc/systemd/system/sandstorm.service

If there were a sandstorm-installer deb, this could be done automatically. Removing /opt/sandstorm should probably only be done by "purge" or not at all.

[Jake Weisz]

The possible concern there is that all of the grains (i.e. user data) is also under /opt/sandstorm, and many people would not expect uninstalling the application to delete all of their data, particularly if they intended to reinstall it.

-Jacob Weisz

[truthliber...@gmail.com]

This isn't going to be hugely useful to the whole community at first, but:
These past two weeks I have been experimenting with a packaging tool for applications I made, and because of that I'm now somewhat tempted to just start making an experimental sandstorm package when I'm done with "my" packages.
I have had so much trouble installing sandstorm using the installer and having it stop at sandcats and not be able to use my certificate that this may really be worth the effort for me.

My thoughts are that the installer has several steps and maybe that could be split into multiple "packages", like one or two standard ones and then "the rest of the installer", or something.
The sandboxing of the installer would be a problem, but it seems to me like there could be a way to have sandstorm's dependencies not collide with the system by simply putting them all in /usr/share/sandstorm or /usr/lib/sandstorm (?) etc, and then maybe there would be a way to put the whole chroot mechanism back together after that. I'm not looking forward to checking all the paths are working, but I'm not sure it's impossible to obey the FHS "better" at least.

I think I'd just naturally have to accept I'd have a slightly insecure sandstorm at first, and then try to fix everything up to match what was intended. But I'm not using it very seriously right now, so that wouldn't upset me.