Any gotchas I should know about?

[Nolan Darilek]

Finally got time to work on my Sandstorm-based ebook reader. It's
basically a front-end that accepts books via upload, runs the Calibre
ebook-convert utility to transform them to htmlz, then serves up the
resulting HTML/CSS/images along with a quick-and-dirty table listing the
books you've uploaded.

I've tested this on my Linux laptop and it works well enough, but in
Sandstorm the exec of the Calibre ebook-convert utility fails with:

events.js:72
throw er; // Unhandled 'error' event
^
Error: spawn ENOENT
at errnoException (child_process.js:1011:11)
at Process.ChildProcess._handle.onexit (child_process.js:802:34)

Here is the library I'm using to do the conversion:

https://github.com/sethvincent/ebook-convert

It doesn't do anything fancy, just spawns the ebook-convert utility and
add a few event handlers for its output.

Here are some things I've investigated of the "is it plugged in?" variety:
1. Again, this works well enough without Sandstorm that I'm ready to
build an SPK for personal use. Didn't think it would fail to spin up the
ebook-convert utility.
2. ebook-convert is present on the VM in /usr/bin.
3. /usr/bin appears to be in the PATH set by sandstorm-pkgdef.
4. The app uses node-temp and temporary directories. I thought perhaps
I'd need to set TMPDIR, but either it is getting set somewhere or
node-temp has a fallback to /tmp. Various stages in the
upload/conversion process write to temporary files/directories and these
all seem to get created, if the functions that create them not failing
is any indication.
5. I've caught all events that the ebook-convert library claims to emit,
but it doesn't seem to output any errors. Looks like it isn't running at
all, but I thought I'd be thorough. It seems like if it couldn't find an
input file for instance, then it'd output something on stderr rather
than fail in a way that looks like the binary itself can't be found.

To be clear, this is in vagrant-spk dev, so it should be able to find
ebook-convert as installed in setup.sh AFAIK.

Is there something in the Sandstorm security model or sandbox that might
prevent grains from spawning new processes? If not, thoughts on why this
might work fine locally but fail in Sandstorm? I can share my code if
there's no obvious reason this should fail--it isn't pretty but it's not
complex either. :)

Thanks.

[Asheesh Laroia]

Hi Nolan! Thanks for emailing.

This might be a problem due to the Sandstorm sandbox.

Given that you're running into a mysterious "file does not exist" type error, the problem is probably that some library you're using assumes that /proc exists, which it doesn't really on Sandstorm.

The thing I want to know next is - what file does not exist, that it wants to access? If you share the source, I can try to look into that in a few days, or maybe someone else can sooner; or, maybe you can enhance the logging so that it's possible to see what it's failing to access.

Let us know!

--
You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Asheesh Laroia]

Also - looking at the very simple, 0-dependencies code of https://github.com/sethvincent/ebook-convert ...

Michael Nutt, if you're on this list, is there a chance you can review Nolan's package and see what's going on? I believe you're familiar with nodejs, which is the main thing required to figure this out, I believe.

[Nolan Darilek]

Cool, thanks, I'll package it up and set up a repo somewhere soon.

I think I recall a while back some app or way to get an interactive shell in an environment like a grain's sandbox. Does such a thing exist, or am I imagining things? If so, I can try downloading the code, modifying it to include my dependencies, then seeing if it is indeed the missing /proc. Not sure why ebook-convert would need /proc but maybe Node itself does for spawning processes.

Thanks.

[Asheesh Laroia]

Happy to help.

As for an interactive shell: https://github.com/sandstorm-io/vagrant-spk/issues/56 (look for the references to "nsenter-devgrain") contains info on a branch of vagrant-spk with a not-fully-tested-but-probably-works way to look within the sandbox.

[Nolan Darilek]

So I haven't done nsenter yet, but it looks like the issue with spawn may very well be the lack of /proc:

https://github.com/nodejs/node-v0.x-archive/issues/10426

That issue isn't specifically spawn-related but the error they're getting looks similar enough that I'd be willing to bet on them sharing a cause. I'm currently investigating this to be sure.

If that is the issue, thoughts on how I might implement this? I'm having horrifying thoughts of using another process, a message queue of some sort, and hoping that mechanism doesn't need /proc to run commands. :/ Hope there's a simpler way. I need to basically run `ebook-convert /tmp/some-uploaded-file .htmlz`, know when it completes, and know the exit status.

Is this something the powerbox might be able to help with? I definitely see the benefit to not letting apps execute binaries by default, but if I'm explicitly building an app to export a command line process and its results to the web, I pretty much need it to be able to run at least whitelisted binaries. :)

Thanks.

[Michael Nutt]

Hey Nolan,

I’d be happy to take a look if you can throw it in a repository somewhere. FWIW I use `exec` just fine without issues in one of my sandstorm apps: https://github.com/mnutt/davros/blob/master/server/dav/statvfs-shim.js#L56

Michael

[Nolan Darilek]

OK, so I did some digging using the nsenter method in the linked issue. I don't see a /usr/bin at all, so the enoent is that /usr/bin/ebook-convert is missing.

So I'm massively confused. I thought the whole point of dev mode was that you had the entire contents of your system available so spk could automatically make a list of dependencies to pack. Does meteor-spk interfere with this somehow? launcher.sh refers to /start.js at one point, for instance, though I never see where start.js is placed in the root nor linked in the manifest. This whole time I've been operating under the assumption that the contents of my VM were available to my Meteor app in dev mode, so I didn't even think to assume that the ebook-convert binary wouldn't be available.

Sorry, I'm working to get a repo available, but due to my attempts to use Meteor 1.3-rc.1 I don't quite have a clean "vagrant-spk up" story yet. And my connection today is slow so I can't quickly burn down/recreate VMs to test changes. But I think this "not all of my VM available in dev mode" bit is the key. :)

On 03/12/2016 01:14 PM, Asheesh Laroia wrote:

[Asheesh Laroia]

Oh, yes - meteor-spk works differently, and we don't document this anywhere. Sorry for the confusion. You're totally right that "not all of my VM available in dev mode" is the key.

The short answer is that you need to add more paths to the bundle. Approximate overview of how to do that:

+  sourceMap = (

+    # The following directories will be copied into your package.

+    searchPath = [

+      ( sourcePath = "/home/vagrant/bundle" ),

+      ( sourcePath = "/opt/meteor-spk/meteor-spk.deps" ),

+      ( sourcePath = "/" ),

+

+      # Include busybox cat & mkdir.

+      ( sourcePath = "/bin/busybox", packagePath = "bin/busybox" ),

+      ( sourcePath = "/usr/local/bin/mkdir", packagePath = "usr/local/bin/mkdir" ),

+      ( sourcePath = "/usr/local/bin/cat", packagePath = "usr/local/bin/cat" ),

+    ]

+  ),

You're going to have to add each file needed by the executable via this sourcePath and packagePath thing.

I would describe this as seriously suboptimal and it's on my roadmap for things to fix with vagrant-spk + meteor.

[Nolan Darilek]

Aha!

OK, maybe I'll hold off until this gets fixed then. /usr/bin/ebook-convert is a Python script, so I'd have to pull in Python, all of its standard libraries, Calibre, etc. Easier to let dev mode figure out this dependency graph for me.

Anyhow, certainly not time-critical. I can do without this ebook app for a while longer. :)

[Asheesh Laroia]

OK! I'll take this as good motivation to address this meteor-spk issue in the future.

[Nolan Darilek]

Out of curiosity, what issues would I face if I switched my app from the official Sandstorm meteor stack to the diy stack and installed my own Meteor/Mongo dependencies, treating everything like the other stacks do? These are the issues I can think up off-hand:

1. Having to pull in my own Node. This can probably be universally automated in the near future by pulling in the Node 0.10X Debian repository. From Meteor 1.3 onward, `meteor node` and `meteor npm` run the versions as shipped with the version of Meteor your app is pinned to, so you can do away with that entirely as long as you're on 1.3+. I think that while Meteor ships with certain Node versions, it requires those versions or later within the same semver series (with the exception of a while back when I think one Node version leaked memory) so the newest 0.10 should be able to run older versions I think.

2. Custom Mongo. I don't understand the full details here. Does Mongo 3.2 and Wired Tiger resolve the concerns that required (or highly advocated for) a fork? In any case, the shift away from guaranteed Mongo to the new Apollo stack means Mongo won't be a guaranteed dependency for too much longer, so you'll start seeing more apps with Rethink/SQLite/whatever Meteor can back an Apollo server with.

I recognize a) you're all busy and b) I'm playing with Meteor versions that won't be released until next week. As such, I don't expect you to have the answers. I'm happy to move my app off of the current Meteor stack, create some shell scripts and try submitting a "meteor-new" stack that maybe attempts to replace meteor-spk with something a bit less magical. :) Then maybe that can become the official meteor stack after a year or so when meteor 1.3 is a distant memory and all of this stuff works with most modern Sandstorm apps. I'm just wondering what gotchas I'll hit with *this*. :)

Thanks.

[Nolan Darilek]

OK, so I updated the meteor-spk issue with this, but I basically have a Meteor 1.3 app running without meteor-spk using the diy stack. Unfortunately `meteor node` attempts to spin up the entire Meteor toolchain, which does something that doesn't play well with Sandstorm's read-only filesystem when launching a grain. However you *can* use `meteor node` hackery to get the path of the node/npm binaries shipped with Meteor, which the particular version used by your app expects to find, then symlink those versions into /usr/local/bin. This will be particularly relevant soon when Meteor actually starts using a modern version of Node and, with any luck, updates it regularly enough that shipping separate binaries will be painful.

I'll see if I can submit a stack PR soon, but I thought I'd update this thread for anyone watching on the sidelines and not following the issue.

[Nolan Darilek]

You can see what I'm doing here:

https://github.com/ndarilek/alexandria/tree/master/.sandstorm

Not sure about the specific Mongo issue you mention but I haven't seen any major issues with this.

Unfortunately I won't be able to get it into a vagrant-spk stack until an --always-include flag for spk lands, because it needs to always include /home/vagrant/bundle. My last week has been busy so I haven't yet filed the issue for that.

In an experiment, it packed a default Meteor app at around 20M, so not too bad. Unfortunately said spk was corrupt somehow. This may be user error--again, I haven't had much time lately. :)

On 03/31/2016 03:55 PM, Asheesh Laroia wrote:

On Thu, Mar 17, 2016 at 3:27 PM, Nolan Darilek <no...@thewordnerd.info> wrote:

OK, so I updated the meteor-spk issue with this, but I basically have a Meteor 1.3 app running without meteor-spk using the diy stack. Unfortunately `meteor node` attempts to spin up the entire Meteor toolchain, which does something that doesn't play well with Sandstorm's read-only filesystem when launching a grain. However you *can* use `meteor node` hackery to get the path of the node/npm binaries shipped with Meteor, which the particular version used by your app expects to find, then symlink those versions into /usr/local/bin. This will be particularly relevant soon when Meteor actually starts using a modern version of Node and, with any luck, updates it regularly enough that shipping separate binaries will be painful.

I'll see if I can submit a stack PR soon, but I thought I'd update this thread for anyone watching on the sidelines and not following the issue.

Good to know all that - thanks for your research!

As another data point, for what it's worth, Kenton's meteor-spk, and similarly the vagrant-spk Meteor stack, use "meteor build" to create a bundle that contains your app and all necessary files, minified nicely.

Unfortunately "meteor build" doesn't configure MongoDB, and has this problem you mentioned, where it ignores any tracing so you can only use files/etc. that show up in the "meteor build", so I know it won't solve all your problems as-is, but I wonder if you can also benefit from piggybacking off "meteor build".