npm error starting meteor web server

[John Lee]

Hi

I'm having trouble starting the web server after a from-source install,
can anybody help?

make -j printed a message to say a web server should be running on port
6080. I only got three processes and mongodb on 6081, and didn't find any
logs that looked interesting:

$ ps auxfwww | grep --color=always sandstor[m]
root 9152 0.0 0.0 2056 276 ? Ss 22:46 0:00 \_ service sandstorm start
root 9153 0.0 0.0 2056 292 ? S 22:46 0:00 \_ service sandstorm start
root 9215 0.0 0.0 2056 84 ? S 22:46 0:00 \_ service sandstorm start
$ sudo netstat -tlpn | grep 608
tcp 0 0 127.0.0.1:6081 0.0.0.0:* LISTEN 8541/mongod


After those exited, sudo /opt/sandstorm/sandstorm start starts those 3
processes again, but sudo /opt/sandstorm/sandstorm start-fe prints
"Requested front-end start." and exits successfully but seems to have no
effect. I see mongo.pid and sandstorm.pid in /opt/sandstorm/var/pid .


So, I tried run-dev.sh to start meteor -- see output at the end. Seems
it's running into trouble trying to install the node capnp bindings using
a bundled copy of npm, then wiping its temporary install directory before
I get a chance to see why the install command it prints is failing. I
tried debugging by inserting a "debugger;" breakpoint where the "capnp:
updating npm dependencies" message is logged, but though the debugger
starts up it doesn't break there, presumably because it starts a
subprocess -- not sure where in the big pile of bundled meteor / npm code
that happens...

Can I stop meteor doing its auto-install thing and just install the capnp
bindings myself maybe?


I followed these intstructions:

https://github.com/sandstorm-io/sandstorm#installing-from-source


I did run into minor compilation problems but I don't think they are
relevant: I guess because I didn't install capnproto under /usr/local -- I
did not install it but just left the build output in the source tree, and
supplied my own pkg-config files and PATH / PKG_CONFIG_PATH env vars when
building sandstorm. As a result I had to edit the sandstorm Makefile to
pass the -I flag to capnp compile so it found the .capnp files. Also
found I had to pass the libsodium package to pkg-config for run-bundle.

After that sandstorm compiled, and I installed that as root with a plain
make install (no GNU stow).

Here's the problematic run-dev.sh output:

$ (cd /home/john/dev/sandstorm/shell/ && ./run-dev.sh)
make: Nothing to be done for `shell-env'.
[[[[[ ~/dev/sandstorm/shell ]]]]]

=> Started proxy.
capnp: updating npm dependencies -- es6-promise, capnp...
npm WARN package.json packages-for-meteor-...@0.0.0 No description
npm WARN package.json packages-for-meteor-...@0.0.0 No repository field.
npm WARN package.json packages-for-meteor-...@0.0.0 No README data
../src/node-capnp/capnp.cc:31:27: fatal error: capnp/dynamic.h: No such file or directory
#include <capnp/dynamic.h>
^
compilation terminated.
make: *** [Release/obj.target/capnp/src/node-capnp/capnp.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack at ChildProcess.onExit (/home/john/.meteor/packages/meteor-tool/.1.0.32.fbybqs++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/dev_bundle/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:267:23)
gyp ERR! stack at ChildProcess.emit (events.js:98:17)
gyp ERR! stack at Process.ChildProcess._handle.onexit (child_process.js:809:12)
gyp ERR! System Linux 3.13.0-39-generic
gyp ERR! command "node" "/home/john/.meteor/packages/meteor-tool/.1.0.32.fbybqs++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/dev_bundle/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/john/dev/sandstorm/shell/packages/capnp/.npm/package-new-ld92c9/node_modules/capnp
gyp ERR! node -v v0.10.29
gyp ERR! node-gyp -v v0.13.1
gyp ERR! not ok
Build failed
npm ERR! ca...@0.1.6 install: `node ./build.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the ca...@0.1.6 install script.
npm ERR! This is most likely a problem with the capnp package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node ./build.js
npm ERR! You can get their info via:
npm ERR! npm owner ls capnp
npm ERR! There is likely additional logging output above.

npm ERR! System Linux 3.13.0-39-generic
npm ERR! command "/home/john/.meteor/packages/meteor-tool/.1.0.32.fbybqs++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/dev_bundle/bin/node" "/home/john/.meteor/packages/meteor-tool/.1.0.32.fbybqs++os.linux.x86_64+web.browser+web.cordova/meteor-tool-os.linux.x86_64/dev_bundle/bin/npm" "install"
npm ERR! cwd /home/john/dev/sandstorm/shell/packages/capnp/.npm/package-new-ld92c9
npm ERR! node -v v0.10.29
npm ERR! npm -v 1.4.14
npm ERR! code ELIFECYCLE
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /home/john/dev/sandstorm/shell/packages/capnp/.npm/package-new-ld92c9/npm-debug.log
npm ERR! not ok code 0


John

[John Lee]

On Sat, 1 Nov 2014, John Lee wrote:

> After that sandstorm compiled, and I installed that as root with a plain make
> install (no GNU stow).

Er, the random reference to GNU stow is a red herring: I mentioned it here
because I thought I'd used it for capnproto. I did not in fact do so, but
I forgot to edit this line to remove the reference.


John

[Kenton Varda]

Hi John,

The node capnp module is actually just a wrapper around the C++ library which is compiled on your host machine by npm. It looks like because you didn't install Cap'n Proto, npm wasn't able to find the headers when compiling the module. It may be possible to pass custom flags to npm somehow, but I am not sure; you would somehow have to get these flags not just through npm but through Meteor. It's probably easiest if you just install Cap'n Proto.

It is strange that this didn't cause make to fail. We've had problems from time to time where npm failures don't get reported up the stack through Meteor but I thought that was fixed.

Note that if you install the binary package then you can skip all of this and get a nice hermetic build that doesn't pollute your system. :)

-Kenton

John

--
You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[John Lee]

On Sat, 1 Nov 2014, Kenton Varda wrote:

> Hi John,
>
> The node capnp module is actually just a wrapper around the C++ library
> which is compiled on your host machine by npm. It looks like because you
> didn't install Cap'n Proto, npm wasn't able to find the headers when
> compiling the module. It may be possible to pass custom flags to npm
> somehow, but I am not sure; you would somehow have to get these flags not
> just through npm but through Meteor.

Right -- but not through setting PKG_CONFIG_PATH (and PATH) to match what
I used to build sandstorm, it turns out (even cutting meteor out of the
picture, which as far as I know is innocent). FWIW, this is because
node-gyp uses child_process.spawn, but does not explicitly pass an options
dict with env: process.env (which is what one needs to pass through env
vars). With that fixed, adding this seems to work:

diff --git a/binding.gyp b/binding.gyp
index d10334c..e7f6df8 100644
--- a/binding.gyp
+++ b/binding.gyp
@@ -15,6 +15,13 @@
'GCC_ENABLE_CPP_EXCEPTIONS': 'YES',
'MACOSX_DEPLOYMENT_TARGET': '10.7'
},
+ }, {
+'libraries': [
+'<!@(pkg-config capnp --libs)'
+],
+'include_dirs': [
+'<!@(pkg-config capnp --cflags-only-I | sed s/-I//g)'
+]
}],
]
}


...then I get a compiler error but I see from your very helpful note in
the source that that's because my cap'n proto is too old.

> It's probably easiest if you just install Cap'n Proto.

I expect so! Extra build steps are a nuisance if you want to change
anything but really I'm just being stubborn ;-)

> It is strange that this didn't cause make to fail. We've had problems from
> time to time where npm failures don't get reported up the stack through
> Meteor but I thought that was fixed.

Are the bindings supposed to be built at sandstorm build time, then?

> Note that if you install the binary package then you can skip all of this
> and get a nice hermetic build that doesn't pollute your system. :)

I guess I don't like to use things I can't build.

Thanks


John

[John Lee]

On Sat, 1 Nov 2014, John Lee wrote:

>> It's probably easiest if you just install Cap'n Proto.
>
> I expect so! Extra build steps are a nuisance if you want to change anything
> but really I'm just being stubborn ;-)

So I admitted defeat and installed it in /usr/local (when I noticed an
installed npm package wasn't enough for meteor to use it and I didn't find
a clearly documented way to install a meteor package locally...).

I now get a running web front end with the cute sandstorm animation.

How can I enable a fake login? It looks like setting IS_TESTING=true in
/opt/sandstorm/sandstorm.conf and restarting sandstorm should do it, but
having supplied an id and secret (github client id 1595880, client secret
doesn't matter, at a guess?), clicking on the github login button actually
pops up a window with a github.com 404. Is this not supposed to provide a
local login for development? That's all I need (or a way to defeat login
completely)...


John

[Kenton Varda]

On Sun, Nov 2, 2014 at 4:05 PM, John Lee <j...@pobox.com> wrote:

How can I enable a fake login?  It looks like setting IS_TESTING=true in /opt/sandstorm/sandstorm.conf

No, don't do that. That's for our test scripts and it disables various security.

 

and restarting sandstorm should do it, but having supplied an id and secret (github client id 1595880, client secret doesn't matter, at a guess?), clicking on the github login button actually pops up a window with a github.com 404.  Is this not supposed to provide a local login for development?  That's all I need (or a way to defeat login completely)...

A client ID looks like "12c4e047ccb4fc4938e4". And yes, the secret is important.

Please follow the on-screen instructions to generate a new OAuth client ID/secret.

-Kenton

[John Lee]

On Sun, 2 Nov 2014, Kenton Varda wrote:

> On Sun, Nov 2, 2014 at 4:05 PM, John Lee <j...@pobox.com> wrote:
>
>> How can I enable a fake login? It looks like setting IS_TESTING=true in
>> /opt/sandstorm/sandstorm.conf
>
>
> No, don't do that. That's for our test scripts and it disables various
> security.

OK, I'll use github.

I think I wasn't clear though: I'm not intending to use this for real data
at the moment, just to try to get a working system, so I can try to
understand it a bit. For that reason my preference was for it to be
disconnected from the world outside of my own machine, and I wasn't
concerned about security.

I appreciate your help, thanks


John

[Kenton Varda]

Yeah, sorry, we don't currently implement any local login system. There are a bunch of reasons for this, including the fact that if you download an app instance from one Sandstorm server and then upload it to another, we want the user IDs to stay consistent. Eventually I'd like to implement something based on federated identity but that's a bit complicated. :)

I guess that some sort of "test account for development box" login might make sense as well. For this, we would intentionally have no password because we wouldn't want people to think it's a good idea to use in production.

Maybe it makes sense to just extend the demo account code for this purpose. FWIW, you can set ALLOW_DEMO_ACCOUNTS=true in your config, though this puts the server in a mode that we generally assume is only meant to run on demo.sandstorm.io so YMMV.

-Kenton

John

[Kenton Varda]

I went ahead and filed an issue for this:

https://github.com/sandstorm-io/sandstorm/issues/150