Help building shiny-server 0.4.0 from source

[Michael Conklin]

I successfully had running shiny-server 0.3.6 on our OpenSUSE 11.2 system. 

Now I tried to install 0.4.0 following the instructions on your website.

1st problem: git clone https://github.com/rstudio/shiny-server.git  produced a 503 error

so, I downloaded the zip file and unpacked it on my machine.

I successfully ran cmake and make

had to change permissions on bin/npm to make it executable

ran make install  which seemed to work as expected.

running the executable throws the following error:

module.js:340

    throw err;

          ^

Error: Cannot find module '../../build/Release/posix'

    at Function.Module._resolveFilename (module.js:338:15)

    at Function.Module._load (module.js:280:25)

    at Module.require (module.js:364:17)

    at require (module.js:380:17)

    at Object.<anonymous> (/usr/local/shiny-server/lib/core/fsutil.js:16:13)

    at Module._compile (module.js:456:26)

    at Object.Module._extensions..js (module.js:474:10)

    at Module.load (module.js:356:32)

    at Function.Module._load (module.js:312:12)

    at Module.require (module.js:364:17)

I am completely stuck now (especially with no 0.3.6 version on the server (the instructions said to uninstall it first)

ANY HELP is greatly appreciated

[Michael Conklin]

Seems like no one has a clue on this. So I am going back to start over.

First problem :

 git clone https://github.com/rstudio/shiny-server.git

Initialized empty Git repository in /root/myserver/shiny-server/.git/

fatal: https://github.com/rstudio/shiny-server.git/info/refs download error - The requested URL returned error: 503

It doesn't appear that there is anything wrong with github....so could this be a problem that the server is behind a proxy server?  What should I be looking for that might cause this problem?  My first attempt of downloading the repository as a zip file and unzipping seems not to work as I get many errors in the build complaining that I don't have a valid git repository. 

ANY IDEAS???  I now have 50 potential users world wide wondering where the server went.

[Thorsten]

I also had that problem. Do you have posix and node.js installed?

 I solved that problem by changing then the line in /usr/local/shiny-server/lib/core/fsutil.js where posix is called to var posix = require('posix');

This error will accure for some other classes where I also changed the corresponding line. After that shiny-server started wiithout problems but now I can't run any shiny application and I don't know why. Hope you have more luck!

Thorsten 

[Floris Padt]

Hi Michael,

I can only say I ran into the same issue yesterday. So it is not your setup. I run SLES SP2 and ended with the same error. I have R-Studio 0.98 and R 3.02 installed

The result was that graphs were not appearing. I couldn solve it. So installed Ubuntu 13.10 next to SLES. This runs perfectly. I understand that this is not a solution to you.

Hope you find a solution (so I can revert back to SLES)

Op vrijdag 8 november 2013 22:35:15 UTC+1 schreef Michael Conklin:

[Jeff Allen]

Hi Michael. Sorry we missed your original post and for the trouble.

I think I know what the problem is but I'm not sure what would fix it on your system. Any chance you could send the output of the command "(cd .. && bin/npm --python="$PYTHON" install)" from the install instructions?

Thanks!

Jeff

[Michael Conklin]

Hi Jeff:

Here is what I get.  Not sure if this is what happened the first time or if some of the errors are due to re-trying.

npm WARN package.json pa...@0.0.1 No repository field.

npm WARN package.json sta...@0.1.2 No repository field.

npm http GET https://github.com/rstudio/node-unixgroups/tarball/af6544c

npm http GET https://registry.npmjs.org/webkit-devtools-agent

npm http GET https://github.com/rstudio/faye-websocket-node/archive/95436354.tar.gz

npm http GET https://github.com/rstudio/node-http-proxy/archive/cfb25b354.tar.gz

npm http GET https://s3.amazonaws.com/rstudio-buildtools/sockjs-e3d3695.tar.gz

npm http 200 https://s3.amazonaws.com/rstudio-buildtools/sockjs-e3d3695.tar.gz

npm http 304 https://registry.npmjs.org/webkit-devtools-agent

npm http 200 https://github.com/rstudio/faye-websocket-node/archive/95436354.tar.gz

npm http 200 https://github.com/rstudio/node-http-proxy/archive/cfb25b354.tar.gz

npm http 200 https://github.com/rstudio/node-unixgroups/tarball/af6544c

npm WARN engine webkit-dev...@0.1.2: wanted: {"node":"0.6 || 0.7 || 0.8"} (current: {"node":"v0.10.21","npm":"1.3.11"})

npm http GET https://registry.npmjs.org/underscore/1.3.3

npm http GET https://registry.npmjs.org/ws/0.4.25

npm WARN `git config --get remote.origin.url` returned wrong result (git://github.com/c4milo/v8-profiler.git)

npm ERR! git clone git://github.com/c4milo/v8-profiler.git fatal: destination directory '/root/.npm/_git-remotes/git-github-com-c4milo-v8-profiler-git-8ad28aa6' already exists.

npm ERR! Error: Command failed: fatal: destination directory '/root/.npm/_git-remotes/git-github-com-c4milo-v8-profiler-git-8ad28aa6' already exists.

npm ERR!

npm ERR!     at ChildProcess.exithandler (child_process.js:637:15)

npm ERR!     at ChildProcess.EventEmitter.emit (events.js:98:17)

npm ERR!     at maybeClose (child_process.js:735:16)

npm ERR!     at Process.ChildProcess._handle.onexit (child_process.js:802:5)

npm ERR! If you need help, you may report this log at:

npm ERR!     <http://github.com/isaacs/npm/issues>

npm ERR! or email it to:

npm ERR!     <np...@googlegroups.com>

npm ERR! System Linux 3.0.80-0.5-default

npm ERR! command "/root/shiny-server/ext/node/bin/node" "/root/shiny-server/ext/node/lib/node_modules/npm/bin/npm-cli.js" "--python=" "install"

npm ERR! cwd /root/shiny-server

npm ERR! node -v v0.10.21

npm ERR! npm -v 1.3.11

npm ERR! code 128

npm http 304 https://registry.npmjs.org/underscore/1.3.3

npm http 304 https://registry.npmjs.org/ws/0.4.25

npm ERR!

npm ERR! Additional logging details can be found in:

npm ERR!     /root/shiny-server/npm-debug.log

npm ERR! not ok code 0

[Jeff Allen]

Michael,

I think your issue is related to your inability to clone GitHub repos. I haven't seen that before (other than momentary hiccups in GitHub's availability), so I'd bet there's a firewall issue. The problem is that you're unable to complete the 'npm install' step since you can't get all the dependencies. We've been talking about restructuring a piece of the project that would eliminate this issue for you. Until that's finalized, you'll need to find a work-around to get the npm dependencies.

If you have access to another OpenSUSE system not behind that firewall, you could try installing there and then copying the 'node_modules' directory over to your current machine. It would need to be a similar architecture to your existing machine, though, as some code will be compiled that might be incompatible if you were on a different OS/architecture. Perhaps your last option would be to get the node_modules using npm on another machine (not behind the firewall, perhaps with a different OS or architecture) then use node-gyp to compile for your own platform.

Sorry this is pretty messy -- we haven't encountered this scenario before so this isn't too well supported. If we are able to whip up a formal solution I'll mention it here and be sure to add it to the wiki.

Jeff

[Michael Conklin]

Thanks Jeff ... let me see if IT will let git through the firewall :)

[Michael Conklin]

So now I try starting over - no response from my IT group but I did manage to clone the github repository by cloning it on my Windows machine and then using SCP to put the directory tree on the server.  

Everything looks good until I get to  (cd .. && bin/npm --python="$PYTHON" install)

First (like I got the last time) I get a complaint that npm is not executable.  I change the permissions  and try again

Now I get the following message:

 (cd .. && bin/npm --python="$PYTHON" install)

-bash: bin/npm: /bin/sh^M: bad interpreter: No such file or directory

I did not get this error before and being a real linux novice I cannot even tell what file it is failing to find.  Any help would be appreciated.

Mike

[Jeff Allen]

Interesting. I'll have to look into the details to see if we can do anything about that, but one patch would be to tell your system to point "/bin/sh" to "/bin/bash" using the following command.

sudo ln -s /bin/bash /bin/sh

Then you can try an npm install again.

Jeff

--
You received this message because you are subscribed to a topic in the Google Groups "Shiny - Web Framework for R" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/shiny-discuss/YdFGLnJy_vs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to shiny-discus...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Michael Conklin]

Currently I get the server to run (crippled) by doing the following:

download node-posix package and compile with npm and then move the result to shiny-server/build/Release/posix  where the server appears to look for the module.

Eliminate webkit-devtools-agent from dependencies in shiny-server package.json list.  (since this seems to always require cloning a git repository which is still impossible on the server)

Now the server compiles and when I execute shiny-server I get the server listening on port 3838 

If I point a web-browser at it. I get the index page with the links to the the various apps. But whenever I try to start an app I get the following:

On the console:

[2013-11-15 15:25:16.500] [INFO] shiny-server - Error getting worker: Error: Connection attempt was aborted.

[2013-11-15 15:25:27.605] [INFO] shiny-server - Error getting worker: Error: Connection attempt was aborted.

[2013-11-15 15:25:38.687] [INFO] shiny-server - Error getting worker: Error: Connection attempt was aborted.

and in the webbrowser:

An error has occurred

The application failed to start.

Connection attempt was aborted.

Since I have made so many changes I suspect this is simply due to the work-arounds I have had to implement to even get this far.  I am still trying to get IT to give me access to github from the server so I can do a proper install.

Some additional information  that did not make it into the group due to my inability to properly reply to emails are reproduced below.

-------------------------------------------------------------------------------------------------------------------------------------------------

Okay...that problem was solved and had to do with cloning the repo on a windows machine and copying the tree to the linux machine.  I started over again, cloning on my ubuntu laptop and copy the tree to the SLES 11.2 server.  This eliminated that bash error (I think it had to do with the CRLF vs LF format of some file somewhere.

So now I am back at the same spot as last week.  This is the end of my npm-debug.log which in the screen output after running the npm install. 

2638 silly gunzTarPerm extractEntry test/core/simple/test-http-proxy.js

2639 error Error: Command failed: fatal: destination directory '/root/.npm/_git-remotes/git-github-com-c4milo-v8-profiler-git-8ad28aa6' already exists.

2639 error

2639 error     at ChildProcess.exithandler (child_process.js:637:15)

2639 error     at ChildProcess.EventEmitter.emit (events.js:98:17)

2639 error     at maybeClose (child_process.js:735:16)

2639 error     at Socket.<anonymous> (child_process.js:948:11)

2639 error     at Socket.EventEmitter.emit (events.js:95:17)

2639 error     at Pipe.close (net.js:466:12)

2640 error If you need help, you may report this log at:

2640 error     <http://github.com/isaacs/npm/issues>

2640 error or email it to:

2640 error     <np...@googlegroups.com>

2641 error System Linux 3.0.80-0.5-default

2642 error command "/root/shiny-server/ext/node/bin/node" "/root/shiny-server/ext/node/lib/node_modules/npm/bin/npm-cli.js" "--python=python" "install"

2643 error cwd /root/shiny-server

2644 error node -v v0.10.21

2645 error npm -v 1.3.11

2646 error code 128

2647 verbose exit [ 1, true ] 

It seems like the screen output only shows this error:

npm ERR! git clone git://github.com/c4milo/v8-profiler.git fatal: destination directory '/root/.npm/_git-remotes/git-github-com-c4milo-v8-profiler-git-8ad28aa6' already exists.

npm ERR! Error: Command failed: fatal: destination directory '/root/.npm/_git-remotes/git-github-com-c4milo-v8-profiler-git-8ad28aa6' already exists.

which is the same as I was getting before. Now, if I were to clone the git similarly here, where do I find the line/file where npm tries to clone it.  If I were to eliminate that line...as if it was cloned already would that be a possibility?  Or, even better, would npm not try to clone the git if it was already present?

Michael Conklin <michael.c...@gmail.com>	2:50 PM (17 hours ago)
to Jeff

if I run npm list in the shiny-server directory I get this:

nuex-apmktsc01:~/shiny-server # npm list

shiny-...@0.4.0 /root/shiny-server

├── ba...@0.0.1

├── cj...@0.2.1

├─┬ client-...@0.2.0

│ ├── coo...@0.2.1

│ └── node-...@0.6.0

├─┬ con...@2.7.11

│ ├── buffer...@0.2.1

│ ├── by...@0.2.0

│ ├── coo...@0.0.5

│ ├── cookie-s...@1.0.1

│ ├── de...@0.7.4

│ ├── formi...@1.0.14

│ ├── fr...@0.1.0

│ └─┬ se...@0.1.1

│   ├── mi...@1.2.11

│   └── range-...@0.0.4

├─┬ faye-we...@0.6.1

│ └── websocke...@0.3.0

├── grace...@2.0.1

├─┬ handl...@1.0.7

│ └── ugli...@1.2.6

├─┬ http-...@0.10.0

│ ├── col...@0.6.2

│ ├── pkg...@0.2.3

│ └─┬ ut...@0.1.7

│   ├── as...@0.1.22

│   ├── deep-...@0.1.0

│   ├── i...@0.3.2

│   ├── mkd...@0.3.5

│   ├── n...@0.2.7

│   └── rim...@1.0.9

├─┬ log...@0.5.8

│ ├── as...@0.1.15

│ └── deq...@1.0.3

├─┬ mo...@1.12.1

│ ├── comm...@0.6.1

│ ├── de...@0.7.4

│ ├── di...@1.0.2

│ ├─┬ gl...@3.2.1

│ │ ├── grace...@1.2.3

│ │ ├── inhe...@1.0.0

│ │ └─┬ mini...@0.2.12

│ │   ├── lru-...@2.3.1

│ │   └── sig...@1.0.0

│ ├── gr...@1.7.0

│ ├─┬ ja...@0.26.3

│ │ └── mkd...@0.3.0

│ └── mkd...@0.3.5

├── mom...@1.7.2

├─┬ opti...@0.3.5

│ └── word...@0.0.2

├── pa...@0.0.1

├── q...@0.9.7

├── q...@0.6.5

├── regexp...@0.0.0

├── rew...@1.1.3

├─┬ se...@0.1.4

│ ├── de...@0.7.4

│ ├── fr...@0.2.0

│ ├── mi...@1.2.11

│ └── range-...@0.0.4

├── sho...@1.2.2

├─┬ si...@1.7.3

│ └─┬ buster...@0.5.6

│   └── buste...@0.6.4

├─┬ soc...@0.3.5

│ ├── faye-we...@0.4.0

│ └── node...@1.3.3

├─┬ sp...@0.2.10

│ └── thr...@2.3.4

├── sta...@0.1.2

├── under...@1.4.2

├── unixg...@0.1.0

└── UNMET DEPENDENCY webkit-devtools-agent *

npm ERR! missing: webkit-devtools-agent@*, required by shiny-...@0.4.0

npm ERR! not ok code 0

if have the cloned git repositories for webkit-devtools-agent and for profiler-V8 (which is a dependency of webkit-devtools-agent), How would I go about compiling them and getting the modules in the right place?

Mike

[Jeff Allen]

It looks like your R processes are crashing before Shiny Server can connect, or some other failure to connect is occurring.

Are you attempting to use a custom Shiny application? I'd recommend trying first with one of the "hello world" examples that comes with shiny (in /examples/01_hello", for instance). You can also check the logs in your log directory (/var/log/shiny-server/) by default to see if you get any more insight about why an application would be crashing.

Jeff

[Michael Conklin]

The hello app exhibits the same behavior.  The log files are created but are completely empty.  Is there a place that I can get instructions for installing shiny-server 0.3.6??? I had that running before and stupidly uninstalled it (per instructions)  to try and build the new version.  I have 50 users who went through a training for our apps a couple of weeks ago and having the server completely unavailable is getting to be a problem.

Michael Conklin <michael....@gmail.com>

to Jeff

[Jeff Allen]

Sure thing. Here is the old README and the wiki pages are still available for Ubuntu and CentOS install instructions, if you had used any information from either of those.

Sorry for the headache with 0.4.0. We're working to expand the footprint of supported distributions. Hopefully we'll go through the process of getting SLES supported before long to save you some trouble in the future.

One afterthought -- I've noticed some connectivity problems with npm this week (from which you would have been downloading various packages). In fact, when I went to get the link for the shiny server 0.3.6 module (here), the site was down. I can't imagine that it would have consistently caused you as much trouble as you've seen this week, but it's worth keeping in mind. Certainly keep an eye out for that if you encounter any issues installing 0.3.6 -- it's still down for me right now.

Best,
Jeff

[Michael Conklin]

I can start the v0.3.6 server and it loads up the index page with the apps that I previously had working. But, now again, I immediately get 

An error has occurred

The application failed to start.

The application exited during initialization.

So same behavior as the 0.4.0 server but a slightly different message.

[Jeff Allen]

Might check your R environment and Shiny version? Any changes you might have made there?

Michael Conklin

November 15, 2013 at 12:40 PM

Jeff Allen

November 15, 2013 at 10:18 AM

Michael Conklin

November 15, 2013 at 10:11 AM

The hello app exhibits the same behavior.  The log files are created but are completely empty.  Is there a place that I can get instructions for installing shiny-server 0.3.6??? I had that running before and stupidly uninstalled it (per instructions)  to try and build the new version.  I have 50 users who went through a training for our apps a couple of weeks ago and having the server completely unavailable is getting to be a problem.

On Friday, November 15, 2013 9:29:24 AM UTC-6, Jeff Allen wrote:

--
You received this message because you are subscribed to a topic in the Google Groups "Shiny - Web Framework for R" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/shiny-discuss/YdFGLnJy_vs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to shiny-discus...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Jeff Allen

November 15, 2013 at 9:29 AM

It looks like your R processes are crashing before Shiny Server can connect, or some other failure to connect is occurring.

Are you attempting to use a custom Shiny application? I'd recommend trying first with one of the "hello world" examples that comes with shiny (in /examples/01_hello", for instance). You can also check the logs in your log directory (/var/log/shiny-server/) by default to see if you get any more insight about why an application would be crashing.

Jeff

On 11/15/2013 9:08 AM, Michael Conklin wrote:

--
You received this message because you are subscribed to a topic in the Google Groups "Shiny - Web Framework for R" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/shiny-discuss/YdFGLnJy_vs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to shiny-discus...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Michael Conklin

November 15, 2013 at 9:08 AM

Michael Conklin <michael.c...@gmail.com>

to Jeff

[Michael Conklin]

Could it be that I updated the packages to shiny 0.8?  That is the only change I made.

> sessionInfo()

R version 3.0.1 (2013-05-16)

Platform: x86_64-suse-linux-gnu (64-bit)

locale:

[1] C

attached base packages:

[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:

[1] shiny_0.8.0

loaded via a namespace (and not attached):

[1] RJSONIO_1.0-3 bitops_1.0-5  caTools_1.14  digest_0.6.3  httpuv_1.2.0

[6] xtable_1.7-1

Michael Conklin <michael....@gmail.com>

to Jeff

[Jeff Allen]

I don't expect that would break anything, but we're not continuing to support 0.3.6 so we haven't tested anything against it. Sorry about that!

Jeff

Michael Conklin

November 15, 2013 at 1:03 PM

Could it be that I updated the packages to shiny 0.8?  That is the only change I made.

> sessionInfo()

R version 3.0.1 (2013-05-16)

Platform: x86_64-suse-linux-gnu (64-bit)

locale:

[1] C

attached base packages:

[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:

[1] shiny_0.8.0

loaded via a namespace (and not attached):

[1] RJSONIO_1.0-3 bitops_1.0-5  caTools_1.14  digest_0.6.3  httpuv_1.2.0

[6] xtable_1.7-1

>

On Friday, November 15, 2013 12:44:48 PM UTC-6, Jeff Allen wrote:

--
You received this message because you are subscribed to a topic in the Google Groups "Shiny - Web Framework for R" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/shiny-discuss/YdFGLnJy_vs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to shiny-discus...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Jeff Allen

November 15, 2013 at 12:44 PM

Might check your R environment and Shiny version? Any changes you might have made there?

[Michael Conklin]

I reverted to shiny 0.6 and now it works.  I wonder if since 0.8 includes the debugging capabiilities and I was never able to get the webkit-agent part linked in the 0.4 version and 3.6 doesn't have that either, if that is causing the problem.  In any event at least I have my previous versions up and running.

Mike