Shiny without web sockets?

[Andy Chu]

I'm interested in using Shiny in a situation where proxies don't support web sockets.

Is there a fundamental reason that Shiny couldn't just use AJAX rather than web sockets?  I took a look through the code and it looks like it is using raw web sockets on the client side (new WebSocket(...)).

On the server side is it using SockJS?  I thought the point of SockJS is that it could fall back on a non-websocket transport in situations where it wasn't available.  And it has a client library to make this possible in a cross-browser way?  But I guess the R side only support web sockets.

I'd be interested to hear any thoughts on this, and also any pointers if there is documentation about the client/server protocol.  Thanks!

Andy

[Joe Cheng]

If you use Shiny Server (and you should!), you should be able to proxy it through non-websocket-supporting proxies. We use SockJS on the Shiny Server side, then Shiny Server opens a websocket connection to the backend R process (localhost-to-localhost).

Andy

--
You received this message because you are subscribed to the Google Groups "Shiny - Web Framework for R" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shiny-discus...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Andy Chu]

On Monday, February 4, 2013 10:16:11 PM UTC-8, Joe Cheng [RStudio] wrote:

If you use Shiny Server (and you should!), you should be able to proxy it through non-websocket-supporting proxies. We use SockJS on the Shiny Server side, then Shiny Server opens a websocket connection to the backend R process (localhost-to-localhost).

Oh OK that's interesting... do you know of anyone running it in such a configuration?  From reading the code it seems like that would not work because shiny.js uses a raw web socket and not SockJS.  The browser would try to upgrade the HTTP connection to Web Socket through the proxy but fail.  But I haven't actually tried it as it requires a fair amount of sys admin work to test with this particular proxy.  I have run it locally and everything works very well.

Thanks for the response.

Andy

[Andy Chu]

To clarify, I guess you are saying that it *should* work without web
sockets. In other words, there is no fundamental reason that Shiny
couldn't just use plain AJAX (if SockJS would work). If you think it
will work I can test it with nginx as a proxy and see what happens, as
I believe nginx doesn't support web sockets right now. But if I
understand it correctly it does seem like the initial HTTP connection
won't be upgradable to Web Sockets through Nginx.

thanks,
Andy

[Joe Cheng]

shiny.js uses a raw WebSocket by default, but the mechanism is extensible and Shiny Server extends it to use SockJS. Look at the HTML source of e.g. http://glimmer.rstudio.com/jcheng/diamonds/ and do a Find for SockJS and you'll see how it's done.

I'm sure you will be able to proxy with nginx, other people (including me) have done it with Apache which also doesn't support proxying of ws and it works fine.

[Andy Chu]

On Tue, Feb 5, 2013 at 12:32 AM, Joe Cheng <j...@rstudio.com> wrote:
> shiny.js uses a raw WebSocket by default, but the mechanism is extensible
> and Shiny Server extends it to use SockJS. Look at the HTML source of e.g.
> http://glimmer.rstudio.com/jcheng/diamonds/ and do a Find for SockJS and
> you'll see how it's done.
>
> I'm sure you will be able to proxy with nginx, other people (including me)
> have done it with Apache which also doesn't support proxying of ws and it
> works fine.

OHH now I see, the shiny-server invokes apps with the SockJSAdapter.R
wrapper, which injects some HTML/JS snippets. Is there a reason it's
done that way? (Rather than the more obvious thing of having the
framework return the full HTTP response and the Node.js server be a
dumb proxy).

I've written a similar server for R web apps. What I did was just
have R open named pipes for request and response, sort of like CGI,
except the process persists across requests. (Named pipes since the R
interpreter sometimes likes to spew stuff on stdout that you don't
intend.)

That way you can just use a vanilla R interpreter and you don't need
any special C extensions. I just use netstrings to delimit requests
and responses. Then the server, which is in Python, just reads from
those named pipes, and turns it into HTTP, like a CGI server would.

But what I was missing is the JavaScript/Bootstrap part, and I think
you've done a great job on that. People were writing CGI-style R web
apps which can be tedious.

Anyway I'm sort of curious why the R process isn't just a plain HTTP
server rather than a Web Sockets server... I think the only advantage
of web sockets is doing server push? It doesn't appear Shiny uses
that as all updates are in response to a user action. On the other
side, I guess if you changed it to use pipes then people couldn't
develop without running a server as the parent of the R process.

Anyway I think we are about to become a paying RStudio customer from
what I understand (or are already), but I am not sure we could use a
Node.js server in our environment as it's quite specialized sys-admin
wise.

Andy

[Joe Cheng]

On Tuesday, February 5, 2013, Andy Chu wrote:

OHH now I see, the shiny-server invokes apps with the SockJSAdapter.R
wrapper, which injects some HTML/JS snippets.  Is there a reason it's
done that way?  (Rather than the more obvious thing of having the
framework return the full HTTP response and the Node.js server be a
dumb proxy).

Mostly, because there is no SockJS server for R (or C for that matter) and it didn't seem to be a good use of our resources to write one. The JS injection is pretty clean though and we will use it for other purposes in the future anyway.

 

Anyway I'm sort of curious why the R process isn't just a plain HTTP
server rather than a Web Sockets server... I think the only advantage
of web sockets is doing server push?  It doesn't appear Shiny uses
that as all updates are in response to a user action.

Server push is actually very important to us, updates can also be time based (invokeLater and reactiveTimer) and in the future we will also have reactive file watchers. This is important for, say, dashboard apps. You can also have a user in one session change a reactive value that causes other users' sessions to push new output to their browsers.

 

  On the other
side, I guess if you changed it to use pipes then people couldn't
develop without running a server as the parent of the R process.

Yeah, the idea is that it is crazy easy to get started--just install.packages and you're ready to play around on localhost (assuming a modern browser). When it comes to deploying an app then we think it's tolerable that you have to install some additional software. 

Anyway I think we are about to become a paying RStudio customer from
what I understand (or are already), but I am not sure we could use a
Node.js server in our environment as it's quite specialized sys-admin
wise.

Great! Do you mean node.js is not allowed by your sys admins, or that there are actual technical limitations (no Linux? very old Linux?)? Would love to hear more!

Thanks for the feedback!

--
Sent from my phone

[Andy Chu]

>> Anyway I'm sort of curious why the R process isn't just a plain HTTP
>> server rather than a Web Sockets server... I think the only advantage
>> of web sockets is doing server push? It doesn't appear Shiny uses
>> that as all updates are in response to a user action.
>
>
> Server push is actually very important to us, updates can also be time based
> (invokeLater and reactiveTimer) and in the future we will also have reactive
> file watchers. This is important for, say, dashboard apps. You can also have
> a user in one session change a reactive value that causes other users'
> sessions to push new output to their browsers.

Hm OK, but I can think of at least a dozen apps that people would want
to write that don't involve any kind of server push. Does it seem
feasible to have an option for the R process to use plain HTTP? In
that case those features would be disabled.

> Yeah, the idea is that it is crazy easy to get started--just
> install.packages and you're ready to play around on localhost (assuming a
> modern browser). When it comes to deploying an app then we think it's
> tolerable that you have to install some additional software.

Yeah that makes sense.

>> Anyway I think we are about to become a paying RStudio customer from
>> what I understand (or are already), but I am not sure we could use a
>> Node.js server in our environment as it's quite specialized sys-admin
>> wise.
>
>
> Great! Do you mean node.js is not allowed by your sys admins, or that there
> are actual technical limitations (no Linux? very old Linux?)? Would love to
> hear more!

We are not running any node.js right now, although that may be
surmountable. But more important is custom auth, and also monitoring.
Our environment is pretty specialized (and depending on the situation
Linux may be old too).

Some people are also concerned about security, since R is not really
meant to be exposed to the web. It's C code is definitely not
designed with that in mind :) So some kind of sandboxing of R
processes may also be desired.

Overall it seems like proxying the R process ourselves is the most
realistic approach, which is why I was asking about HTTP. Any
comments on that appreciated.

thanks,
Andy

[Joe Cheng]

> Does it seem feasible to have an option for the R process to use plain HTTP?

That doesn't seem like something we'd prioritize anytime soon, sorry. It would mostly benefit people who are implementing their own Shiny Server, in which case emulating websockets over HTTP is not a particularly hard problem (SockJS, socket.io, Kaazing, or just do your own Comet based impl).

> Overall it seems like proxying the R process ourselves is the most

> realistic approach, which is why I was asking about HTTP.   Any
> comments on that appreciated.

Sandboxing, custom auth, and monitoring is all coming for the enterprise version of Shiny Server later this year. If you don't want to use Node.js you'll basically have to rewrite everything we did (and will do) for Shiny Server, but in C (or whatever). In other words you'd have to really, really not want to use Node to make that a worthwhile tradeoff!

If you build Node.js from source then it's possible even pretty old versions of Linux would be supported. I can't say we've tested on anything older than a couple of years though (CentOS 5, Ubuntu 10.04). I do think it kind of unlikely that there'd be a setup that'd be able to support R 2.15 but not Node, as the latter has FAR fewer dependencies.

Andy

[Andy Chu]

> Sandboxing, custom auth, and monitoring is all coming for the enterprise
> version of Shiny Server later this year. If you don't want to use Node.js
> you'll basically have to rewrite everything we did (and will do) for Shiny
> Server, but in C (or whatever). In other words you'd have to really, really
> not want to use Node to make that a worthwhile tradeoff!
>
> If you build Node.js from source then it's possible even pretty old versions
> of Linux would be supported. I can't say we've tested on anything older than
> a couple of years though (CentOS 5, Ubuntu 10.04). I do think it kind of
> unlikely that there'd be a setup that'd be able to support R 2.15 but not
> Node, as the latter has FAR fewer dependencies.

OK thanks for the answer. Related question: why does shiny require R
2.15? I had to build my own R to try it. At work we're using R 2.14.
And I use an Ubuntu Natty machine which isn't very old, and it uses
2.12.

thanks,
Andy