Shiny apps don't work over nginx proxy using https

[Vasant Marur]

0down votefavorite

I am trying to secure access to some shiny apps being served over our web server via https and using auth0 for login. Shiny apps don't work over https, though they work over http. System info: nginx/1.10.2 Shiny Server v1.5.3.838 Node.js v6.10.0 CentOS Linux release 7.3.1611 (Core) console.log link https://gist.github.com/vasantm/4339b248143fa6b3f8c4d0f1773f33e9 site.conf link https://gist.github.com/vasantm/d98c8ae9de8a8667d9d2e6e59945de62 shiny-server.conf link https://gist.github.com/vasantm/43a94acd7b9fcf573cdea2a200218425. I used this reference to configure nginx and shiny Running Shiny Server with a Proxy Any clues to get shiny working over SSL?

also posted on https://stackoverflow.com/questions/44397818/shiny-apps-greyed-out-nginx-proxy-over-ssl

[Alan Dipert]

Hi Vasant,

Your site.conf appears to have a hanging quote: https://gist.github.com/vasantm/d98c8ae9de8a8667d9d2e6e59945de62#file-sites-conf-L41

It's also missing the "map" rule in the nginx configuration on https://support.rstudio.com/hc/en-us/articles/213733868-Running-Shiny-Server-with-a-Proxy.

That said, I'm surprised nginx started up at all. I was able to visit your https url and observe the wss:// connection problem, and see nginx's "400 Bad Request" response.

If fixing these errors in site.conf doesn't work, the next place to look would be nginx's error logs.

Alan

[Vasant Marur]

Hey Alan,

Yup I had fixed that earlier.

Here's the updated config, I put it all in /etc/nginx/nginx.conf. Earlier I had the http directive in nginx.conf and a sites.conf under /etc/nginx/conf.d.

http {

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '

                      '$status $body_bytes_sent "$http_referer" '

                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    error_log /var/log/nginx/error.log warn;

    sendfile            on;

    tcp_nopush          on;

    tcp_nodelay         on;

    keepalive_timeout   65;

    types_hash_max_size 2048;

    include             /etc/nginx/mime.types;

    default_type        application/octet-stream;

    # Load modular configuration files from the /etc/nginx/conf.d directory.

    # See http://nginx.org/en/docs/ngx_core_module.html#include

    # for more information.

    include /etc/nginx/conf.d/*.conf;

  map $http_upgrade $connection_upgrade {

      default upgrade;

      ''      close;

    }

 server {

        listen 80 default_server;

        listen [::]:80 default_server;

        server_name kristallab.bwh.harvard.edu;

        return 301 https://$server_name$request_uri;

location / {

        #rewrite ^/shiny/(.*)$ /$1 break;

        proxy_pass http://127.0.0.1:3838;

        proxy_redirect http://127.0.0.1:3838/ $scheme://$server_name/;

        proxy_http_version 1.1;

        proxy_set_header Upgrade $http_upgrade;

        proxy_set_header Connection $connection_upgrade;

        proxy_read_timeout 20d;

    }

}# Settings for a TLS enabled server.

#

    server {

        #SSL config

        listen       443 ssl http2 default_server;

        listen       [::]:443 ssl http2 default_server;

        ssl_certificate /etc/pki/tls/certs/kristallab.bwh.harvard.edu_bundle.crt;

        ssl_certificate_key /etc/pki/tls/private/kristallab.bwh.harvard.edu.key;

        ssl_session_cache shared:SSL:10m;

        ssl_session_timeout  10m;

        ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128:AES256:AES:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK';

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

        ssl_prefer_server_ciphers on;

        ssl_stapling on; # Requires nginx >= 1.3.7

        ssl_stapling_verify on; # Requires nginx => 1.3.7

        add_header Strict-Transport-Security max-age=15768000;

        #Server name is our external domain name

        server_name  kristallab.bwh.harvard.edu;

        # Root location

        root         /usr/share/nginx/html;

        # files to serve

        index index.html index.htm

        #Load configuration files for the default server block.

        include /etc/nginx/default.d/*.conf;

        access_log /var/log/nginx/our_site_443.log main;

        error_log /var/log/nginx/our_site_443_err.log warn;

        # Open shiny server - for public apps

location / {

        #rewrite ^/shiny/(.*)$ /$1 break;

        proxy_set_header    Host $server_name;

        proxy_set_header    X-Real-IP $remote_addr;

        proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_set_header    X-Forwarded-Proto $scheme;

        proxy_pass http://127.0.0.1:3838;

         #proxy_http_version 1.1;

         #proxy_set_header Upgrade $http_upgrade;

         #proxy_set_header Connection $connection_upgrade;

        proxy_read_timeout 90;

        proxy_redirect http://127.0.0.1:3838/ $scheme://$server_name/;

        }

I still get greyed out apps. I then uncommented the highlighted part of the config above and still got errors.

here's the latest console.log

WebSocket connection to 'wss://kristallab.bwh.harvard.edu/sample-apps/rmd/__sockjs__/n=DDUjuaZmdQBFbAe8Ni/345/w1oqw659/websocket' failed: Error during WebSocket handshake: Unexpected response code: 400WrappedWebSocket @ VM220:35

VM221:35 WebSocket connection to 'wss://kristallab.bwh.harvard.edu/sample-apps/hello/__sockjs__/n=NCXw2RGlge88iybomV/805/n61pj5fb/websocket' failed: Error during WebSocket handshake: Unexpected response code: 400WrappedWebSocket @ VM221:35

shiny-server-client.min.js:formatted:935Wed Jun 07 2017 18:08:39 GMT-0400 (EDT) [INF]: Error: WebSocket closed

shiny-server-client.min.js:formatted:935Wed Jun 07 2017 18:08:39 GMT-0400 (EDT) [INF]: Connection closed. Info: {"isTrusted":false}

shiny-server-client.min.js:formatted:265Wed Jun 07 2017 18:08:39 GMT-0400 (EDT) [DBG]: SockJS connection closed

shiny-server-client.min.js:formatted:265Wed Jun 07 2017 18:08:39 GMT-0400 (EDT) [DBG]: Channel 0 is closed

shiny-server-client.min.js:formatted:265Wed Jun 07 2017 18:08:39 GMT-0400 (EDT) [DBG]: Removed channel 0, 0 left

shiny-server-client.min.js:1 Wed Jun 07 2017 18:08:39 GMT-0400 (EDT) [INF]: Error: WebSocket closed

shiny-server-client.min.js:1 Wed Jun 07 2017 18:08:39 GMT-0400 (EDT) [INF]: Connection closed. Info: {"isTrusted":false}

shiny-server-client.min.js:1 Wed Jun 07 2017 18:08:39 GMT-0400 (EDT) [DBG]: SockJS connection closed

shiny-server-client.min.js:1 Wed Jun 07 2017 18:08:39 GMT-0400 (EDT) [DBG]: Channel 0 is closed

shiny-server-client.min.js:1 Wed Jun 07 2017 18:08:39 GMT-0400 (EDT) [DBG]: Removed channel 0, 0 left 

[Alan Dipert]

Vasant,

I haven't tried it myself but that configuration does look reasonable.

I think a good next step would be to extract from nginx why exactly it is serving that 400. One way to do this is to configure logging at the info level, by replacing this line:

error_log /var/log/nginx/error.log warn;

with this one:

error_log /var/log/nginx/error.log info;

Then, reload nginx and reproduce the problem, and see if anything illuminating turns up in /var/log/nginx/error.log

Alan

[Vasant Marur]

Alan,

thank you for taking the time.

I set the error_log to info as you suggested.

Here's the snippet from the error log

2017/06/08 12:33:05 [info] 17856#0: *6 client 170.223.178.73 closed keepalive connection

and my access log is at https://gist.github.com/vasantm/450ee5c42f97541786497c56e1f3249a#file-access-log

Also wondering if this could be causing the issue

the server has 2 IPs one external as seen above in the logs and another internal one.

so the set up is

client ---->URL [external IP] --> server[on which shiny-server and nginx as proxy are located]

[Alan Dipert]

Vasant,

Have you had any luck since your last message?

I don't see anything obviously strange going on in the access log. I'm surprised by the brevity of the error log -- I expected more there.

One additional thing I noticed is that there is this in your configuration: include /etc/nginx/conf.d/*.conf; Is it possible other configuration is being loaded atop yours?

I think a good next step could be to reduce your configuration to exactly what's on the "Shiny Server: Running with a Proxy" and going from there.

Alan

[Vasant Marur]

Hey Alan,

Nope no luck and it moved down the priority list as I had analysis to get done.

I tried it again with the reduced config as you suggested and still the same.

The shiny apps load fine over http but not over https. They load and turn grey.

I turned error to level info for SSL and this stood out

2017/06/21 17:34:43 [error] 26426#0: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 170.223.178.73, server: kristallab.bwh.harvard.edu, request: "GET /sample-apps/rmd/ HTTP/1.1", upstream: "http://[::1]:3838/sample-apps/rmd/", host: "kristalweb.dipr.partners.org", referrer: "https://kristallab.bwh.harvard.edu/"

2017/06/21 17:34:43 [warn] 26426#0: *3 upstream server temporarily disabled while connecting to upstream, client: 170.223.178.73, server: kristallab.bwh.harvard.edu, request: "GET /sample-apps/rmd/ HTTP/1.1", upstream: "http://[::1]:3838/sample-apps/rmd/", host: "kristalweb.dipr.partners.org", referrer: "https://kristallab.bwh.harvard.edu/"

2017/06/21 17:34:57 [error] 26426#0: *10 connect() failed (111: Connection refused) while connecting to upstream, client: 170.223.178.73, server: kristallab.bwh.harvard.edu, request: "GET /sample-apps/rmd/shared/shiny.min.js HTTP/1.1", upstream: "http://[::1]:3838/sample-apps/rmd/shared/shiny.min.js", host: "kristalweb.dipr.partners.org", referrer: "https://kristallab.bwh.harvard.edu/sample-apps/rmd/"

2017/06/21 17:34:57 [warn] 26426#0: *10 upstream server temporarily disabled while connecting to upstream, client: 170.223.178.73, server: kristallab.bwh.harvard.edu, request: "GET /sample-apps/rmd/shared/shiny.min.js HTTP/1.1", upstream: "http://[::1]:3838/sample-apps/rmd/shared/shiny.min.js", host: "kristalweb.dipr.partners.org", referrer: "https://kristallab.bwh.harvard.edu/sample-apps/rmd/"

any clues?

About me -->http://vasantmarur.net

--
You received this message because you are subscribed to a topic in the Google Groups "Shiny - Web Framework for R" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/shiny-discuss/-E2pm6VJxRs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to shiny-discuss+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/shiny-discuss/05d8225c-8284-4e32-93bc-235c47eff262%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Joe Cheng]

Looks like it's trying to use IPv6 ([::1]) to connect to Shiny Server instead of IPv4 (127.0.0.1), we only support IPv4 currently. Maybe this will help?

https://serverfault.com/questions/527317/disable-ipv6-in-nginx-proxy-pass

To unsubscribe from this group and all its topics, send an email to shiny-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/shiny-discuss/05d8225c-8284-4e32-93bc-235c47eff262%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Shiny - Web Framework for R" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shiny-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/shiny-discuss/CA%2B2i%3DBWaYEK79q_foeOP6iPrXeiuYmnvHZDuYQziUGjtNvR1FA%40mail.gmail.com.

[Joe Cheng]

Or maybe try removing "http2" from the listen directives, that is one difference between the http and https configs that seems a little suspicious.