500 error when deploying to shinyapps.io when behind a HTTP proxy

[jb2cool]

I have followed the Getting started with shinyapps.io (https://shiny.rstudio.com/articles/shinyapps.html) tutorial and when bypassing my proxy server and going out to the internet directly everything works just fine when deploying/publishing the app. Now in our corporate environment I really ought to sit behind the proxy so I closed the firewall rules and set the http_proxy and https_proxy variables and it bombs out with a 500 error. Unfortunately I can't really get much info out of the proxy as to what's failing so thought I should ask here if anyone else has seen similar issues and if so, how did they resolve them. I'm not super confident though as I've seen a couple of threads mentioning proxies and they just seemed to give up in the end and open the firewall.

I'm not doing anything particularly fancy, I just have a pfSense proxy (based on squid) running on port 8080. This should be a pretty typical setup.

[Joshua Spiewak]

Could you share the application url or its name and the account name?

Could you share a screen shot of the 500 error?

[jb2cool]

Application URL is http://jb2cool.shinyapps.io/diamonds. This is just the demo app from here (https://shiny.rstudio.com/articles/shinyapps.html). As far as i can see the app works just fine once it's been uploaded my issue seems to be related to the upload process.

I'll try and get you a screenshot but i'll have to build up a fresh machine for this.

Jason

[Joshua Spiewak]

So once the application is uploaded the proxy is not an issue, it is only when deploying the application?

I suggest setting a number of the rsconnect logging options: options(rsconnect.http.trace = TRUE), options(rsconnect.http.verbose = TRUE), options(rsconnect.http.trace.json = TRUE)

And then run rsconnect::deployApp() and send me the output.

[jb2cool]

Joshua,
Sorry it's taken so long for me to reply, here are the outputs you requested;

> options(rsconnect.http.trace = TRUE)
> options(rsconnect.http.verbose = TRUE)
> options(rsconnect.http.trace.json = TRUE)
> rsconnect::deployApp()
*   Trying 10.85.4.54...
* Connected to 10.85.4.54 (10.85.4.54) port 8080 (#0)
* Establish HTTP proxy tunnel to api.shinyapps.io:443
> CONNECT api.shinyapps.io:443 HTTP/1.1
Host: api.shinyapps.io:443
User-Agent: rsconnect/0.8.5
Proxy-Connection: Keep-Alive
Date: Tue, 10 Oct 2017 13:30:54 GMT
X-Auth-Token: C2FF02FD5959A1043740253C721A5D55
X-Auth-Signature: MjYzMDY2MzBlZjA0ZDcxNjczZmQzYjIzMTEyYzUzNmUxNzVjNzA2Mjc3ZDdjMTI5MDAzYTM4MzM4NjcxZTQxNQ==; version=1
X-Content-Checksum: d41d8cd98f00b204e9800998ecf8427e

< HTTP/1.1 200 Connection established
< 
* Proxy replied OK to CONNECT request
* successfully set certificate verify locations:
*   CAfile: C:/Users/SIS/Documents/R/win-library/3.4/rsconnect/cert/cacert.pem
  CApath: none
* SSL connection using TLSv1.0 / ECDHE-RSA-AES128-SHA
* Server certificate:
* 	 subject: CN=api.rstudio.cloud
* 	 start date: 2017-03-16 00:00:00 GMT
* 	 expire date: 2018-04-16 12:00:00 GMT
* 	 subjectAltName: api.shinyapps.io matched
* 	 issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
* 	 SSL certificate verify ok.
> GET /v1/applications/?filter=account_id:98018&filter=name:diamond2&count=100&offset=0 HTTP/1.1
User-Agent: rsconnect/0.8.5
Host: api.shinyapps.io
Accept: */*
Date: Tue, 10 Oct 2017 13:30:54 GMT
X-Auth-Token: C2FF02FD5959A1043740253C721A5D55
X-Auth-Signature: MjYzMDY2MzBlZjA0ZDcxNjczZmQzYjIzMTEyYzUzNmUxNzVjNzA2Mjc3ZDdjMTI5MDAzYTM4MzM4NjcxZTQxNQ==; version=1
X-Content-Checksum: d41d8cd98f00b204e9800998ecf8427e

< HTTP/1.1 200 OK
< Date: Tue, 10 Oct 2017 13:30:52 GMT
< Content-Type: application/json; charset=UTF-8
< Content-Length: 531
< Connection: keep-alive
< Server: nginx/1.4.6 (Ubuntu)
< Etag: "a08d964e87969d27de05d04e8ab5e160a47ff81d"
< Access-Control-Allow-Origin: https://rstudio.cloud
< 
* Connection #0 to host 10.85.4.54 left intact
GET /v1/applications/?filter=account_id:98018&filter=name:diamond2&count=100&offset=0 529ms
*   Trying 10.85.4.54...
* Connected to 10.85.4.54 (10.85.4.54) port 8080 (#0)
* Establish HTTP proxy tunnel to api.shinyapps.io:443
> CONNECT api.shinyapps.io:443 HTTP/1.1
Host: api.shinyapps.io:443
User-Agent: rsconnect/0.8.5
Proxy-Connection: Keep-Alive
Date: Tue, 10 Oct 2017 13:30:59 GMT
X-Auth-Token: C2FF02FD5959A1043740253C721A5D55
X-Auth-Signature: ODk4MDQ0MDhiNzViMWFjOGNjM2MxOTQ0N2RlOGMzNGZiYmU1ZDFiMzIzY2FiZTU1NzRlOWY4YTMxODg0NGNiNw==; version=1
X-Content-Checksum: 7dacce3ccbf0616571f0cf22e83e32d5
Content-Type: application/json

< HTTP/1.1 200 Connection established
< 
* Proxy replied OK to CONNECT request
* successfully set certificate verify locations:
*   CAfile: C:/Users/SIS/Documents/R/win-library/3.4/rsconnect/cert/cacert.pem
  CApath: none
* SSL connection using TLSv1.0 / ECDHE-RSA-AES128-SHA
* Server certificate:
* 	 subject: CN=api.rstudio.cloud
* 	 start date: 2017-03-16 00:00:00 GMT
* 	 expire date: 2018-04-16 12:00:00 GMT
* 	 subjectAltName: api.shinyapps.io matched
* 	 issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
* 	 SSL certificate verify ok.
> POST /v1/bundles HTTP/1.1
User-Agent: rsconnect/0.8.5
Host: api.shinyapps.io
Accept: */*
Date: Tue, 10 Oct 2017 13:30:59 GMT
X-Auth-Token: C2FF02FD5959A1043740253C721A5D55
X-Auth-Signature: ODk4MDQ0MDhiNzViMWFjOGNjM2MxOTQ0N2RlOGMzNGZiYmU1ZDFiMzIzY2FiZTU1NzRlOWY4YTMxODg0NGNiNw==; version=1
X-Content-Checksum: 7dacce3ccbf0616571f0cf22e83e32d5
Content-Type: application/json
Content-Length: 143
Expect: 100-continue

< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 201 Created
< Date: Tue, 10 Oct 2017 13:30:57 GMT
< Content-Type: application/json; charset=UTF-8
< Content-Length: 1446
< Connection: keep-alive
< Server: nginx/1.4.6 (Ubuntu)
< Access-Control-Allow-Origin: https://rstudio.cloud
< 
* Connection #0 to host 10.85.4.54 left intact
POST /v1/bundles 539ms
<< {
	"application" : 223579,
	"content_type" : "application/x-tar",
	"content_length" : 28690,
	"checksum" : "9c77afb9d2d296fc081a6c39ffefd176"
}
*   Trying 10.85.4.54...
* Connected to 10.85.4.54 (10.85.4.54) port 8080 (#0)
* Establish HTTP proxy tunnel to shinyapps-upload.s3.amazonaws.com:443
> CONNECT shinyapps-upload.s3.amazonaws.com:443 HTTP/1.1
Host: shinyapps-upload.s3.amazonaws.com:443
User-Agent: rsconnect/0.8.5
Proxy-Connection: Keep-Alive
Content-Type: application/x-tar
Content-Length: 28690
Content-MD5: nHevudLSlvwIGmw5/+/Rdg==

< HTTP/1.1 500 Internal Server Error
< Server: squid
< Mime-Version: 1.0
< Date: Tue, 10 Oct 2017 13:30:57 GMT
< Content-Type: text/html;charset=utf-8
< Content-Length: 3098
< X-Squid-Error: ERR_ICAP_FAILURE 0
< Vary: Accept-Language
< Content-Language: en
< X-Cache: MISS from SRV-32
< X-Cache-Lookup: NONE from SRV-32:8080
< Via: 1.1 SRV-32 (squid)
< Connection: keep-alive
< 
* Received HTTP code 500 from proxy after CONNECT
* Closing connection 0
Error in function (type, msg, asError = TRUE)  : 
  Received HTTP code 500 from proxy after CONNECT
Timing stopped at: 0.02 0 0.02

[Joshua Spiewak]

It would appear that when it comes time to upload the bundle (which we do directly to a pre-signed URL to S3), something fails in your squid server: X-Squid-Error: ERR_ICAP_FAILURE 0

It looks like you might be using Web Safety ICAP Server in conjunction with squid. So I would make sure that you are on current versions and have things configured properly there.

--
You received this message because you are subscribed to the Google Groups "shinyapps.io Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shinyapps-use...@googlegroups.com.
To post to this group, send email to shinyap...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/shinyapps-users/b35b8ead-6b75-4dc1-8b96-2294c5d0c1cc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[jb2cool]

I thought that might be the suggestion and my clamav and all packages are up to date on my proxy (pfSense). This is working for everything else whether it be R/RStudio, web browsers, Linux package managers so unsure that there is really anything wrong with my proxy, yes i can see that the trace is saying that the AV is not happy with this but don't know why that would be the case when it's happy with everything else.

On Tuesday, 12 September 2017 18:22:36 UTC+1, jb2cool wrote:

[Joshua Spiewak]

Is there anything in its logs?

Linux package managers are downloading items, i.e. HTTP GET. In this case the call is an HTTP PUT with a 28k body. What happens when you try to upload a file to S3 with the proxy in place, e.g. "aws s3 cp my.tar.gz s3://my-s3-bucket" ?

--

You received this message because you are subscribed to the Google Groups "shinyapps.io Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shinyapps-use...@googlegroups.com.
To post to this group, send email to shinyap...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/shinyapps-users/ebd1fafc-5ee2-48e4-a303-b42867edda4f%40googlegroups.com.