BigQuery API requests for queries giving 404 on Shinyapps.io but 200 locally

119 views
Skip to first unread message

Mark Edmondson

unread,
May 3, 2018, 7:57:53 AM5/3/18
to shinyapps.io Users
Hello Shinyapps team,

I have a vexing bug reported that I can't fathom why is happening.  Users report they can authenticate and perform other API calls such as requesting lists of tables, but when performing a query they get a 404 back.  The exact same Shiny app locally works as expected.

This behaviour looks to have started recently, so I wonder if anything could have changed on shinyapps.io that may have prompted it.

An example app demonstrating the bug is deployed here https://mark.shinyapps.io/bq-bug/

I took out the logs and compared to local requests via httr verbose logging, and they both look identical aside the 404 when requested from shinyapps.io - see below.

Does anyone have a clue what may be going on?  I.P. filtering is not possible by BigQuery, and its not authentication as other API calls can be performed.

Yours sincerely,
Mark

Logs below:

# Request on Shinyapps.io - 404 Not Found

2018-05-03T11:41:02.568644+00:00 shinyapps[336676]: 2018-05-03 11:41:02> Request: https://www.googleapis.com/bigquery/v2/projects/suri-bq/queries/
2018-05-03T11:41:02.576952+00:00 shinyapps[336676]: 2018-05-03 11:41:02> Body JSON parsed to: {"kind":"bigquery#queryRequest","query":"SELECT EXACT_COUNT_DISTINCT(Campaign) FROM [suri-bq:suridata.AD_PERFORMANCE_REPORT] WHERE CustomerId ='1234' AND Network = 'Search Network'","maxResults":1000,"useLegacySql":true,"useQueryCache":true,"defaultDataset":{"datasetId":"suridata","projectId":"suri-bq"}}
2018-05-03T11:41:02.580921+00:00 shinyapps[336676]: -> POST /bigquery/v2/projects/suri-bq/queries/ HTTP/1.1
2018-05-03T11:41:02.580925+00:00 shinyapps[336676]: -> Host: www.googleapis.com
2018-05-03T11:41:02.580923+00:00 shinyapps[336676]: -> User-Agent: googleAuthR/0.6.2 (gzip)
2018-05-03T11:41:02.580925+00:00 shinyapps[336676]: -> Accept: application/json, text/xml, application/xml, */*
2018-05-03T11:41:02.580928+00:00 shinyapps[336676]: ->
2018-05-03T11:41:02.580926+00:00 shinyapps[336676]: -> Content-Type: application/json
2018-05-03T11:41:02.580926+00:00 shinyapps[336676]: -> Accept-Encoding: gzip
2018-05-03T11:41:02.580927+00:00 shinyapps[336676]: -> Authorization: Bearer XXXXXXXXXXX
2018-05-03T11:41:02.580928+00:00 shinyapps[336676]: -> Content-Length: 311
2018-05-03T11:41:02.667620+00:00 shinyapps[336676]: <- Date: Thu, 03 May 2018 11:41:02 GMT
2018-05-03T11:41:02.667745+00:00 shinyapps[336676]: <- Content-Type: text/html; charset=UTF-8
2018-05-03T11:41:02.581034+00:00 shinyapps[336676]: >> {"kind":"bigquery#queryRequest","query":"SELECT EXACT_COUNT_DISTINCT(Campaign) FROM [suri-bq:suridata.AD_PERFORMANCE_REPORT] WHERE CustomerId ='1234' AND Network = 'Search Network'","maxResults":1000,"useLegacySql":true,"useQueryCache":true,"defaultDataset":{"datasetId":"suridata","projectId":"suri-bq"}}
2018-05-03T11:41:02.581057+00:00 shinyapps[336676]:
2018-05-03T11:41:02.667508+00:00 shinyapps[336676]: <- HTTP/1.1 404 Not Found
2018-05-03T11:41:02.667858+00:00 shinyapps[336676]: <- Server: ESF
2018-05-03T11:41:02.667971+00:00 shinyapps[336676]: <- Content-Length: 1598
2018-05-03T11:41:02.668081+00:00 shinyapps[336676]: <- X-XSS-Protection: 1; mode=block
2018-05-03T11:41:02.668190+00:00 shinyapps[336676]: <- X-Frame-Options: SAMEORIGIN
2018-05-03T11:41:02.668294+00:00 shinyapps[336676]: <- X-Content-Type-Options: nosniff


# Request Locally - 200 OK

2018-05-03 13:49:27> Request: https://www.googleapis.com/bigquery/v2/projects/suri-bq/queries/
2018-05-03 13:49:27> Body JSON parsed to: {"kind":"bigquery#queryRequest","query":"SELECT EXACT_COUNT_DISTINCT(Campaign) FROM [suri-bq:suridata.AD_PERFORMANCE_REPORT] WHERE CustomerId ='1234' AND Network = 'Search Network'","maxResults":1000,"useLegacySql":true,"useQueryCache":true,"defaultDataset":{"datasetId":"suridata","projectId":"suri-bq"}}
-> POST /bigquery/v2/projects/suri-bq/queries/ HTTP/1.1
-> Host: www.googleapis.com
-> User-Agent: googleAuthR/0.6.2 (gzip)
-> Accept: application/json, text/xml, application/xml, */*
-> Content-Type: application/json
-> Accept-Encoding: gzip
-> Authorization: Bearer XXXXXXXXX
-> Content-Length: 311
->
>> {"kind":"bigquery#queryRequest","query":"SELECT EXACT_COUNT_DISTINCT(Campaign) FROM [suri-bq:suridata.AD_PERFORMANCE_REPORT] WHERE CustomerId ='1234' AND Network = 'Search Network'","maxResults":1000,"useLegacySql":true,"useQueryCache":true,"defaultDataset":{"datasetId":"suridata","projectId":"suri-bq"}}

<- HTTP/1.1 200 OK
<- Cache-Control: no-cache, no-store, max-age=0, must-revalidate
<- Pragma: no-cache
<- Expires: Mon, 01 Jan 1990 00:00:00 GMT

Mark Edmondson

unread,
May 3, 2018, 8:28:27 AM5/3/18
to shinyapps.io Users
Could it be the gap that has been introduced here?:


2018-05-03T11:41:02.580925+00:00 shinyapps[336676]: -> Accept: application/json, text/xml, application/xml, */*
2018-05-03T11:41:02.580928+00:00 shinyapps[336676]: ->
2018-05-03T11:41:02.580926+00:00 shinyapps[336676]: -> Content-Type: application/json

Why would that appear on shinyapps.io ?

Joshua Spiewak

unread,
May 7, 2018, 9:10:15 AM5/7/18
to shinyapps.io Users
I haven't been able to find anything to indicate what might be happening, and I can't think of any recent change that could have this sort of effect.
Could you include the output of sessionInfo() and rsconnect::appDependencies()?
Are you able to prove that the same credentials are being used locally and on shinyapps.io?

Mark Edmondson

unread,
Jun 8, 2018, 9:07:15 AM6/8/18
to shinyapps.io Users
Just following up on this, it turned out the BigQuery API stopped supporting trailing slashes for this call, so once that was removed it works again.
Reply all
Reply to author
Forward
0 new messages