503 errors uploading to GCS via blobstore API

[Jeff Schnitzer]

I'm setting up a demo environment on a new appid and running into some problems with GCS. Unless otherwise mentioned, all of this is using the new console.

The first strange thing is that a default bucket was not created. When making appids before, the default bucket was created automatically. With a little digging I found a button in the old console App Settings that added a default GCS bucket, and that seemed to work. I have a default bucket named gearlaunch-hub-demo.appspot.com.

Now when I try to upload (using the same code that works on other appids), I get a 503 error on the client. There's nothing in the logs about this error.

Any idea what's up, or how I can get uploads working?

My appid is gearlaunch-hub-demo. The options passed to createUploadUrl are:

UploadOptions.Builder.withGoogleStorageBucketName(AppIdentityServiceFactory.getAppIdentityService().getDefaultGcsBucketName())

Thanks,

Jeff

[Patrice (Cloud Platform Support)]

Hi Jeff,

Just doing the normal checks : Is billing enabled on the project? GCS needs billing, so it's possible you get 503 if it isn't (or if your daily budget is still at 0, the default value).

Cheers

[Jeff Schnitzer]

Billing is enabled with a valid card. There's also plenty of the startup credit. With a little hunting I found the Daily Budget setting which was 0, I upped that. Still getting 503 - should it recognize the change immediately?

This budget setting has not been necessary in the other two test environments that I have deployed this code to, which are still working fine. What changed?

Thanks,

Jeff

[Joseph Letness]

Hi Jeff,

Does your app-id have permission to write to the bucket? I ran into this issue just a few days ago and I needed to explicitly give my app write access to the api by adding the app's service account (your-...@appspot.gserviceaccount.com) in the Permissions tab. That did the trick, no more 503 errors.

Hope this helps.

Joe

[Patrice (Cloud Platform Support)]

Jeff, I wasn't sure that the daily budget was the culprit here, but I can tell you that for certain APIs, your account is considered "free" if you don't have billing enabled AND a daily budget > 0. This is why I wanted to make sure your account was properly configured first. If this was the issue and you changed it, the change is immediate.

Joseph has a very valid point here. Does the ACL of your application contains the service account of the other?

[Jeff Schnitzer]

Yep, gearlaunc...@appspot.gserviceaccount.com is listed in the permissions with Can Edit.

Comparing against the appids that work, however, I notice that the working appid has a permission granted to PROJEC...@cloudservices.gserviceaccount.com. I tried adding that to the new app but it is in "Invitation sent" state (and uploads still produce 503).

I'm stumped.

Jeff

[Jeff Schnitzer]

One more data point. 

Trying to upload a file via the Storage Browser gives me "Backend Error":

http://monosnap.com/image/0pWxvlS0jXgGtGsryv5E3zgnMMpfm9.png

This bucket seems to be broken somehow. How can I get it fixed?

Thanks,

Jeff

[Patrice (Cloud Platform Support)]

Hmmm that is very peculiar. Is it something that is reproducible on OTHER buckets in the same project?

Because as you say, this bucket seems to be completely off. If you can reproduce on another bucket in the same app, this might be something else.

Cheers, and have a good weekend :)

[Jeff Schnitzer]

Other buckets seem to work fine. However, I need the bucket returned from AppIdentityServiceFactory.getAppIdentityService().getDefaultGcsBucketName(), otherwise my build and deploy process gets complicated.

I just created a new appid in the new console, -demo2, and uploaded my code. This did not create a GCS default bucket, which seems contrary to what I read here: https://cloud.google.com/appengine/kb/general#cloudintegration

I went to the old console App Settings and clicked the "Enable Cloud Integration" button. That created the default bucket. But I'm back to the same problem with the previous appid - I can't upload to this bucket, not even from the console.

AFAICT, there's no way to create a new appid and use the default bucket. It's just broken across the board. This means I can't create any new environments, and I'm starting to run into problems running client demos on my developer sandbox environment...

Help,

Jeff

[Jeff Schnitzer]

This feels like a GAE bug. I filed this issue:

https://code.google.com/p/googleappengine/issues/detail?id=12140

Thanks,

Jeff

[Michael David Pedersen]

I encountered this issue too, but Joe's suggestion to add the app to the GCS bucket's permissions resolved it for me.

To be specific, I added a new "User" entity with name "app...@appspot.gserviceaccount.com" and with "Owner" access to the "app_id.appspot.com" bucket, via console.developers.google.com.

[Jeff Schnitzer]

Thanks! That seemed to get things working. I actually never noticed the little dropdown menu that let me set up bucket-specific ACLs - I thought Joe was referring to the Permissions project page (sorry Joe).

However, this permission is not present on my older, working appids... something is broken somewhere.

Jeff

[husayt]

Jeff, I forgot about my pains with cloudstorage after finding Cloudberry's explorer for GCS.

http://www.cloudberrylab.com/free-google-storage-explorer.aspx

It's a must if you need to access your files regularly in a efficient way.

PS. I am not anyway affiliated with them. Just know the pain of standard GCS tools.

[Jeff Schnitzer]

Cool. My needs are strictly programmatic, but I'll keep that in mind for the future.

Jeff