Uploads to Blobstore failing

[Shaun Budhram]

Hi,

I have an older application using the Blobstore API.  I think I inadvertently upgraded to Google Cloud Storage when I used 'gcloud' to deploy the app.  A new storage bucket <appid>.appspot.com was created.

The problem I'm having is that whenever my app uses 'blobstore.create_upload_url' to create an upload URL, the the upload/redirect to the handler is now failing.  I searched the forum and found this post:

https://groups.google.com/g/google-appengine/c/0-U4xe0Dp0M/m/Lxc9AAw2AQAJ

This line in particular:

It created a new app-id.appspot.com bucket but my app engine service account did not have permissions. Once I gave it permission it started working again with the blobstore items showing up in the new "default" bucket.

This suggests that my app needs permission to the storage bucket, which didn't happen automatically.  

My question is, is this a code-level change that I need to perform when creating the upload url, or is this a configuration somewhere in the web console?  Does anyone know how I can link permissions between this storage bucket and my app engine service account for this app? 

Any help is greatly appreciated.  Thanks!

[Shaun Budhram]

Also for reference, this is what the permissions on the the bucket look like.  I think these look correct, but I'm not sure?  These were created automatically (except for my name, which I added as a test)

[Joshua Smith]

Search the list archives last October for this subject "Uploads to Legacy Blobstore are Failing with 500, nothing in the logs"

I ended up giving up and switching to new blobstore, but at the end of the thread, I think someone did identify the permission magic to fix the problem.

-Joshua

On Mar 24, 2021, at 12:10 PM, Shaun Budhram <shaunb...@gmail.com> wrote:

Also for reference, this is what the permissions on the the bucket look like.  I think these look correct, but I'm not sure?  These were created automatically (except for my name, which I added as a test)

<Screen Shot 2021-03-24 at 9.09.48 AM.png>

On Wednesday, March 24, 2021 at 8:15:04 AM UTC-7 Shaun Budhram wrote:

Hi,

I have an older application using the Blobstore API.  I think I inadvertently upgraded to Google Cloud Storage when I used 'gcloud' to deploy the app.  A new storage bucket <appid>.appspot.com was created.

The problem I'm having is that whenever my app uses 'blobstore.create_upload_url' to create an upload URL, the the upload/redirect to the handler is now failing.  I searched the forum and found this post:

https://groups.google.com/g/google-appengine/c/0-U4xe0Dp0M/m/Lxc9AAw2AQAJ

This line in particular:

It created a new app-id.appspot.com bucket but my app engine service account did not have permissions. Once I gave it permission it started working again with the blobstore items showing up in the new "default" bucket.

This suggests that my app needs permission to the storage bucket, which didn't happen automatically.  

My question is, is this a code-level change that I need to perform when creating the upload url, or is this a configuration somewhere in the web console?  Does anyone know how I can link permissions between this storage bucket and my app engine service account for this app? 

Any help is greatly appreciated.  Thanks!

-- 
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/546775b5-e326-4bed-a4bc-0d74adead697n%40googlegroups.com.

[Shaun Budhram]

Hi Joshua!  Yes, this is the exact thread I'm referencing in the initial post.  I saw someone mentioned there was a solution in that thread with setting permissions, but they didn't give any specifics on exactly what needed to be set.  If anyone knows what exactly 'permission magic' is needed to resolve this, it would be really helpful.  

The screenshot I included in the original post shows the permissions I have for the bucket.  To me, it looks like 'Owners of the project: puzzleflow-dev' should cover the necessary permission.  But, I must be missing something.

Thanks!

[George (Cloud Platform Support)]

You are right, owners of the project are expected to enjoy the needed permission. More specifically, the service account should be granted the objects.create permission per bucket.

[Shaun Budhram]

Thanks George for the comment.  How can I verify that permission is on?  I tried turning on 'Uniform' access control, and that didn't work.  

Also, as a test, I made the container 'Public' - meaning I added 'allUsers' as a 'Storage Legacy Bucket Owner', and a 'Storage Legacy Object Owner'.  This worked!  But, I really don't want this public.  I just want my app to be able to use it.  So, I guess I just need to figure out what's misconfigured here and how to fix it.

[George (Cloud Platform Support)]

To determine which service account is involved, you may check Listing service accounts. To grant to your newly identified service account the desired role, Granting, changing, and revoking access to resources is relevant. 

[Shaun Budhram]

Thanks.  I was able to get this working by getting the service account with 'gcloud iam service-accounts list'.

I put in this value as a new 'Member', and gave it the roles 'Storage Legacy Bucket Owner' and 'Storage Legacy Object Owner'.  This uploads the files properly.

However, since I'm uploading images, it seems like my calls to images.get_serving_url(...) have broken.  Not sure if the method of getting these urls has changed with the switch to google cloud storage.  I'll try to resolve this independently and open a separate thread if necessary.

Thanks.

[George (Cloud Platform Support)]

You may find related information on the Images API for Python 2 Overview page. You are right about opening a separate thread for each separate issue, in principle. Maybe worth mentioning: this is a discussion group for Google App Engine, and related problems and trends. When it comes strictly to programming and coding in specific languages, you’ll be at an advantage to rather post your questions on Stackoverflow, to gain this way access to a large number of experts; it is meant for providing you help with coding.