Reading/Writing Blobs Failing Silently Over the last 24 hours

[notreadbyhumans]

For the last 24 hours or so we have been experiencing issues reading and writing blobs through App Engine.

Writes generate a 502 and reads generate 500 errors, but don't receive an elevated log level in the console.

Specifically the write error is occurring during /ah_/upload and the read error is occurring after send_blob is called (Python), but before the file is rendered.

No updates have been made to the app for a couple of weeks. This issue seems to be affecting all users, but not all blobs i.e. users can access some of their files, but not others. 

Has anyone else been experiencing this?

[John Lowry]

Hi,

We do have a known issue in which serving Cloud Storage objects using the Blobstore API with a pre-generated blob key is failing for a small number of apps starting approximately Oct 13 2014.

We are working on fixing this. 

As a workaround, you can re-generate the blob key from the Cloud Storage file path. 

However, if this is urgent then you can send me your app ID privately and I will provide a workaround fix.

I am not aware of any problems uploading using the Blobstore API. Can you contact me privately to provide your app ID and a URL where I can reproduce.

John