Uploading an image to cloud storage: gcs.open() vs signed URLs

[Suresh Jeevanandam]

Hi,

  In my application, I need to store a set of images with a particular kind.

  These will be uploaded by a small set of users (~20) but be read by a lot.

   Now I am wondering which way to go about this. Storing the images in the data-store blobs vs using the outdated blob-store vs using cloud storage. I am trying to use cloud storage as it seems to be the recommended one.

   Again with using cloud storage, I have to choose among few options. It appears the easiest option is to use the cloudstorage module and read and write from/to the bucket in the requests. But I also see few posts about signed URLs. As far as I could see, the signed URLs have the advantage that the data moves between the client and the storage bucket without having to go through appengine.

   Let's say I choose the easier option of routing the data movement through appengine, what am I losing:

1. Delays ?

2. Cost? Does read, write from appengine incur additional cost. Since uploading is a rare thing in my application, is there an easy way to give direct URL links for images stored in cloud-storage with access restriction (something like signed URLs for reading) - Sorry if I miss something here stated obviously in the documentation somewhere. I see information about this scattered around and all confusing. I wish we had a more organized documentation.

-

Suresh

[Richard Cheesmar]

I did this by going up to server and then onto the cloud and then store the image url in appropriate database...I wouldn't do it this way now, in fact I plan to change this to client --> cloud direct, I do this for video and it works great now. It was a bit of a fuss pot to setup, but once you have done the donkey work, it's fine. I create a signed url on the server with an ajax request, send it back and initiate the upload.


So go down the client --> cloud route. it's a much cleaner solution.

[Richard Cheesmar]

Forgot to mention, once the images are uploaded onto the cloud you will need to pass image urls and some kind of reference to id what image is what back to the server for reference later in your code...a simple ndb model would suffice.

On Saturday, August 19, 2017 at 9:48:49 PM UTC+3, Suresh Jeevanandam wrote:

[Yannick (Cloud Platform Support)]

Hello Suresh, to answer your questions:

1) While there is no timing information available to quantify the added delay, it should not prove to be an issue. Make sure that the bucket you want to use is in the same region as your App Engine application to minimize delays.

2) Network ingress is free for both App Engine and Google Cloud Storage so you should not incur any additional network charges, though it might result in your application getting more requests and thus using a bit more instance hours.

3) Signed urls for reading would simply be signed urls, as those can be made to allow  read-only access to objects. If your goal is to give access to a known list of accounts you could try looking into authenticated browser downloads.

I hope this helps. Let me know if and how I should expand on my answer.

[Suresh Jeevanandam]

Thanks, Yannick. I will give a try first for signed-URLs and if it proves to be effort taking I will just use the plain read/write in the requests.

Thanks for the clarifications.

-

Suresh

[Suresh Jeevanandam]

Hi Richard,

  Thanks for the suggestions. I will give a try to signed-URLs. I am not coming from web development background and the instructions for signed-URLs just scare me as if it is a big complex thing. Sometimes such things prove to be easy when we sit down to implement :). I will give a try.

-

Suresh

[Richard Cheesmar]

Hey, Suresh,

Did you manage it?