Google Cloud Storage create_upload_url -- App Engine Flexible Python

[Jon G via StackOverflow]

On a regular (non-flexible) instance of Google App Engine, you can use the Blobstore API and create a URL to allow a user to upload a file directly into your Blobstore. When it is uploaded, your app engine application is notified of the location of the file and can process it. An example of the python code is:

from google.appengine.ext import blobstore
upload_url = blobstore.create_upload_url('/upload_photo')

See the Blobstore docs.

Switching to Google App Engine Flexible Environment, usage of the Blobstore has been largely replaced by Cloud Storage. In such a case, is there an equivalent of create_upload_url?

My current implementation takes a standard file upload to a python Flask application. Then proceeds with something like:

from flask import request
from google.cloud import storage

uploaded_file = request.files.get('file')

gcs = storage.Client()
bucket = gcs.get_bucket(bucket_name)
blob = bucket.blob(blob_name)
blob.upload_from_string(
    uploaded_file.read(),
    content_type=uploaded_file.content_type
)

This seems like it is doubling the network load compared with create_upload_url because the file is coming into my app engine instance and then immediately being copied out. So the uploader will be made to wait extra time whilst this is happening. Presumably I will also incur extra App Engine charges for this. Is there a better way?

I have workers that later process the uploaded file, but I tend to download the file from Cloud Storage again in their code because I don't think you can assume that the worker will still have access to a file stored in the instance file system. Therefore I don't get any benefit of having the file uploaded to my instance rather than direct to it's storage location.

Please DO NOT REPLY directly to this email but go to StackOverflow:
http://stackoverflow.com/questions/42087048/google-cloud-storage-create-upload-url-app-engine-flexible-python

[Jeffrey Godwyll via StackOverflow]

Your current implementation is missing just one thing to get something close to create_upload_url for Google Cloud Storage. Each blob created with the new Google Cloud Storage Client has a public_url property:

from flask import request
from google.cloud import storage

uploaded_file = request.files.get('file')

gcs = storage.Client()
bucket = gcs.get_bucket(bucket_name)
blob = bucket.blob('blob_name')
blob.upload_from_string(
    uploaded_file.read(),
    content_type=uploaded_file.content_type
)
url = blob.public_url

Please DO NOT REPLY directly to this email but go to StackOverflow:

http://stackoverflow.com/questions/42087048/google-cloud-storage-create-upload-url-app-engine-flexible-python/42110534#42110534

[Jeffrey Godwyll via StackOverflow]

Each blob created with the new Google Cloud Storage Client has a public_url property:

from flask import request
from google.cloud import storage

uploaded_file = request.files.get('file')

gcs = storage.Client()
bucket = gcs.get_bucket(bucket_name)
blob = bucket.blob('blob_name')
blob.upload_from_string(
    uploaded_file.read(),
    content_type=uploaded_file.content_type
)
url = blob.public_url

--

With the Blobstore, a GAE system handler in your instance takes care of the uploaded file you pass to the upload url created. I'm not sure if it's an issue handling it yourself in your code. If your current approach is problematic, you might want to consider doing the upload client side and not pass the file through App Engine at all. GCS has a REST API and the cloud storage client uses it underneath, so you can read and upload the file directly to GCS on the client side if it's more convenient. There's firebase.google.com/docs/storage/web/upload-files to ease you through the process