image.ServingURL very slow (~200ms)

138 views
Skip to first unread message

David de Kloet

unread,
Apr 23, 2016, 8:35:18 AM4/23/16
to Google App Engine
Hello,

I just noticed that turning a blob key into an image serving URL is very slow, often over 200ms.
Is this expected or am I doing something wrong?
I assumed that turning a blob key into a serving URL would be a simple string transformation, but even if it needs to make a request to another service, 200ms seems very slow.
If it makes any difference, I'm using Go, and I just noticed that the "appengine" libraries have been deprecated by "google.golang.org/appengine", but I haven't updated yet.

Thanks,
David

Christian F. Howes

unread,
Apr 23, 2016, 3:46:44 PM4/23/16
to Google App Engine
i haven't measured the time it takes for the transformation recently, but note that the serving URL does not change, so i only generate it when a new blob is uploaded and store the URL on the object in my datastore so that it takes 0 time to generate in the future.

David de Kloet

unread,
Apr 23, 2016, 6:03:00 PM4/23/16
to google-a...@googlegroups.com
I need the blob key to be able to delete the blob, so I thought it wouldn't make sense to store both the blob key and the image URL.
It might make sense for the reason you mention, but I'd first like to understand why it's so slow or whether I'm doing something wrong, before I store redundant data.

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/MxQE5A_dKVM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
 To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/4bb00a30-3b0b-4555-8c98-20cc6a299ca7%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Adam (Cloud Platform Support)

unread,
Apr 23, 2016, 7:17:04 PM4/23/16
to Google App Engine
Turning a blob key into a serving URL isn't a simple string transformation. The blob must be made available on a special image serving infrastructure that supports resizing and cropping. The Python documentation provides some more details on this than the Go docs do.

The recommended practice is to cache the serving URL and only regenerate it when needed, as Christian mentioned.

On Saturday, April 23, 2016 at 6:03:00 PM UTC-4, David de Kloet wrote:
I need the blob key to be able to delete the blob, so I thought it wouldn't make sense to store both the blob key and the image URL.
It might make sense for the reason you mention, but I'd first like to understand why it's so slow or whether I'm doing something wrong, before I store redundant data.
On Sat, Apr 23, 2016 at 9:46 PM, Christian F. Howes wrote:
i haven't measured the time it takes for the transformation recently, but note that the serving URL does not change, so i only generate it when a new blob is uploaded and store the URL on the object in my datastore so that it takes 0 time to generate in the future.

On Saturday, April 23, 2016 at 5:35:18 AM UTC-7, David de Kloet wrote:
Hello,

I just noticed that turning a blob key into an image serving URL is very slow, often over 200ms.
Is this expected or am I doing something wrong?
I assumed that turning a blob key into a serving URL would be a simple string transformation, but even if it needs to make a request to another service, 200ms seems very slow.
If it makes any difference, I'm using Go, and I just noticed that the "appengine" libraries have been deprecated by "google.golang.org/appengine", but I haven't updated yet.

Thanks,
David

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/MxQE5A_dKVM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscribe@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.

David de Kloet

unread,
Apr 24, 2016, 5:28:10 AM4/24/16
to google-a...@googlegroups.com
Thanks Adam and Christian!

On Sun, Apr 24, 2016 at 1:17 AM, 'Adam (Cloud Platform Support)' via Google App Engine <google-a...@googlegroups.com> wrote:
Turning a blob key into a serving URL isn't a simple string transformation. The blob must be made available on a special image serving infrastructure that supports resizing and cropping. The Python documentation provides some more details on this than the Go docs do.

The recommended practice is to cache the serving URL and only regenerate it when needed, as Christian mentioned.

On Saturday, April 23, 2016 at 6:03:00 PM UTC-4, David de Kloet wrote:
I need the blob key to be able to delete the blob, so I thought it wouldn't make sense to store both the blob key and the image URL.
It might make sense for the reason you mention, but I'd first like to understand why it's so slow or whether I'm doing something wrong, before I store redundant data.
On Sat, Apr 23, 2016 at 9:46 PM, Christian F. Howes wrote:
i haven't measured the time it takes for the transformation recently, but note that the serving URL does not change, so i only generate it when a new blob is uploaded and store the URL on the object in my datastore so that it takes 0 time to generate in the future.

On Saturday, April 23, 2016 at 5:35:18 AM UTC-7, David de Kloet wrote:
Hello,

I just noticed that turning a blob key into an image serving URL is very slow, often over 200ms.
Is this expected or am I doing something wrong?
I assumed that turning a blob key into a serving URL would be a simple string transformation, but even if it needs to make a request to another service, 200ms seems very slow.
If it makes any difference, I'm using Go, and I just noticed that the "appengine" libraries have been deprecated by "google.golang.org/appengine", but I haven't updated yet.

Thanks,
David

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/MxQE5A_dKVM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/MxQE5A_dKVM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.

Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/a11701b9-21ac-421b-b02f-a489d911f2ca%40googlegroups.com.

David de Kloet

unread,
May 1, 2016, 1:09:16 PM5/1/16
to google-a...@googlegroups.com
I have a related question. I noticed that the served images have Cache-Control:max-age=0.
Are the served images not immutable? If I write a new blob to the same blob key, would that be served under the same image URL?
Is there a way to create an image serving URL that serves an immutable, cacheable image?

On Sun, Apr 24, 2016 at 1:17 AM, 'Adam (Cloud Platform Support)' via Google App Engine <google-a...@googlegroups.com> wrote:
Turning a blob key into a serving URL isn't a simple string transformation. The blob must be made available on a special image serving infrastructure that supports resizing and cropping. The Python documentation provides some more details on this than the Go docs do.

The recommended practice is to cache the serving URL and only regenerate it when needed, as Christian mentioned.

On Saturday, April 23, 2016 at 6:03:00 PM UTC-4, David de Kloet wrote:
I need the blob key to be able to delete the blob, so I thought it wouldn't make sense to store both the blob key and the image URL.
It might make sense for the reason you mention, but I'd first like to understand why it's so slow or whether I'm doing something wrong, before I store redundant data.
On Sat, Apr 23, 2016 at 9:46 PM, Christian F. Howes wrote:
i haven't measured the time it takes for the transformation recently, but note that the serving URL does not change, so i only generate it when a new blob is uploaded and store the URL on the object in my datastore so that it takes 0 time to generate in the future.

On Saturday, April 23, 2016 at 5:35:18 AM UTC-7, David de Kloet wrote:
Hello,

I just noticed that turning a blob key into an image serving URL is very slow, often over 200ms.
Is this expected or am I doing something wrong?
I assumed that turning a blob key into a serving URL would be a simple string transformation, but even if it needs to make a request to another service, 200ms seems very slow.
If it makes any difference, I'm using Go, and I just noticed that the "appengine" libraries have been deprecated by "google.golang.org/appengine", but I haven't updated yet.

Thanks,
David

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/MxQE5A_dKVM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/MxQE5A_dKVM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.

Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/a11701b9-21ac-421b-b02f-a489d911f2ca%40googlegroups.com.

Adam (Cloud Platform Support)

unread,
May 2, 2016, 3:52:35 PM5/2/16
to Google App Engine
Interestingly, it seems that images served directly from Cloud Storage instead of Blobstore set "Cache-Control: public, max-age=86400, no-transform", so an option may be to use the Images service with GCS objects instead of blobkeys. There is an open feature request to allow the Images service to respect the GCS cache control metadata instead.

Alternatively, if you don't require the transformations or obfuscated URLs provided by the Images service, you could link directly to images uploaded to a public Google Cloud Storage bucket. 

On Sunday, May 1, 2016 at 1:09:16 PM UTC-4, David de Kloet wrote:
I have a related question. I noticed that the served images have Cache-Control:max-age=0.
Are the served images not immutable? If I write a new blob to the same blob key, would that be served under the same image URL?
Is there a way to create an image serving URL that serves an immutable, cacheable image?
On Sun, Apr 24, 2016 at 1:17 AM, 'Adam (Cloud Platform Support)' via Google App Engine <google-appengine@googlegroups.com> wrote:
Turning a blob key into a serving URL isn't a simple string transformation. The blob must be made available on a special image serving infrastructure that supports resizing and cropping. The Python documentation provides some more details on this than the Go docs do.

The recommended practice is to cache the serving URL and only regenerate it when needed, as Christian mentioned.

On Saturday, April 23, 2016 at 6:03:00 PM UTC-4, David de Kloet wrote:
I need the blob key to be able to delete the blob, so I thought it wouldn't make sense to store both the blob key and the image URL.
It might make sense for the reason you mention, but I'd first like to understand why it's so slow or whether I'm doing something wrong, before I store redundant data.
On Sat, Apr 23, 2016 at 9:46 PM, Christian F. Howes wrote:
i haven't measured the time it takes for the transformation recently, but note that the serving URL does not change, so i only generate it when a new blob is uploaded and store the URL on the object in my datastore so that it takes 0 time to generate in the future.

On Saturday, April 23, 2016 at 5:35:18 AM UTC-7, David de Kloet wrote:
Hello,

I just noticed that turning a blob key into an image serving URL is very slow, often over 200ms.
Is this expected or am I doing something wrong?
I assumed that turning a blob key into a serving URL would be a simple string transformation, but even if it needs to make a request to another service, 200ms seems very slow.
If it makes any difference, I'm using Go, and I just noticed that the "appengine" libraries have been deprecated by "google.golang.org/appengine", but I haven't updated yet.

Thanks,
David

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/MxQE5A_dKVM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscribe@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/MxQE5A_dKVM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscribe@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.

David de Kloet

unread,
May 5, 2016, 9:16:43 AM5/5/16
to google-a...@googlegroups.com
I didn't realize that GCS objects have their own Cache-Control header specified. Thanks for the suggestion.

On Mon, May 2, 2016 at 9:52 PM, 'Adam (Cloud Platform Support)' via Google App Engine <google-a...@googlegroups.com> wrote:
Interestingly, it seems that images served directly from Cloud Storage instead of Blobstore set "Cache-Control: public, max-age=86400, no-transform", so an option may be to use the Images service with GCS objects instead of blobkeys. There is an open feature request to allow the Images service to respect the GCS cache control metadata instead.

Alternatively, if you don't require the transformations or obfuscated URLs provided by the Images service, you could link directly to images uploaded to a public Google Cloud Storage bucket. 

On Sunday, May 1, 2016 at 1:09:16 PM UTC-4, David de Kloet wrote:
I have a related question. I noticed that the served images have Cache-Control:max-age=0.
Are the served images not immutable? If I write a new blob to the same blob key, would that be served under the same image URL?
Is there a way to create an image serving URL that serves an immutable, cacheable image?
On Sun, Apr 24, 2016 at 1:17 AM, 'Adam (Cloud Platform Support)' via Google App Engine <google-a...@googlegroups.com> wrote:
Turning a blob key into a serving URL isn't a simple string transformation. The blob must be made available on a special image serving infrastructure that supports resizing and cropping. The Python documentation provides some more details on this than the Go docs do.

The recommended practice is to cache the serving URL and only regenerate it when needed, as Christian mentioned.

On Saturday, April 23, 2016 at 6:03:00 PM UTC-4, David de Kloet wrote:
I need the blob key to be able to delete the blob, so I thought it wouldn't make sense to store both the blob key and the image URL.
It might make sense for the reason you mention, but I'd first like to understand why it's so slow or whether I'm doing something wrong, before I store redundant data.
On Sat, Apr 23, 2016 at 9:46 PM, Christian F. Howes wrote:
i haven't measured the time it takes for the transformation recently, but note that the serving URL does not change, so i only generate it when a new blob is uploaded and store the URL on the object in my datastore so that it takes 0 time to generate in the future.

On Saturday, April 23, 2016 at 5:35:18 AM UTC-7, David de Kloet wrote:
Hello,

I just noticed that turning a blob key into an image serving URL is very slow, often over 200ms.
Is this expected or am I doing something wrong?
I assumed that turning a blob key into a serving URL would be a simple string transformation, but even if it needs to make a request to another service, 200ms seems very slow.
If it makes any difference, I'm using Go, and I just noticed that the "appengine" libraries have been deprecated by "google.golang.org/appengine", but I haven't updated yet.

Thanks,
David

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/MxQE5A_dKVM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/MxQE5A_dKVM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/MxQE5A_dKVM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/b5f39815-74bc-46bb-8b2c-4b97a12219e5%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages