HR Issues (But Up time is great)

149 views
Skip to first unread message

Brandon Wirtz

unread,
Mar 19, 2011, 4:32:59 PM3/19/11
to google-a...@googlegroups.com

 

So they are all in one place, here are the things I know to be an Issue with HR:

 

1.       Caching doesn’t work, and because of the way headers are forced to be set all requests expire immediately so the user won’t cache either.

(Headers from a request)

HTTP/1.1 405 Method Not Allowed
Content-Type: text/html; charset=utf-8
Cache-Control: no-cache, must-revalidate
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Set-Cookie: S=apphosting=5hoCHm4x9c3ZrrxQEN5Q4A; path=/
Date: Sat, 19 Mar 2011 19:37:21 GMT
Pragma: no-cache
Server: Google Frontend
Content-Length: 0

This happens even if you programmatically set output to have some other expiration, pragma, and/or cache-control.

 

 

 

 

2.       Charts in the dash board only go back 24 hours.

Vs. up to 30 days for Master Slave.

3.       Data Store Viewer always errors if you try to edit entities.

 

This works most of the time when I use MS… (not always but mostly)

 

4.       This isn’t so much an “issue” as an observation.  I get a higher error rate on HR at the same time I get higher rates on MS.  Sometimes I get errors on MS and won’t get them on HR, but it seems anytime my error rate goes up on HR it is up on MS as well. 

 

Most  of these errors are “memcache.add failed”  [ ok in my app that is the only error I ever get, but that’s cause my code is perfect it’s the platform that has issues :-) ]

 

 

image001.png
image002.png
image003.png

nickmilon

unread,
Mar 20, 2011, 5:02:59 PM3/20/11
to Google App Engine
If caching does not work in HR this is a real problem.
Could it be by design ?
Have others observed this ?

happy coding:-)
Nick



On Mar 19, 10:32 pm, "Brandon Wirtz" <drak...@digerat.com> wrote:
> So they are all in one place, here are the things I know to be an Issue with
> HR:
>
> 1.       Caching doesn't work, and because of the way headers are forced to
> be set all requests expire immediately so the user won't cache either.
>
> (Headers from a request)
>
> HTTP/1.1 405 Method Not Allowed
> Content-Type: text/html; charset=utf-8
> Cache-Control: no-cache, must-revalidate
> Expires: Fri, 01 Jan 1990 00:00:00 GMT
> Set-Cookie: S=apphosting=5hoCHm4x9c3ZrrxQEN5Q4A; path=/
> Date: Sat, 19 Mar 2011 19:37:21 GMT
> Pragma: no-cache
> Server: Google Frontend
> Content-Length: 0
>
> This happens even if you programmatically set output to have some other
> expiration, pragma, and/or cache-control.
>
> 2.       Charts in the dash board only go back 24 hours.
>
> Vs. up to 30 days for Master Slave.
>
> 3.       Data Store Viewer always errors if you try to edit entities.
>
> This works most of the time when I use MS. (not always but mostly)
>
> 4.       This isn't so much an "issue" as an observation.  I get a higher
> error rate on HR at the same time I get higher rates on MS.  Sometimes I get
> errors on MS and won't get them on HR, but it seems anytime my error rate
> goes up on HR it is up on MS as well.  
>
> Most  of these errors are "memcache.add failed"  [ ok in my app that is the
> only error I ever get, but that's cause my code is perfect it's the platform
> that has issues :-) ]
>
>  image001.png
> 43KViewDownload
>
>  image002.png
> 32KViewDownload
>
>  image003.png
> 51KViewDownload

Simon Knott

unread,
Mar 21, 2011, 4:57:30 AM3/21/11
to google-a...@googlegroups.com
Are you using Python or Java?

My experience for a Java app is:

1) Caching seems to work fine for my app - how are you being forced to set headers?  Do you have sessions enabled?
2) My dashboard only goes back 24hrs as well.
3) I can edit entities fine in the datastore - does this error happen when you edit properties of a specific type, or for any properties?
4) I experience a bit more latency for MemCache on the HR version of my app, than the MS version.  I don't have anywhere near the amount of traffic you do however, so I haven't noticed any actual errors in this area.

Ikai Lan (Google)

unread,
Mar 21, 2011, 2:51:34 PM3/21/11
to Google App Engine
Thanks for the feedback! I have observed the same as Simon with slightly increased latency, but not a higher error rate.

I'll raise the issue with the Dashboard.

Ikai Lan 
Developer Programs Engineer, Google App Engine



--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

Brandon Wirtz

unread,
Mar 21, 2011, 3:03:39 PM3/21/11
to google-a...@googlegroups.com

Python.

 

Can I check your headers?  (like via a URL) because even my static files I can’t get the cache-control / expire to be honored.

--

Brandon Wirtz

unread,
Mar 21, 2011, 4:01:52 PM3/21/11
to google-a...@googlegroups.com

Ikai,

 

What about the Cache Control/Pragma/Expires headers.  Any ideas as to if those always being set to do not cache is by design?

 

Oh, and it isn’t exactly a higher error rate, it is just “an error rate” there are times you can’t write to mem-cache  it’s not a big deal, just something that would be cool if it didn’t happen (or as much).

 

-Brandon

Ikai Lan (Google)

unread,
Mar 21, 2011, 4:22:49 PM3/21/11
to Google App Engine
It's not by design, but it's not a high priority at the moment since we do not guarantee edge caching.

Ikai Lan 
Developer Programs Engineer, Google App Engine


Brandon Wirtz

unread,
Mar 21, 2011, 4:30:18 PM3/21/11
to google-a...@googlegroups.com

Ikai,

 

It’s not for edge Caching, it is so that when I serve the CSS and JPG files for a page that subsequent pages that use those same assets will not require re-downloading the images and CSS.

 

It’s also to help Curb the Google Bot, which has its own “fun” issues one of which being that you can’t specify a crawl rate on an AppEngine Hosted site in webmaster tools, and if the cache control is always set to expire, Google bot will crawl 10k pages, finish, and say “Hey All these expired let’s crawl them again and see what changed.  On several of my sites Google Bot is 80% of my traffic.  Which is fine, but he never clicks on any ads :-) so he is really hard to monetize.

Simon Knott

unread,
Mar 21, 2011, 6:26:30 PM3/21/11
to google-a...@googlegroups.com, Brandon Wirtz
Hi Brandon,

The difference between our apps is that I'm using the Java SDK - I don't know whether that makes a difference.  

My request headers for static resources are as follows:
  1. Accept:
    */*
  2. Accept-Charset:
    ISO-8859-1,utf-8;q=0.7,*;q=0.3
  3. Accept-Encoding:
    gzip,deflate,sdch
  4. Accept-Language:
    en-GB,en-US;q=0.8,en;q=0.6
  5. Cache-Control:
    max-age=0
  6. Connection:
    keep-alive
  7. Cookie:
    S=apphosting=lJdUb2lQFokus??f0BDw; ACSID=AJKiYcHLHu0ykssZOsp0RuHamFvmI1omlz-???0XKQKzFrO_K_gnleJxN36w249QujsmCt-nc5oberCcX67nwhPjbuWjO7CNilEgT8iSoGzBDXFqERzEJQIbCcl8zuoNodt9J7v7AMGsr8UAnGiTNyS_64fp9XHquPUjcOWxB-NYXj1sUBEHSc0ydm84a8noDh67exBjzjAz4YqhceiQ1LGOiy3I9avRQ1eljUkjTkvRZb32gwrFpTIGNDS5yp0q_2909_u0Q1YsFLBADy7HS6W38iAv9qTEuqyO23Z5jB5KfXU2Cuot_PTi_uQH2kIjjI6-r2g3cvRv4GGhRTz9NtvGfZbtWOayK1jEKKRhAOeUZPjEE38N8VIke2n-UsguOBnViIOOSM4ELgL_RAnx6ztrhU_5bvaoevRa2Xab7urhurhWtVZCQzv3UxbJro1Y????hGZoTdApvGOpUHQH51EQZ6xiyGGz4kgYlToUzyZLBHLYFvPN8CyMTFrdTGzlaUJAaGMOF2_W5W3EE7s0GtiOf6_1XqNSoNPzod9BzCI_Mhk
  8. Host:
    xxx
  9. If-None-Match:
    "-n5uDA"
  10. Referer:
    xxx
  11. User-Agent:
    Mozilla/5.0 (Windows NT 6.1) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.14 Safari/534.24

My response headers are as follows:

Status Code:
304 Not Modified
Date:
Mon, 21 Mar 2011 22:16:12 GMT
ETag:
"-n5uDA"
Server:
Google Frontend

I've sent you a link to one of my images for you to test.

Cheers,
Simon

Stephen

unread,
Mar 21, 2011, 6:54:13 PM3/21/11
to google-a...@googlegroups.com
On Sat, Mar 19, 2011 at 8:32 PM, Brandon Wirtz <dra...@digerat.com> wrote:
>
> 1. Caching doesn’t work, and because of the way headers are
> forced to be set all requests expire immediately so the user won’t cache either.
>
> (Headers from a request)
>
> HTTP/1.1 405 Method Not Allowed
> Content-Type: text/html; charset=utf-8
> Cache-Control: no-cache, must-revalidate
> Expires: Fri, 01 Jan 1990 00:00:00 GMT
> Set-Cookie: S=apphosting=5hoCHm4x9c3ZrrxQEN5Q4A; path=/
> Date: Sat, 19 Mar 2011 19:37:21 GMT
> Pragma: no-cache
> Server: Google Frontend
> Content-Length: 0


These look like headers from a response, not a request, and the
response code is 405: Method Not Allowed. ie. the request was
something other than a GET, POST, PUT or DELETE, in which case you
might expect the error response to force a no-cache header.

Are you sure your test is accurate?

Brandon Wirtz

unread,
Mar 21, 2011, 10:33:27 PM3/21/11
to google-a...@googlegroups.com
I swear someone cheated... Or ran over and flipped a switch when I asked...
Cause out of the same tool, now I get

HTTP/1.1 200 OK
Via: HTTP/1.1 GWA (remote cache hit)
Accept-Ranges: bytes
Last-Modified: Wed, 02 Mar 2011 01:25:48 GMT
Etag: "159827d-169b0-49d75caeb5f00"
Content-Type: image/png
Age: 6985
Cache-Control: public; max-age=300
Date: Tue, 22 Mar 2011 02:29:27 GMT
Server: Google Frontend
Content-Length: 92592


Which even says that Google gave me a cache hit. Which is interesting cause
the Dashboard on the app doesn't claim it has any of those. (even though My
MS version of the app does)

So just ignore me, it seems to work well enough, I'm just partially insane.
:-)

--

Simon Knott

unread,
Mar 22, 2011, 5:38:13 AM3/22/11
to google-a...@googlegroups.com, Brandon Wirtz
I'm glad to hear it's now working for you!

Ikai Lan (Google)

unread,
Mar 22, 2011, 1:22:50 PM3/22/11
to Google App Engine
Good to hear it's working. If I hear anything about what changed, I'll let you guys know.

Ikai Lan 
Developer Programs Engineer, Google App Engine


Brandon Wirtz

unread,
Mar 24, 2011, 1:41:13 PM3/24/11
to google-a...@googlegroups.com

Looks like HR now has Dash board details going back 36 hours…. Not great… But better. Hoping this means that in 72 hours we will have data going back 72 hours :-)

 

--

image001.png
Reply all
Reply to author
Forward
0 new messages