Questions about request logs (their circular buffer, its size etc.)

[peterk]

Reading some previous discussions and things alluded to therein, some
questions cropped up.

I'm considering an application that quite fundamentally requires
access to all its request logs.

So I guess I'd be recommended to use appcfg.py to periodically pull
down the logs and append them to a txt file, or do whatever else I
need to do to persist them.

In previous discussion I saw Google folks say that logs are stored in
a circular buffer. Meaning if you had enough requests, they'd
overwrite older requests. Meaning that if you had enough requests or
were downloading the logs sufficiently infrequently, you might lose
logs.

So question number 1: how big is the circular buffer for each
application?

2: Changes to that buffer size could detrimentally impact on my
application if revised downward (silently in particular), so are there
any guarantees that this buffer size won't be reduced in the future?

3: Is there any possibility that this buffer could be exposed via the
quota system so that an app could purchase a larger buffer size if it
were helpful?

I'm flashing forward to a busy site and somewhat worrying about how
frequently I'll have to download the logs, and whether it would be
feasible to download them with a certain frequency anyway (e.g. if the
log download time is quite long - as I've found it can be). Any
answers or advice would be very welcome. Thanks!

[Ikai L (Google)]

The last time I checked, we guaranteed 90 days of logs regardless of size. This is subject to change, however. I don't know what size we will set, but like many of our other products, it'll likely start at a number we think is reasonable based on overall usage and trend upward.

Anything is possible with regards to billed features - we just can't give an ETA unless it's explicitly stated on the roadmap.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
Ikai Lan
Developer Programs Engineer, Google App Engine
http://googleappengine.blogspot.com | http://twitter.com/app_engine

[peterk]

Hey Ikai, thanks for the reply.

Are you sure about the 90 day thing? I was reading a thread here where
Nick Johnson suggested it was a fixed buffer size:

http://groups.google.com/group/google-appengine/browse_thread/thread/faa52812efdbeaaa/a22d1a02a97e95df?lnk=gst&q=appengine+logs+circular+buffer#a22d1a02a97e95df

In that case, the person asking seemed to be only able to keep 20
minutes of log data before old logs started to be overwritten (a busy
site I guess!).

Has something changed? Could you check out what the case actually is?

If I might, I'd like to request if log related 'quotas' could be
exposed and published, and expandable with money if possible! :)
They're quite critical to an app I'm trying to plan, and while the
current setup seems to be OK for my needs, the blindness to what's
happening under the covers and what might happen going forward is a
wee bit disconcerting.

Cheers,

> > google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com>

[Nick Johnson (Google)]

Hi Peter,

Both are true: We keep 90 days of request logs, but only a fixed amount of application logging.

-Nick Johnson

To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.

To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

[peterk]

Thanks Nick, that's nice and clear now.

Do you know if there's any upper limit on the frequency with which
logs can be downloaded?

Cheers,

On Apr 16, 2:03 pm, "Nick Johnson (Google)" <nick.john...@google.com>
wrote:

> Hi Peter,
>
> Both are true: We keep 90 days of request logs, but only a fixed amount of
> application logging.
>
> -Nick Johnson
>
>
>
>
>
> On Thu, Apr 15, 2010 at 7:11 PM, peterk <peter.ke...@gmail.com> wrote:
> > Hey Ikai, thanks for the reply.
>
> > Are you sure about the 90 day thing? I was reading a thread here where
> > Nick Johnson suggested it was a fixed buffer size:
>

> >http://groups.google.com/group/google-appengine/browse_thread/thread/...

> > > > google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com><google-appengine%2Bunsubscrib

> > e...@googlegroups.com>
> > > > .
> > > > For more options, visit this group at
> > > >http://groups.google.com/group/google-appengine?hl=en.
>
> > > --
> > > Ikai Lan
> > > Developer Programs Engineer, Google App Enginehttp://
> > googleappengine.blogspot.com|http://twitter.com/app_engine
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-a...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
>
> --

> Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. ::
> Registered in Dublin, Ireland, Registration Number: 368047
> Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
> 368047
>
> --
> You received this message because you are subscribed to the Google Groups "Google App Engine" group.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
> For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en.