Does the Log Viewer actually work for anyone?
132 views
Skip to first unread message
Jason Collins
Apr 17, 2012, 6:33:24 PM4/17/12
to Google App Engine
Is it just me, or does the Log Viewer searching basically not work?
I have my log storage configured to give me about 11 days of logs, but
I can rarely actually find stuff (that I _know_ occurred within my
window) by searching - both regex- and label-based searching. We run
at about 100+QPS, so trying to catch something in action is basically
impossible.
I'm guessing the log indexing runs async to the log entries
themselves, and it's just not able to keep pace, or the indexing jobs
themselves are just failing.
Does anyone actually have any luck searching the logs in the Log
Viewer on a reasonably high rate application?
j
I have my log storage configured to give me about 11 days of logs, but
I can rarely actually find stuff (that I _know_ occurred within my
window) by searching - both regex- and label-based searching. We run
at about 100+QPS, so trying to catch something in action is basically
impossible.
I'm guessing the log indexing runs async to the log entries
themselves, and it's just not able to keep pace, or the indexing jobs
themselves are just failing.
Does anyone actually have any luck searching the logs in the Log
Viewer on a reasonably high rate application?
j
Takashi Matsuo
Apr 18, 2012, 12:59:17 AM4/18/12
to google-a...@googlegroups.com
Hi Jason,
Do you mind filing an issue on the issue tracker with some details?
(e.g. app-id, example queries, etc)
If you mind making the info public, you can also send it directly to me.
Thanks,
-- Takashi
> --
> You received this message because you are subscribed to the Google Groups "Google App Engine" group.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
>
--
Takashi Matsuo | Developer Advocate | tma...@google.com | 03-6384-9224
Gregory D'alesandre
Apr 18, 2012, 2:45:47 AM4/18/12
to google-a...@googlegroups.com
Hey Jason,
Please do fill out an issue but this is indeed a known issue. We don't do currently index for regex and labels so when you do a search across your records it will take quite a while and we stop and return whatever results we've found after 10 seconds. We are looking to improve this at some point but it does mean right now you'll get inconsistent results when trying to search across a lot of logs.
Hope that makes sense!
Greg
Jason Collins
Apr 18, 2012, 1:13:37 PM4/18/12
to Google App Engine
Thanks Greg - it totally makes sense and explains why something that
is even quite recent does not appear on a "high-rate" application.
Proppy just demo'd a BigQuery-based log analysis in the App Engine
Chat - you should just go straight to that solution under the hood...
j
> > Takashi Matsuo | Developer Advocate | tmat...@google.com | 03-6384-9224
is even quite recent does not appear on a "high-rate" application.
Proppy just demo'd a BigQuery-based log analysis in the App Engine
Chat - you should just go straight to that solution under the hood...
j
Gregory D'alesandre
Apr 18, 2012, 1:46:55 PM4/18/12
to google-a...@googlegroups.com
Ah, c'mon now you are giving away the ending! ;) We are looking into this as a possibility. BigQuery has some awesome capabilities and paired with logs its quite effective!
Greg
Prateek Malhotra
Apr 27, 2012, 12:25:46 PM4/27/12
to google-a...@googlegroups.com
Was an issue created for this? I can't seem to find it and I would like to star it. I've been searching through my logs trying to determine a few errors we're getting and all this time I thought my lack of results was my fault. Kinda glad (or maybe not so glad) that it is Google's fault...
-Prateek
Greg
> > > To post to this group, send email to google-appengine@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
>
> > --
> > Takashi Matsuo | Developer Advocate | tmat...@google.com | 03-6384-9224
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appengine@googlegroups.com.
> > To unsubscribe from this group, send email to
--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to google-appengine+unsubscribe@googlegroups.com.
Jason Collins
Apr 27, 2012, 2:54:30 PM4/27/12
to Google App Engine
I didn't create an issue for this because it seemed like something was
underway already.
If you create one, I'll star it.
j
On Apr 27, 10:25 am, someone1 <someo...@gmail.com> wrote:
> Was an issue created for this? I can't seem to find it and I would like to
> star it. I've been searching through my logs trying to determine a few
> errors we're getting and all this time I thought my lack of results was my
> fault. Kinda glad (or maybe not so glad) that it is Google's fault...
>
> -Prateek
>
>
>
>
>
>
>
> On Wednesday, April 18, 2012 1:46:55 PM UTC-4, Greg D'Alesandre wrote:
>
> > Ah, c'mon now you are giving away the ending! ;) We are looking into
> > this as a possibility. BigQuery has some awesome capabilities and paired
> > with logs its quite effective!
>
> > Greg
>
> > On Wed, Apr 18, 2012 at 10:13 AM, Jason Collins <jason.a.coll...@gmail.com
> >> google-a...@googlegroups.com.
underway already.
If you create one, I'll star it.
j
On Apr 27, 10:25 am, someone1 <someo...@gmail.com> wrote:
> Was an issue created for this? I can't seem to find it and I would like to
> star it. I've been searching through my logs trying to determine a few
> errors we're getting and all this time I thought my lack of results was my
> fault. Kinda glad (or maybe not so glad) that it is Google's fault...
>
> -Prateek
>
>
>
>
>
>
>
> On Wednesday, April 18, 2012 1:46:55 PM UTC-4, Greg D'Alesandre wrote:
>
> > Ah, c'mon now you are giving away the ending! ;) We are looking into
> > this as a possibility. BigQuery has some awesome capabilities and paired
> > with logs its quite effective!
>
> > Greg
>
> >> > > > To unsubscribe from this group, send email to
> >> > > google-appengi...@googlegroups.com.
> >> > > > For more options, visit this group at
> >> > >http://groups.google.com/group/google-appengine?hl=en.
>
> >> > > --
> >> > > Takashi Matsuo | Developer Advocate | tmat...@google.com |
> >> 03-6384-9224
>
> >> > > --
> >> > > You received this message because you are subscribed to the Google
> >> Groups
> >> > > "Google App Engine" group.
> >> > > To post to this group, send email to
> >> google-a...@googlegroups.com.
> >> > >http://groups.google.com/group/google-appengine?hl=en.
>
> >> > > --
> >> > > Takashi Matsuo | Developer Advocate | tmat...@google.com |
> >> 03-6384-9224
>
> >> > > --
> >> > > You received this message because you are subscribed to the Google
> >> Groups
> >> > > "Google App Engine" group.
> >> > > To post to this group, send email to
> >> > > To unsubscribe from this group, send email to
> >> > > google-appengi...@googlegroups.com.
> >> > > For more options, visit this group at
> >> > >http://groups.google.com/group/google-appengine?hl=en.
>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "Google App Engine" group.
> >> To post to this group, send email to google-a...@googlegroups.com.
> >> > >http://groups.google.com/group/google-appengine?hl=en.
>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "Google App Engine" group.
> >> To unsubscribe from this group, send email to
> >> google-appengi...@googlegroups.com.
Prateek Malhotra
Apr 27, 2012, 4:08:15 PM4/27/12
to google-a...@googlegroups.com
http://code.google.com/p/googleappengine/issues/detail?id=7422
> >> google-appengine@googlegroups.com.
> >> > > > To unsubscribe from this group, send email to
> >> > > google-appengine+unsubscribe@googlegroups.com.
> >> > > > For more options, visit this group at
> >> > >http://groups.google.com/group/google-appengine?hl=en.
>
> >> > > --
> >> > > Takashi Matsuo | Developer Advocate | tmat...@google.com |
> >> 03-6384-9224
>
> >> > > --
> >> > > You received this message because you are subscribed to the Google
> >> Groups
> >> > > "Google App Engine" group.
> >> > > To post to this group, send email to
> >> google-appengine@googlegroups.com.
> >> > > To unsubscribe from this group, send email to
> >> > > google-appengine+unsubscribe@googlegroups.com.
> >> > > For more options, visit this group at
> >> > >http://groups.google.com/group/google-appengine?hl=en.
>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "Google App Engine" group.
> >> To post to this group, send email to google-appengine@googlegroups.com.
> >> To unsubscribe from this group, send email to
> >> google-appengine+unsubscribe@googlegroups.com.
Kenneth
Apr 29, 2012, 3:59:49 AM4/29/12
to google-a...@googlegroups.com
The symptom that really annoys me is when you have an error on your dashboard, click on it and there's nothing in the logs. Poof! It never happened!
Even fixing that use case would be great, something is putting it on the dashboard, maybe include the time of one of the errors on the dashboard so we can drill down that way? Time queries work well.
Prateek Malhotra
May 4, 2012, 9:18:31 AM5/4/12
to google-a...@googlegroups.com
Do you have a link to the log analysis demo? I'd be interested in learning more about it. (My Google-Fu didn't turn up any results)
> > > To post to this group, send email to google-appengine@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > google-appengine+unsubscribe@googlegroups.com.
> > > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
>
> > --
> > Takashi Matsuo | Developer Advocate | tmat...@google.com | 03-6384-9224
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appengine@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscribe@googlegroups.com.
Johan Euphrosine
May 4, 2012, 9:30:58 AM5/4/12
to google-a...@googlegroups.com
Demo is here:
--
Johan Euphrosine (proppy)
Developer Programs Engineer
Google Developer Relations
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/-RSMoQIbKt0J.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
Johan Euphrosine (proppy)
Developer Programs Engineer
Google Developer Relations
Prateek Malhotra
May 4, 2012, 10:21:24 AM5/4/12
to google-a...@googlegroups.com
*Mind Blown*
Looks really cool. It's always nice to see how Google uses their own APIs. This will be a good resource to learn from. Thank you very much!
Demo is here:
Reply all
Reply to author
Forward
0 new messages