Appengine Logs and BigQuery

[aswath satrasala]

Hi,

I am sending the Appengine Log to BigQuery using the LogViewer screen.

In the BigQuery browser, I see that a table is created for every 1 day of log. 

Question:

How can I send the logs to the same table instead of to different table every day.

-Aswath

www.AccountingGuru.net

www.VeersoftSolutions.com

[aswath satrasala]

I want to analyze the logs over a period of time. How can I run queries, if the logs are sent to a new bigquery table every day. 

[Nickolas Daskalou]

You can use table unions, as explained on this page under the FROM heading:

https://cloud.google.com/bigquery/query-reference#from

To quote that page:

===========
Note: Unlike many other SQL-based systems, BigQuery uses the comma syntax to indicate table unions, not joins. This means you can run a query over several tables with compatible schemas as follows:

// Find suspicious activity over several days
SELECT FORMAT_UTC_USEC(event.timestamp_in_usec) AS time, request_url
  FROM [applogs.events_20120501], [applogs.events_20120502], [applogs.events_20120503]
  WHERE event.username = 'root' AND NOT event.source_ip.is_internal;

Queries that perform a union over large number of tables can expect to run more slowly than the same query over a single table with the same amount of data. The difference in performance can be up to 50 ms per additional table. The maximum number of tables you can perform a union on is 1,000.
===========

Hope this helps.

Nick

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CAKFF5G6DnF40ORj7ca4jWt%3DRtFVFnBCnf_-sE634xZ8v6BYjrQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Jason Collins]

Even easier: use table wildcards.

 https://cloud.google.com/bigquery/query-reference#tablewildcardfunctions