How to analyze only the new data?

[dhanya]

Hi,

I am creating a view from a log file(HDFS input). This log file will
be updated frequently. I would like to update the view once per day.
Is it possible to read and update only the new data in the log file?
Please provide me with some examples.

Thanks,
Dhanya

[Leo Dagum]

Hi Dhanya,

Views are executed at runtime, if you want to exclude partial day results from the result set you can put that directly into the view definition.  The curdate() builtin function returns the current date, so your view definition should restrict by something like 'timestamp < curdate()' where "timestamp" is presumably some timestamp field in your logs.  The datetime builtin functions are documented in the 'scalar functions' chapter of:

http://cloudbase.sourceforge.net/index.html#userDoc

You'll need to make sure that date formats are consistent for the comparison to work properly.

Hope that helps,

- leo

---

From: dhanya <dhanya.a...@gmail.com>
To: CloudBase <cloudba...@googlegroups.com>
Sent: Tue, March 30, 2010 3:34:06 AM
Subject: How to analyze only the new data?

--
You received this message because you are subscribed to the Google Groups "CloudBase" group.
To post to this group, send email to cloudba...@googlegroups.com.
To unsubscribe from this group, send email to cloudbase-users+unsub...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cloudbase-users?hl=en.

[Dhanya Aishwarya Palanisamy]

Hi Leo,

I am sorry for a late response. I am not able to understand clearly your reply. I have created the view from a log file. Now I would like to update the view to include only new records.
In SQL we use Create or Replace View, will that work for cloudbase.

Please do explain.

Thanks,
Dhanya

To unsubscribe from this group, send email to cloudbase-use...@googlegroups.com.

[Leo Dagum]

Hi Dhanya,

 

I guess I don't really understand what you're trying to do.  But if you want to change the view, then you need to do a 'drop view' and then a 'create view' with the new view logic.

 

Thanks,

 

- leo

---

From: Dhanya Aishwarya Palanisamy <dhanya.a...@gmail.com>
To: cloudba...@googlegroups.com
Sent: Fri, April 9, 2010 3:34:50 AM
Subject: Re: How to analyze only the new data?