Gurantee log
46 views
Skip to first unread message
rekby
Sep 27, 2011, 5:06:09 PM9/27/11
to google-a...@googlegroups.com
Do log record gurantee for each request - I want write into log instructions for incremental change data and make incremental backup on my computer by download and parse log messages instead of analise datastore.
Ikai Lan (Google)
Sep 28, 2011, 4:40:34 PM9/28/11
to google-a...@googlegroups.com
We don't make guarantees, but so far no one has mentioned missing log messages except when we have a bug in our logging.
The issue with your plan is that logs are rotated out on a FIFO basis per log level if you run out of space, so unless you frequently download logs, you may miss entire blocks of data that will never be recoverable.
It sounds like this is a bit of a hack anyway. Can you describe what you are doing? Maybe we can come up with a better solution.
On Tue, Sep 27, 2011 at 2:06 PM, rekby <timofey...@gmail.com> wrote:
Do log record gurantee for each request - I want write into log instructions for incremental change data and make incremental backup on my computer by download and parse log messages instead of analise datastore.
--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/m7SraIrgxc8J.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Jan Zawadzki / Hapara
Sep 28, 2011, 7:01:26 PM9/28/11
to Google App Engine
This only works if you don't have a lot of traffic.
As Ikai mentioned, these are limited-size FIFO logs, so if you start
hitting any significant traffic, you will find that your INFO logs
hold a minute's worth of data - longer than a transfer would take you.
Write to a DB, OR, use RPC and URL fetch to "ping" an external server
that will store the Apache log files for you.
J
As Ikai mentioned, these are limited-size FIFO logs, so if you start
hitting any significant traffic, you will find that your INFO logs
hold a minute's worth of data - longer than a transfer would take you.
Write to a DB, OR, use RPC and URL fetch to "ping" an external server
that will store the Apache log files for you.
J
Timofey Koolin
Sep 30, 2011, 4:02:44 PM9/30/11
to google-a...@googlegroups.com
I have billing system on GAE.
It is thing to have backup of my datastore. Now I have small record count and can read all of it for backup every day. But in future - I need any incremental method of backup my data.
I have 3 things:
- log operations in logging system and replicate on my server.
- log operations in log kind and replicatite on my server (but it add pay for write, read and delete operations for each logical change in datastore).
- save timestamp field in my entities and log of deleted object.
save data small in log - cheap and faster then write it in datastore.
2011/9/29 Jan Zawadzki / Hapara <jan.za...@hapara.com>
--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
С уважением,
Кулин Тимофей.
Телефон: +7 (4852) 974793
ICQ: 114902104
email: tim...@koolin.ru
Blog: http://timofey.koolin.ru
Jan Zawadzki / Hapara
Oct 11, 2011, 8:12:53 PM10/11/11
to Google App Engine
Most people would expect to see a proper database record for any sort
of financial tracking. Log files aren't robust enough
On Oct 1, 9:02 am, Timofey Koolin <timofey.koo...@gmail.com> wrote:
> I have billing system on GAE.
> It is thing to have backup of my datastore. Now I have small record count
> and can read all of it for backup every day. But in future - I need any
> incremental method of backup my data.
> I have 3 things:
> - log operations in logging system and replicate on my server.
> - log operations in log kind and replicatite on my server (but it add pay
> for write, read and delete operations for each logical change in datastore).
> - save timestamp field in my entities and log of deleted object.
>
> save data small in log - cheap and faster then write it in datastore.
>
> 2011/9/29 Jan Zawadzki / Hapara <jan.zawad...@hapara.com>
> email: timo...@koolin.ru
> Blog:http://timofey.koolin.ru
of financial tracking. Log files aren't robust enough
On Oct 1, 9:02 am, Timofey Koolin <timofey.koo...@gmail.com> wrote:
> I have billing system on GAE.
> It is thing to have backup of my datastore. Now I have small record count
> and can read all of it for backup every day. But in future - I need any
> incremental method of backup my data.
> I have 3 things:
> - log operations in logging system and replicate on my server.
> - log operations in log kind and replicatite on my server (but it add pay
> for write, read and delete operations for each logical change in datastore).
> - save timestamp field in my entities and log of deleted object.
>
> save data small in log - cheap and faster then write it in datastore.
>
> Blog:http://timofey.koolin.ru
Ikai Lan (Google)
Oct 11, 2011, 8:19:10 PM10/11/11
to google-a...@googlegroups.com
I agree with Jan. If you're doing financial transactions, the cost of lost transactions is generally much, much more expensive than what you'd save not saving everything in the datastore. Don't try to save money here by using logs - you'll lose data.
Reply all
Reply to author
Forward
0 new messages