Database Inconsistency

[z33m]

Im writing a twitter app. It accepts commands as Direct Messages, so i
have setup a third party cronjob service to invoke a handler that
processes DMs at regular intervals. I have a Model 'Info' that has
just one entry, it stores some common data which are used in many
places in the App(in this case, the time when the messages were
processed recently). The general pattern of my handler is like this:

msgs = api.GetDirectMessages(since = info.msg_polled)
if not msgs:
return
logging.info('Processing Messages since %s ' % str(info.msg_polled))
for msg in msgs:
...process commands...
logging.info('Processed Message :- @%s : %s' %
(msg.sender_screen_name, msg.text))

info.msg_polled = datetime.datetime.now()
info.put()

But sometimes i get logs like this :

I 03-30 07:50AM 10.973
Processing Messages since Sun, 29 Mar 2009 11:41:59 GMT
I 03-30 07:50AM 11.122
Processed Message :- @foo : Foo_Bar
-------------------------------------------------------
I 03-30 07:46AM 08.014
Processing Messages since Sun, 29 Mar 2009 11:41:59 GMT
I 03-30 07:46AM 08.130
Processed Message :- @foo : Foo_Bar

Here, it seems that info is not getting commited to the database. The
message is processed multiple number of times, sometimes upto 10+
times before the msg_polled value changes. But i am not getting any
Datastore exceptions. This happens only once in a while.

Any help is appreciated.

[Jeff S]

Hi z33m,

Is it possible that the info model which contains the msg_polled time
is not being updated and still contains an old value for msg_polled? I
think it would be helpful to see the code/logic used to obtain the
last polled time info. It could be that multiple requests are started,
all of which use the last committed info, before the info.put() in one
of the requests is completed.

Thank you,

Jeff

[z33m]

I checked the whole code, im not caching the object or anything..
The info entry is actually used by a few other cronjob processes also,
they all change another field in the same record.. but none of them
use or change msg_polled value
could the problem be related to that?.. i mean, info is global data
like counters right?

The logic used to retrieve info is as below

class Info(db.Model):
msg_polled = db.DateTimeProperty(auto_now_add = True)
.... More Properties ....

@classmethod
def get_info(cls):
info = cls.all().get()
if not info:
info = cls()
info.put()
return info

info = Info.get_info()