Ignoring errors!!

[Waleed Abdulla]

I noticed that when an update fails for some reason (unauthorized, bug in after_send, ...etc) that App Rocket simply continues with the next record. While that might be good for certain cases, it has some drawbacks:

- In my first run, I had an Unauthorized error (wrong key), but the daemon continued pushing all updates and getting the same error again and again. It won't stop until you either kill the process or let it go through the whole table. 

- When a record is skipped, you basically lose that update. Since App Rocket uses the update timestamp to determine what to replicate, there is no way to replicate only the missed records. So when an error happens, then it won't help to simply fix the problem and run app rocket again. You need to reset the AppRocket timestamp to an older time, but that will replicate everything since that time, not only the missed records.

- And, finally, if you lose records here and there, it's hard to notice and the problem could go unnoticed for a long time. 

    I believe the default behavior should be to stop the replication (after a few attempts) and wait for someone to fix the issue. This way, after fixing the problem, the replication continues from where it stopped. 

My 2 cents, 

Waleed

[Waleed Abdulla]

Looking at the code, it looks like the intention is actually to stop on errors after 3 attempts, but it's not doing that for all errors (specifically, errors that happen on the app engine side don't seem to stop the process). If anyone is interested, here is a quick update:

In station.py, in send_now(), replace this line:

    logging.error(self.name + u"%s" % response)

with this:

    raise Exception("Rocket error: " + self.name + u"%s" % response)

Waleed

[Kurt Daal]

This is now fixed in Subversion trunk.