Updates from 2013-07-18

[Michael McCallister]

Parsing and loading now seems to be working well. After the current run completes, I will have pulled all the Goko games in from 2013-03-01 to the most recent day available on archive-dominionlogs.goko.com. Now, I have to tackle the challenge of the scanner.

Many of the processes run using the incremental_scanner object to track which games they've seen/processed. This scanner assumes that a "high water mark" game ID can be used to track the most recent game processed, but Goko game ids don't work that way.

I need to implement a new scanner that works around this, and then re-implement the analyze, goals, trueskill, buys, etc. to leverage this new scanner.

I'm probably going to do some of this work in a local development environment because I'm nervous about iterating on this big of a change in the prod environment and potentially fouling up the data.

Mike