And can this be done portably across UNIX and Windows? Thanks.
Where is the data coming from? Two suggestions:
1) There is a MySQL option to not commit every insert, sorry I can't remember
what it is, but it REALLY speeds things up (especially if you have lots of
keys). Or drop all the keys load, the data, and recreate the keys. Sounds odd,
but it is a recommended method.
2) If data is coming from a file now, use LOAD DATA directly, don't go through
Python. You may even find writing to file and then loading it is very fast. I
recommend tab delimited as that seems to work well.