Sqlalchemy delete, update or keep row iterating large dataset in sqlite

[tec...@gmail.com]

Hi guys,
I've already asked about my problem on stackoverflow (http://stackoverflow.com/questions/33211941/sqlalchemy-delete-update-or-keep-row-in-large-dataset). It seems like hit some unusual corner case in my application because no one could help me out.

I'm working on simple but large dataset in sqlite using sqlalchemy. I need to iterate over all rows in a table and based on some business logic either update, delete or keep the row.

Because of the size of my data I used the code from http://stackoverflow.com/a/13670855/5462298 to page my query data. The code works fine as long as no rows are deleted during the processing. Otherwise it will skip over some data. Using yield_per() doesn't seem to work using sqlite for me because I update data during the query which result in a "(sqlite3.ProgrammingError) Cannot operate on a closed database." exception.

What is the proper solution using sqlalchemy for my problem?

Greetings and Thanks in advance,

 Tech Q.