Chunksize not working in pandas.DataFrame.to_sql

[gayathri nagarajan]

I use 0.15 version of panda. I use the below code, but the chunksize parameter is not working,It doesn't  write the records in batches to the DB. I totally have 50,000 records in my DF and it takes more time to execute. Kindly help. 

from sqlalchemy import create_engine

import iopro.pyodbc

engine = create_engine('mssql+pyodbc://147.117.20.144/HRMS_Test')

df.to_sql('test',engine, if_exists='append',index= False, index_label=None, chunksize=100)

[gayathri nagarajan]

from sqlalchemy import create_engine

import iopro.pyodbc

engine = create_engine('mssql+pyodbc://server_name/table_name')

df.to_sql(table_name,engine, if_exists='append',index= False, index_label=None, chunksize=100)

[Joris Van den Bossche]

How do you know it does not write the records in batches?

That it takes a bit longer is not surprising I think, as more insert statements are called.

Regards,
Joris

--
You received this message because you are subscribed to the Google Groups "PyData" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pydata+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Joris Van den Bossche]

see also https://github.com/pydata/pandas/issues/8729

[Paul Hobson]

My guess is that it is writing in batches, but not committing the inserts until the all of them have been completed.

-p