BLK Load API and memory use

[mpeppler@peppler.org [Team Sybase]]

We're using the ENABLE_BULK_LOAD connection property which lets
jConnect use the bulk-load API via the preparedStatement API. This
works very well, but we're seeing a strange memory usage issue, where
the parameters for rows sent to the server are not freed after
executeBatch(), even if we call clearParameters()/clearBatch().

We use the following loop:

PreparedStatement writeStmt = blk.prepareStatement("insert
mp_bulk_test values(?, ?, ?, ?, ?, ?, ?, ?, ?)");

// Thread.sleep(10000);

long s0 = System.currentTimeMillis();
int count = 0;
int batchSize = 10000;
while(true) {
int id = count;
String tx = "Testing " + count;

writeStmt.setInt(1, id);
writeStmt.setString(2, tx);
writeStmt.setString(3, tx);
writeStmt.setString(4, tx);
writeStmt.setString(5, tx);
writeStmt.setString(6, tx);
writeStmt.setString(7, tx);
writeStmt.setString(8, tx);
writeStmt.setString(9, tx);
writeStmt.addBatch();
if (++count % batchSize == 0) {
System.out.println("Flushed " + batchSize + " rows (total " +
count + " rows)");
long t0 = System.currentTimeMillis();
writeStmt.executeBatch();
blk.commit();
long t1 = System.currentTimeMillis();
System.out.println("Commit took " + (t1 - t0) + " ms");

writeStmt.clearBatch();
writeStmt.clearParameters();

// writeStmt.close();
// writeStmt = blk.prepareStatement("insert mp_bulk_test
values(?, ?, ?, ?, ?, ?, ?, ?, ?)");
}

if (count > 1000000) {
break;
}
}
writeStmt.executeBatch();
blk.commit();
writeStmt.close();
long s1 = System.currentTimeMillis();
System.out.println("Wrote 1000000 rows in " + (s1 - s0) + " ms");


In my test I'm using jConnect (TM) for JDBC(TM)/7.00(Build 26544)/P/
EBF19093/JDK16/Fri May 13 4:16:47 2011

If we close and recreate the preparedStatement after each commit the
memory use stays well bounded, and the load is in fact faster
(probably because jConnect doesn't need to walk through large arrays
of data...)

Any idea what we might be doing wrong here?

Thanks!

Michael