I am trying to download millions of records from a remote DB (Amazon RedShift) into a local MongoDB database (3.0.2). However, an exception sometimes occurs as below.
>Processing up to User 180000
>Sent SQL to RedShift for retrieving individual users' logs
>Received individual users' logs successfully
>Inserting to local MongoDB...
Exception in thread "main" com.mongodb.MongoException$Network: Connection reset
at com.mongodb.WriteResult.getLastError(WriteResult.java:96)
at com.mongodb.WriteResult.getLastError(WriteResult.java:73)
at com.mongodb.WriteResult.getField(WriteResult.java:141)
at com.mongodb.WriteResult.getError(WriteResult.java:118)
at edu.njit.mongodb.MongoDBManager.insertOneLog(MongoDBManager.java:46)
at edu.njit.userlog.FreqUserLogDownloader.queryIndividualUserLog(FreqUserLogDownloader.java:122)
at edu.njit.userlog.FreqUserLogDownloader.main(FreqUserLogDownloader.java:158)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read1(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at org.bson.io.Bits.readFully(Bits.java:46)
at org.bson.io.Bits.readFully(Bits.java:33)
at org.bson.io.Bits.readFully(Bits.java:28)
at com.mongodb.Response.<init>(Response.java:40)
at com.mongodb.DBPort.go(DBPort.java:124)
at com.mongodb.DBPort.go(DBPort.java:88)
at com.mongodb.DBPort.findOne(DBPort.java:143)
at com.mongodb.DBPort.runCommand(DBPort.java:148)
at com.mongodb.DBPort.getLastError(DBPort.java:138)
at com.mongodb.DBPort.tryGetLastError(DBPort.java:171)
at com.mongodb.WriteResult.getLastError(WriteResult.java:94)
... 6 more
The MongoDB log shows (I am sorry that I do not keep the whole log):
***aborting after fassert() failure
The "edu.njit.mongodb.MongoDBManager.insertOneLog(MongoDBManager.java:46)" refers to my code:
public void insertOneLog(ResultSet rs) throws SQLException {
BasicDBObject doc = new BasicDBObject();
doc.put( "uid", rs.getString( "uid" ) );
doc.put( "url", rs.getString( "url" ) );
doc.put( "additionalinfo", reformatAddiInfo( rs.getString( "additionalinfo" ) ) );
WriteResult wr = coll.insert( doc );
if(wr.getError() != null) // This is the Line 46
System.err.println("Fail To Insert: " + wr.getError());
}
I am wondering the reason of this exception and how I can avoid it.
Thanks!