Trouble with ConcurrentModificationException

[igorrumiha]

Hello,

I'm trying to build a web application with Scala and MongoDB. I'm
using mongo-java-driver 2.5.2 with casbah 2.1.1. Yesterday I tried to
do some load testing and I stumbled on an annoying issue: lots of
ConcurrentModificationExceptions thrown when I issue concurrent
requests to the web server.
The (shortened) stack trace is this:

Apr 6, 2011 7:41:21 AM com.mongodb.DBTCPConnector$MyPort error
SEVERE: MyPort.error called
java.util.ConcurrentModificationException
at java.util.AbstractList
$Itr.checkForComodification(AbstractList.java:372)
at java.util.AbstractList$Itr.next(AbstractList.java:343)
at org.bson.BSON.applyDecodingHooks(BSON.java:191)
at org.bson.BasicBSONCallback._put(BasicBSONCallback.java:150)
at org.bson.BasicBSONCallback.gotDate(BasicBSONCallback.java:106)
at org.bson.BSONDecoder.decodeElement(BSONDecoder.java:156)
at org.bson.BSONDecoder._decode(BSONDecoder.java:63)
at org.bson.BSONDecoder.decode(BSONDecoder.java:42)
at com.mongodb.Response.<init>(Response.java:63)
at com.mongodb.DBPort.go(DBPort.java:110)
at com.mongodb.DBPort.go(DBPort.java:75)
at com.mongodb.DBPort.call(DBPort.java:65)
at com.mongodb.DBTCPConnector.call(DBTCPConnector.java:215)
at com.mongodb.DBApiLayer$MyCollection.__find(DBApiLayer.java:295)
at com.mongodb.DBCursor._check(DBCursor.java:354)
at com.mongodb.DBCursor._hasNext(DBCursor.java:484)
at com.mongodb.DBCursor.hasNext(DBCursor.java:509)
at com.mongodb.casbah.MongoCursorBase$class.hasNext(MongoCursor.scala:
73)
at com.mongodb.casbah.MongoCursor.hasNext(MongoCursor.scala:497)
at scala.collection.Iterator$class.foreach(Iterator.scala:631)
at com.mongodb.casbah.MongoCursor.foreach(MongoCursor.scala:497)
at scala.collection.generic.Growable$class.$plus$plus
$eq(Growable.scala:48)
at scala.collection.TraversableOnce
$class.toList(TraversableOnce.scala:399)
at com.mongodb.casbah.MongoCursor.toList(MongoCursor.scala:497)
at com.rumiha.reputacija.controller.Pages$.makeHomePage(Pages.scala:
32)

These are all read operations (no writes at all). The query looks for
entries with a Date property larger than a certain time in the past
(I'm serializing org.joda.time.DateTime to the db). This results in 69
objects returned. For each object I do one additional query from a
very small collection (6 entries only).

I don't know if this has any significance but I noticed that all the
exceptions happen when decoding the Date field (the line "at
org.bson.BasicBSONCallback.gotDate(BasicBSONCallback.java:106)" is in
every stack trace.)

I'm not doing any thread management myself, this is all done by jetty.
I tried modifying the jetty thread pool size and also the number of
connections the Java Mongo driver makes to the Mongo server. No
difference. I analyzed the heap dump of the application and found only
one com.mongodb.Mongo instance (as suggested by the java driver
documentation.) This problem happens with any concurrency level above
1.

I don't see anyone else complaining about this so I'm thinking, I must
be doing something wrong. Any thoughts?

Thanks,

Igor

[axlfu]

I think it does not matter with mongdb.
What's your source code?
You shoule not modify your ArrayList(such as add/remove) while doing
foreach, is this arrayList is global and all threads can access?

[Nat]

Are you using any GridFS?
http://api.mongodb.org/scala/casbah/2.1.1/tutorial.html

[igorrumiha]

I'm not using any mutable data structures. This exception happens when
iterating through the result cusor and making a List out of it.
The code that does that looks like this:

val monthly = statsAggregatedColl.find(
(("statDate" $gte (DateTime.now -
12.months).withDayOfMonth(1).withTime(0,0,0,0)) ++
("range" -> "month")),
MongoDBObject(
"statDate" -> 1,
"totals" -> 1,
"uniqueVisitors" -> 1)).toList

statDate is the Date object, totals is an object that contains two
Long values, uniqueVisitors is a Long value. That's it.

As the stack trace shows, the exception is thrown when the code in org/
bson/BSON.java (static method applyDecodingHooks()) is iterating
through a List<Transformer>. That's not under my control.

[igorrumiha]

Nope...

On Apr 6, 9:10 am, Nat <nat.lu...@gmail.com> wrote:
> Are you using any GridFS?http://api.mongodb.org/scala/casbah/2.1.1/tutorial.html

[Nat]

How did you install JodaDateTimeDeserializer then? You should call it
only once.

[igorrumiha]

Yes! Thanks! That seems to be the issue.
I had trouble with determining the best place to register the
JodaDateTime Serializers/Deserializers. Even when I thought that the
serialisers/deserializers were active I would get the:

java.lang.IllegalArgumentException: can't serialize class
org.joda.time.DateTime

for the first invocation of the query. Sometimes, subsequent
invocations would work OK. So I resorted to calling
RegisterJodaTimeConversionHelpers() in every function that used joda
DateTime for storing or fetching from the DB. Now I know this is a bad
idea.

Now I added Deregister.... calls at the end of each function that
registers the serializers and the ConcurrentModificationExceptions are
completely gone. Now I do get the "can't serialize class
org.joda.time.DateTime" error like above. That issue I will fix more
easily, I hope.

Thanks again.

Igor.

[Brendan W. McAdams]

You should only run the Joda registration once, when you create a connection to MongoDB. In normal circumstances you should never need to deregister it and you'll definitely not want to repeatedly register.

> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>

[igorrumiha]

Yup, that's what I did now. I don't know why it didn't come to me
before...
Thanks.