Query on Mongo hexadecimal string using lift record module not returning values.

[Gautam Mr]

Issue:
1.If I store a hexadecimal string as a String field in a SubDocument (which is of type JsonObject) then it gets stored as ObjectId, does not store as string. And because of which queries like ObjectIdTestColl.findAll ("subDocList.userid" -> "4cb6dd2d5a53a0bc7ae819d5") does not return any value.
2. And if I store a hexadecimal string as StringField under MongoRecord (in blow example ObjectIdTestColl.userId ), though it gets stored as String, I can't get the results through queries like  ObjectIdTestColl.findAll ("userId" -> "4cb6dd2d5a53a0bc7ae819d5")

Can someone tell me how to solve this issue?
Would appreciate your help.

Thanks
Gautam

Below are the code samples:

ObjectIdTestColl is the collection which holds a subdocument ObjectIdSub which has an a String attribute

package rnd.objectidtest
import java.util.UUID
import net.liftweb.mongodb.{JsonObject, JsonObjectMeta, MongoDB, DefaultMongoIdentifier, MongoAddress, MongoHost}
import net.liftweb.mongodb.record.{MongoRecord, MongoId, MongoMetaRecord}
import net.liftweb.mongodb.record.field.{MongoPasswordField, JsonObjectField, MongoListField, ObjectIdField, MongoJsonObjectListField}
import net.liftweb.record.field.{StringField, IntField, BooleanField, DateTimeField, DoubleField, EmailField, LocaleField, LongField, TimeZoneField}

class ObjectIdTestColl extends MongoRecord[ObjectIdTestColl] with MongoId[ObjectIdTestColl] {
    def meta = ObjectIdTestColl
    object name extends StringField(this, 30)
    object userId extends StringField(this, 30) //pk from User Collection
    object subDocList extends MongoJsonObjectListField[ObjectIdTestColl, ObjectIdSub](this, ObjectIdSub)

}
object ObjectIdTestColl extends ObjectIdTestColl with MongoMetaRecord[ObjectIdTestColl] {
  override def collectionName = "objectidtest"
}

-------------------------------------------------------------------------
package rnd.objectidtest
import net.liftweb.mongodb.{JsonObject, JsonObjectMeta}

class ObjectIdSub (var userId:Option[String], var subName:Option[String]) extends JsonObject [ObjectIdSub] {
    def meta = ObjectIdSub
}
object ObjectIdSub extends JsonObjectMeta [ObjectIdSub]
--------------------------------------------------------------------

package rnd.objectidtest
import net.liftweb.mongodb.{MongoDB, DefaultMongoIdentifier, MongoAddress, MongoHost}
import socialrpm.model.dataaccess.persistence.context._
import net.liftweb.json.JsonDSL._

object ObjectIdTest {
    def initializeMongo {
           MongoDB.defineDb(DefaultMongoIdentifier, MongoAddress(MongoHost("localhost", 27017), "test"))
    }

    def main (args:Array[String]) {
        initializeMongo
        createRecord
        findRecord
       
     }

    private def findRecord {
        val results:List[ObjectIdTestColl] = ObjectIdTestColl.findAll ("userId" -> "4cb6dd2d5a53a0bc7ae819d5")
        //val results:List[ObjectIdTestColl] = ObjectIdTestColl.findAll ("subDocList.userid" -> "4cb6dd2d5a53a0bc7ae819d5")
        println ("size:::" + results.size)
        results.foreach { result:ObjectIdTestColl =>
            println ("id::" + result.id)
        }
    }
private def createRecord {
        val oi:ObjectIdTestColl = ObjectIdTestColl.createRecord
        oi.name ("RPM")
        oi.userId ("4cb6dd2d5a53a0bc7ae819d5")
        oi.subDocList (fillSubDoc)
        oi.save
    }

    private def fillSubDoc:List[ObjectIdSub] = {
        val s:ObjectIdSub = new ObjectIdSub (None, None)
        s.userId = Some ("4cb6dd2d5a53a0bc7ae819d5")
        s.subName = Some ("subname")
        List (s)
    }

}
-----------------------------------

[Tim Nelson]

Hi,

Your first issue is just a typo. Change "subDocList.userid" to
"subDocList.userId" and you should be good.

For the second issue, you have 2 options. The first is to change
userId to an ObjectIdField. Your query will work as is if you do this.

The second option is to change how you build your query. Instead of
using the JsonDSL, use com.mongodb.QueryBuilder and pass in the String
value of the ObjectId. This way it won't get converted to an actual
ObjectId.

Tim

> --
> You received this message because you are subscribed to the Google Groups
> "Lift" group.
> To post to this group, send email to lif...@googlegroups.com.
> To unsubscribe from this group, send email to
> liftweb+u...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/liftweb?hl=en.
>

[Gautam Mr]

Thanks Tim for your quick response. Its my bad, was a typo while putting it in this email.

OK now I get what exactly is happening, but I don't really get why it needs to be like the way it is.

1. Looks like lift assumes any 24 character hexadecimal string as ObjectId and converts it to ObjectId before persisting (if I use lift while creating a record). Since it gets stored as ObjectId it works fine while doing doing JsonDSL query. Now if I use any other script or mongo shell to add / update a new record then it would store as a String not as ObjectId. Now thats exactly what happened in my case. Now as it gets stored as a String, lift query would never be able to find it, as it would always convert to an ObjectId before querying.
I don't understand the logic behind amusing any 24 character hexadecimal number to be an ObjectId.

2. I cannot declare the attribute as ObjectId as I don't intend to store ObjectId. But at times it gets some value which are 24 char hex and that's where it causes issue.

Why lift needs to assume any 24 char hex as ObjectId, can't it be explicit through declaration?

3. One more thing which I found of inconvenience is the whole mongo update through MongoMetaRecord. It does not return anything, if something goes wrong while executing update it stays silent. Does not propagate any message back. Is there a way to get these details?

Once again appreciate your response, it really helps.

Thanks
Gautam

[Tim Nelson]

Hi,

1-2: Every String that passes through JObjectParser (JsonObjects) is
tested to see if it is a valid ObjectId using ObjectId.isValid. If it
is it's transformed to an ObjectId. At the time I wrote this, that was
the only way I could figure out to handle ObjectIds, which is required
for MongoDocument. Unfortunately, those that wrote their apps with
this convention rely on this behavior. I do have a patch up on
reviewboard that will allow for a better way to handle this, but we
decided to leave the isValid checks in for backwards compatibility.

3. This is the default MongoDB behavior. If you want to make sure the
update happened correctly, you have to call getLastError directly on
the database. This is what MongoDB.useSession is for, the call to
getLastError must be on the same thread that your update is on,
useSession ensures that. There is a save function that you can use
that will throw an exception, if there was an error. You just need to
call it with save(true) to turn on strict-mode. See [1] for more info
on getLastError. This is the save function for reference:

def save(inst: BaseRecord, strict: Boolean): Boolean = saveOp(inst) {
if (strict) // strict mode will call getLastError and throw an
exception if there was an error
MongoDB.useSession(mongoIdentifier) {
db =>
val coll = db.getCollection(collectionName)
coll.setWriteConcern(DB.WriteConcern.STRICT)
coll.save(inst.asDBObject)
coll.setWriteConcern(DB.WriteConcern.NORMAL)
}
else
MongoDB.useCollection(mongoIdentifier, collectionName) {
coll => coll.save(inst.asDBObject)
}
}

Tim

1 - http://www.mongodb.org/display/DOCS/Last+Error+Commands

[Tim Nelson]

I just realized I was telling you about getLastError and then I showed
you an example that uses WriteConcern. WriteConcern.STRICT tells
mongo-java-driver to call getLastError, and if there was an error,
throw an Exception. You can either use WriteConcern as I did in the
save function, or you can call getLastError and handle it however you
want.

[Gautam Mr]

Thanks Tim for sharing the details. This helps a lot.

Thanks
Gautam