Converting an app that uses a storage interface to mongo

[Vaevictus]

Hi all,

I am looking into using mongo for our web application that I have taken over the maintenance for and have several questions so please excuse all the spam :)

My first question is how I can integrate mongo into our framework model. This _should_ be easy as when the app was designed the storage layer was abstracted away from the developer using a storage interface which implement the following methods (there are more but I have left them out for simplicity)

 /// <summary>

        /// Returns a ArrayList of objects that comply to the provided template. Query by example using <paramref name="template"/> as the example.

        /// </summary>

        /// <param name="template">Search example</param>

        /// <returns>Arraylist of objects that are similar to the example provided.</returns>

        public override ArrayList Get(object template)

        {

            return this.Get(template, null, null);

        }

        public override ArrayList Get(object template, ConstraintList constraints)

        {

            return this.Get(template, constraints, null);

        }

        public override ArrayList Get(object template, ConstraintList constraints, SortExpression sort)

        {

        //do some mongo stuff

     

        }

    public override int GetCount(object template)

        {

            return 1;

        }

        public override int GetCount(object template, Search.ConstraintList constraints)

        {

           

        }

        public override Dictionary<object, int> GetDistinctCount(object template, string propertyName)

        {

           

        }

        public override Dictionary<object, int> GetDistinctCount(object template, string propertyName, Search.ConstraintList constraints)

        {

           

        }

         public override void Delete(object obj)

        {

            return;

        }

public override void Set(object obj)

        {

            //what type are we getting?

            string typename = obj.GetType().ToString();

            string connectionString = "mongodb://localhost";

            MongoServer server = MongoServer.Create(connectionString);

            MongoDatabase dataSource = server.GetDatabase(this.dataSourceDescription);

            using (server.RequestStart(dataSource))

            {

                MongoCollection<BsonDocument> coll =

                 dataSource.GetCollection<BsonDocument>(typename);

                //        BsonDocument book = new BsonDocument {

                //    { "author", "Ernest Hemingway" },

                //    { "title", "For Whom the Bell Tolls" }

                //};

                coll.Insert(obj);

            }

        }

//this is essentially the update command

public override object SetRemoteObject(object obj)

        {

}

   public override object GetObjectByID(string storageID, Type type)

        {

            string typename = type.ToString();

            string connectionString = "mongodb://localhost";

            MongoServer server = MongoServer.Create(connectionString);

            MongoDatabase dataSource = server.GetDatabase(this.dataSourceDescription);

            using (server.RequestStart(dataSource))

            {

                // a series of operations that must be performed on the same connection

                MongoCollection<BsonDocument> coll = dataSource.GetCollection(typename);

                var query = new QueryDocument();

                query.Add("id", BsonValue.Create(storageID));

                //loop through the template object and foreach non-null property add it to the query, TODO must be a better way

                BsonDocument document = coll.FindOne(query);

                return document;

            }

        }

As you can see, every call to a storage function in the application passes a template object (which differ in type of course depending what part of the app/what plugin is calling the funciton etc)

my problem is using this template object to build a query

here is an example of a call to get using a template object:

 User template = (User)FormatterServices.GetUninitializedObject(typeof(User));

            template.GlobalLogonName = User.getGlobalLogonName(Configuration.ServerName, logonName);

            ArrayList resultList = container.Get(template);

 Now, because the class User contains datetimes which cannot be null, when I build a querydocument by iterating through all the templates properties using reflection the querydocument will never return a value from the db because it's searching for users that have datefields initialised to 1/1/1970

 

Here is my junk code that I am currently using to query the db.

The below code works well because all I am interested in is the document ID:

public override object GetObjectByID(string storageID, Type type)

        {

            string typename = type.ToString();

            string connectionString = "mongodb://localhost";

            MongoServer server = MongoServer.Create(connectionString);

            MongoDatabase dataSource = server.GetDatabase(this.dataSourceDescription);

            using (server.RequestStart(dataSource))

            {

                // a series of operations that must be performed on the same connection

                MongoCollection<BsonDocument> coll = dataSource.GetCollection(typename);

                var query = new QueryDocument();

                query.Add("id", BsonValue.Create(storageID));

               

                BsonDocument document = coll.FindOne(query);

                return document;

            }

        }

   public override ArrayList Get(object template, ConstraintList constraints, SortExpression sort)

        {

            //what type are we getting?

            Type t = template.GetType();

            string typename = t.ToString();

            //todo config

            string connectionString = "mongodb://localhost";

            MongoServer server = MongoServer.Create(connectionString);

            MongoDatabase dataSource = server.GetDatabase(this.dataSourceDescription);

            

            ArrayList result = new ArrayList();

            using (server.RequestStart(dataSource))

            {

                // a series of operations that must be performed on the same connection

                MongoCollection<BsonDocument> coll = dataSource.GetCollection(typename);

                QueryDocument query = buildQueryDocumentFromObjectTemplate(template);

                

                //if query length is 0, then we want all documents of this type in the db

                if (query.Count() == 0)

                {

                    MongoCursor<BsonDocument> results = coll.FindAll();

                    foreach (BsonDocument document in results)

                    {

                        //result.Add(document);

                        // object someObject = document.Deserialize(BsonReader.Create(document), template.GetType(), null);

                        try

                        {

                            object someObject = BsonSerializer.Deserialize(document, t);

                            result.Add(someObject);

                            string breaker = "dfdf";

                        }

                        catch (Exception e)

                        {

                            //me

                            string test = "sd";

                        }

                    }

                }

                else

                {

                    foreach (BsonDocument document in coll.Find(query))

                    {

                        // object someObject = document.Deserialize(BsonReader.Create(document), template.GetType(), null);

                        try

                        {

                            object someObject = BsonSerializer.Deserialize(document, t);

                            result.Add(someObject);

                            

                        }

                        catch (Exception e)

                        {

                            string grrrrr = "sdsdsd";

                        }

                    }

                }

                //somehow convert my results to the original type so calling method can work with them

                //

               

            }

            //; 

            //try

            //{

            //    server.RequestDone();

            //}

            //catch

            //{

            //    Debug.WriteLine("couldnt close request");

            //}

            Debug.WriteLine("done" + typename);

            return result;

            //dataSource.GetCollection(template.GetType().ToString());

        }

 private static QueryDocument buildQueryDocumentFromObjectTemplate(object template)

        {

            var query = new QueryDocument();

            Type t = template.GetType();

            //loop through the template object and foreach non-null property add it to the query, TODO must be a better way

            foreach (PropertyInfo p in t.GetProperties())

            {

                if (p.GetCustomAttributes(typeof(Storage.Attributes.StorageIgnoreAttribute), false).Count() > 0)

                    continue;

                // Write the name and the value

                if (p.GetValue(template, null) != null)

                {

                    try

                    {

                        query.Add(new BsonElement(p.Name, BsonValue.Create(p.GetValue(template, null))));

                    }

                    catch

                    {

                    }

                }

            }

            return query;

        }

So, in short, whats the best way to query mongodb using a search template object that can have any type and return the search results as an arraylist of objects.

Thanks for any pointers,

Craig

[Robert Stam]

Wow. That's pretty long... I'll try to find some of the questions in
there and answer them.

1. public override void Set(object obj)

The code seems to imply that obj could be of any type (but probably
not BsonDocument). Be aware that when you call coll.Insert(obj) the
driver is going to serialize the obj instance to a BsonDocument (using
an automatically generated BsonClassMap unless there is a custom
serializer registered for obj's type).

2. public override void GetObjectByID(string storageID, Type type)

You are returning an instance BsonDocument, which is not symmetrical
with Set, which accepts any type. Use BsonSerializer.Deserialize like
you did in the Get method.

3. Template objects with struct values

Query by example works best where the template classes either have a
dynamic set of members (like a dictionary) or where none of the
members are structs (so null can be used to indicate omitted values).
You can either switch to Nullable<T> types for your structs (e.g.
DateTime? in C# syntax) or change your reflection code to check for
the "empty" values. If you were writing a generic method you could
compare against default(T), but since you are using reflection you are
probably going to have instantiate an empty value of your struct type
to compare against (and equality to the default value could imply a
member to omit from the query).

4. RequestStart

You rarely need to use RequestStart unless you are in a multithreaded
environment and you are concerned about your writes becoming
immediately visible to the same thread that wrote it (so RequestStart
says do all operations for this thread using the same connection).
Otherwise just let connection pooling do its thing.

5. public override ArrayList Get(object template, ConstraintList
constraints, SortExpression sort)

You don't actually need two different queries. You can pass null to
the query parameter of Find and it will behave like FindAll. You also
have a comment about converting the results to the original type, but
they should already be of the original type because you passed that
type as the second argument to BsonSerializer.Deserialize.

6. private static QueryDocument
buildQueryDocumentFromObjectTemplate(object template)

This is where your structs are tripping you up (e.g. DateTime) because
they are never null. I'm not really sure how to best test for this,
but here's a code snippet that seems to work:

publicstatic bool IsNullOrDefault(object obj) {
if (obj == null) { return true; }
var type = obj.GetType();
if (!type.IsValueType) { return false; }
var defaultValue = Activator.CreateInstance(type);
return obj.Equals(defaultValue); // must use Equals not ==

[Vaevictus]

Robert, 

Thanks a bunch for your detailed reply, it's given me a lot to chew on and am sure it will help a lot.

Cheers,

Craig