How can I add a new field to a schema and then update existing records with new field data?

[Ken Kopelson]

Greetings :)

I'm really enjoying using Mongoose in the MEAN stack I'm developing with! Kudos to the teams working on all this great software!  I'm having one issue that I've been wresting all day and can't seem to make work, no matter how much I've scoured the documentation I could find online.

Basically, I have an existing schema and existing data in the database.  I want to add a new field to the schema, which is an array of strings.  Then, when I write any existing records back as an update, I want to be able to add the new field information to those records.  I know this can easily be done in MongoDB itself, but I very much like the schema system that Mongoose provides, so I don't want to eliminate it at all.  It just seems that it must be possible to do this, since even relational DBs can add a new field to a table and allow you to start filling the data into existing rows.

I've tried simply adding the new field to the schema, but it gets ignored by Mongoose when I do save.  Then, I've also specified the "add" method on the schema, but same result.  Then, I specified "strict: false" on the schema, but still the same thing.  I also tried doing a save specifying { $set: { workUnits: ['value']}} as the body, but that didn't work either.  Here is the schema as it now stands.

var busUnitSchema = new Schema({
  uniqueId: { type: String, unique: true },
  name: { type: String, index: true },
  tags: [String],
  companyUnit: { type: Schema.Types.ObjectId, ref: 'BusUnit'},
  parentUnit: { type: Schema.Types.ObjectId, ref: 'BusUnit'},
  billing: [{
    amount: Number,
    payType: { type: String, enum: consts.PAY_TYPE },
    payFreq: { type: String, enum: consts.PAY_FREQ },
    chargeToUnit: { type: Schema.Types.ObjectId, ref: 'BusUnit'},
    ccNumber: String,
    expDate: Date,
    nameOnCard: String
  }],
  contactUser: { type: Schema.Types.ObjectId, ref: 'User' },
  contactEmail: String,
  workUnits: [String],
  lastUpdatedBy: { type: Schema.Types.ObjectId, ref: 'User' }
},{strict: false});
busUnitSchema.add({ workUnits: [String] },'');

I'm thinking that if this is not something supported, that is a huge hole for actually being pleasant to use in production.  If that IS the case, I'm interested in people's thoughts on what I can do to make it work, perhaps developing a plugin or something if needed.

Many thanks for any help on this :)

[Joe Wagner]

Hey Ken,

If I understand your question, it should be easy to do this with Mongoose, e.g.

BusUnit.findOne({uniqueId: 'some old existing doc id'}, function (err, doc) {
    if (err) return done(err);
    // Create the new field if it doesn't exist yet
    doc.workUnits || (doc.workUnits = [];)
    doc.workUnits.push('value');

    doc.save(done);
});

One thing to note is that you don't have to use the $set operator explicitly when doing a save, Mongoose uses it for you.

Sorry if I'm not understanding your issue, but I hope that helps.

-Joe

[Ken Kopelson]

Hi Joe,

Hey, thanks a million for that answer.  It actually worked!  I know it seems silly now, but I tried so many things yesterday.  The one thing I DIDN'T try was adding an empty array and then transferring each array item to it.  I was just assigning the whole array to the destination field: doc.workUnits = incomingArray.  So I guess it was just assigning a reference and not copying the array.  I've solved that now by making a function that updates an existing record (in the backend) from an incoming object (from the front-end) that can have just a subset of the items to adjust.  As a thanks for your answer, here are a couple handy functions I wrote to work with JSON (the compare in particular is bloody fast, comparing good sized JSON objects in less than 1 ms)  The compare works great for change detection on a form:

/**
 * Compare two JSON strings
 * Author: Ken Kopelson
 */
module.exports.compareStr = function(str1, str2) {
  return this.compare(JSON.parse(str1),JSON.parse(str2));
};


/**
 * Compare two JSON objects to see if they represent the same information
 * Note: this is only for JSON objects, which is a subset of Javascript Objects
 * Author: Ken Kopelson
 */
module.exports.compare = function(obj1, obj2) {
  var count1 = 0;
  if (obj1 && obj2) {
    for (var k in obj1) {
      if (obj1.hasOwnProperty(k)) {
        ++count1;
        var frObj = obj1[k];
        var toObj = obj2[k];
        if ('object' !== typeof frObj) {
          if (frObj !== toObj)
            return false;
        }
        else if (frObj instanceof Date) {
          if (!(toObj instanceof Date))
            return false;
          if (frObj.getTime() !== toObj.getTime())
            return false;
        }
        else if ('object' === typeof frObj) {
          /**
           * This will handle objects and arrays
           */
          if ('object' !== typeof toObj)
            return false;
          if (!this.compare(frObj, toObj))
            return false;
        }
        else {
          throw new Error('Invalid type encountered by JSON compare');
        }
      }
    }
  }
  else
    return false;
  return count1 === Object.keys(obj2).length;
};


/**
 * Deep copies all the members of one JSON object into another JSON object
 * WARNING: this ONLY works for JSON objects, NOT Javascript objects
 * @param obj1 source object holding the new field values
 * @param obj2 destination object
 * @returns {boolean}
 */
module.exports.updateObj = function(obj1, obj2) {
  if (obj1) {
    for (var k in obj1) {
      if (obj1.hasOwnProperty(k)) {
        var frObj = obj1[k];
        var toObj = obj2[k];
        if ('object' !== typeof frObj) {
          obj2[k] = frObj;
        }
        else {
          if (frObj instanceof Date){
            obj2[k] = new Date(frObj.getTime());
          }
          else if (frObj instanceof Array) {
            // Always make the target array new...
            obj2[k] = [];
            this.updateObj(frObj, obj2[k]);
          }
          else if (frObj instanceof Object){
            if (!toObj)
              obj2[k] = {};
            this.updateObj(frObj, obj2[k]);
          }
        }
      }
    }
  }
};

[diwyanshu tomar]

Hye Joe,

  i as a beginner with mongooseJS. is completely overwhelmed with the issue of manipulating the schema structure. Like the issue of "adding a new field  to existing schema."

Considering the scenario of the original post where "ken" mentions the same issue .

let us say, if we have a thousand records in db upon the original schema for "BusUnit", how will we be able to  make changes to exiting 1000 records. I am quite in need of a solution for this situation.

As, it is stated that MongoDB is a No-SQL database, which utilizes the fact that initial development of an application does not need to have a well developed blueprint in terms of data field requirements, they can change quickly as per the business logic is required.

Now if we use mongoose to control  our Model structure on top of MongoDB, there must a solid workaround for such situation.

Need your help on this issue.

Thanks

Diwyanshu

[Jason Crawford]

You can add a field to a schema without updating any records. The existing records just won't have a value for that key. That's OK.

You should ideally write your code to handle that case.

However, if you do need to modify the existing records for some reason, you can just write a simple script to iterate through all of them and update them. Or even issue a batch update call to update them all at once, if you just want to set them all to some constant value.

Hope that helps,

Jason

--
Documentation - http://mongoosejs.com/
Plugins - http://plugins.mongoosejs.com/
Bug Reports - http://github.com/learnboost/mongoose
Production Examples - http://mongoosejs.tumblr.com/
StackOverflow - http://stackoverflow.com/questions/tagged/mongoose
Google Groups - https://groups.google.com/forum/?fromgroups#!forum/mongoose-orm
Twitter - https://twitter.com/mongoosejs
IRC - #mongoosejs
---
You received this message because you are subscribed to the Google Groups "Mongoose Node.JS ODM" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongoose-orm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[angelcam...@gmail.com]

Hi Ken, I'm new using mongodb and I'm the same issue as you, I read Joe's answer but didn't understand so much. Could you explain to me a little better where to exactly execute those lines to add a field?