Can I introduce new v3.2 servers to a RS of v2.4 servers?

1,254 views
Skip to first unread message

Bruno Bronosky

unread,
Oct 26, 2016, 12:11:28 AM10/26/16
to mongodb-user
Before you tell me to RTFM, I have read:
https://docs.mongodb.com/manual/release-notes/2.6-upgrade/ as well as countless other docs. I'm still asking the question because I cannot get a clear answer on whether those different versions can co exist in Replica Set if no upgrading is going to take place.

I have a Replica Set of 3 very old MongoDB v2.4 servers running on AWS. I'd like to replace these 3 rather than upgrade them because the OS hasn't been maintained. I'm new to this role and I don't know what the person before me did. I just want to start fresh. For that reason I want to be very clear that I have no interest in "upgrading" these servers.

What I'd like to do is...
Step 0:
    v2.4 Primary   (172.30.0.21)
    v2.4 Secondary (172.30.0.22)
    v2.4 Secondary (172.30.0.23)

Step 1:
    v2.4 Primary   (172.30.0.21)
    v2.4 Secondary (172.30.0.22)
    v2.4 Secondary (172.30.0.23)
    v3.2 Secondary (172.30.0.31)
    v3.2 Secondary (172.30.0.32)
    v3.2 Secondary (172.30.0.33)


Step 2:
    v2.4 Primary   (172.30.0.21)
    v3.2 Secondary (172.30.0.31)
    v3.2 Secondary (172.30.0.32)
    v3.2 Secondary (172.30.0.33)

Step 3:
    v3.2 Secondary (172.30.0.31)
    v3.2 Secondary (172.30.0.32)
    v3.2 Secondary (172.30.0.33)

Step 4:
    v3.2 Primary   (172.30.0.31) # via natural election
    v3.2 Secondary (172.30.0.32) # so it could be this one
    v3.2 Secondary (172.30.0.33) # or maybe this one

So now you know the why and the proposed what. I want to know:
1. Is introducing 3.2 servers to the RS of 2.4 servers is okay to do?
2. Is this how you would do it?

I'd really like to AVOID doing a rolling upgrade to 2.6, then a rolling upgrade to 3.0, then starting at Step 1 with s/v2.4/v3.0/

Stephen Steneker

unread,
Oct 26, 2016, 2:57:34 AM10/26/16
to mongodb-user
On Wednesday, 26 October 2016 15:11:28 UTC+11, Bruno Bronosky wrote:
Before you tell me to RTFM, I have read:
https://docs.mongodb.com/manual/release-notes/2.6-upgrade/ as well as countless other docs. I'm still asking the question because I cannot get a clear answer on whether those different versions can co exist in Replica Set if no upgrading is going to take place.

Hi Bruno,

You cannot skip major versions in the upgrade path using replication. Adjacent major versions (2.4 and 2.6, 2.6 and 3.0, or 3.0 and 3.2) are only supported for the purposes of upgrades, and aside from replacing the binaries there may be other steps required (for example, updating authentication schema or checking your driver/application for other compatibility changes).

Can you clarify what you mean by "no upgrading going to take place"? If you plan to use a newer major server release (2.6+) with older data files, that is implicitly an upgrade. The prerequisites are outlined in the upgrade notes you have already reviewed, which includes:
  • (MongoDB 3.0) All replica set members must be running version 2.6 before you can upgrade them to version 3.0
  • (MongoDB 3.2) All replica set members must be running version 3.0 before you can upgrade them to version 3.2

So now you know the why and the proposed what. I want to know:
1. Is introducing 3.2 servers to the RS of 2.4 servers is okay to do?

This is definitely not OK and will not work.

 
2. Is this how you would do it?

I'd really like to AVOID doing a rolling upgrade to 2.6, then a rolling upgrade to 3.0, then starting at Step 1 with s/v2.4/v3.0/

Can you confirm a few points:
  • What O/S version are you using?
  • Are you OK with downtime (if that avoids doing successive rolling upgrades)?
  • Is your deployment using authentication?
  • Do you have a large number of authenticated users?

There are several approaches you could consider:


The straightforward way to upgrade would be importing your deployment into Cloud Manager Automation, which will take care of the successive upgrades and related steps for you. The Automation feature is available with a 30-day free trial, which should give you enough time to complete your migration. Note: Automation support for MongoDB 2.4 is scheduled to be removed in January 2017 (see: MongoDB Compatibility).


2) Follow the documented process
 
If you want to avoid downtime, successive upgrades are the safer and supported path. I would recommend staggering the rolling upgrades (eg. perhaps one a week) so you have opportunity to test and any resolve any issues before continuing to the next major version upgrade. 

 
3) Try a shortcut
 
If you are OK with downtime and recreating your users, a more direct approach would be to dump & restore your data. There are some nuances since MongoDB 2.4 is several major releases out of date and you are basically rolling up three years of server updates into one leap.
 
In particular, the v1 authentication schema used in MongoDB 2.4 cannot be directly restored into MongoDB 3.2's v5 authentication schema. In the normal upgrade path the authentication schema would be successively upgraded.

A general path which may have some speed bumps (particularly if you haven't reviewed the compatibility notes or updated the driver used by your application):
    • Prior to any server upgrades, make sure that your application and driver are compatible with MongoDB 3.2
    • Use mongodump (2.4) to backup your 2.4 deployment
      • If you are using authentication, move (or remove) any user authentication files from the dump directory
      • Example of checking for files on Linux: `ls -la dump/*/system.users*`
    • Create a new replica set using MongoDB 3.2
      • Use mongorestore (3.2) to restore your backup 
      • Recreate your users
4) Consider MongoDB Atlas

As a variation on the prior options, you could consider using MongoDB Atlas which is hosted MongoDB as a service. This is similar to approach #3 (backing up your data in 2.4 and restoring into a 3.2 deployment) with the benefits of Cloud Manager (monitoring, backup, and provisioning) as well as streamlined operational management. 

Irrespective of which approach you choose, I would test the process thoroughly with your application and a backup of your data in a staging environment to ensure there are no unexpected challenges.

Regards,
Stephen

Bruno Bronosky

unread,
Oct 27, 2016, 11:35:41 PM10/27/16
to mongodb-user


On Wednesday, October 26, 2016 at 2:57:34 AM UTC-4, Stephen Steneker wrote:
On Wednesday, 26 October 2016 15:11:28 UTC+11, Bruno Bronosky wrote:
Before you tell me to RTFM, I have read:
https://docs.mongodb.com/manual/release-notes/2.6-upgrade/ as well as countless other docs. I'm still asking the question because I cannot get a clear answer on whether those different versions can co exist in Replica Set if no upgrading is going to take place.

Hi Bruno,

You cannot skip major versions in the upgrade path using replication. Adjacent major versions (2.4 and 2.6, 2.6 and 3.0, or 3.0 and 3.2) are only supported for the purposes of upgrades, and aside from replacing the binaries there may be other steps required (for example, updating authentication schema or checking your driver/application for other compatibility changes).

Can you clarify what you mean by "no upgrading going to take place"? If you plan to use a newer major server release (2.6+) with older data files, that is implicitly an upgrade.

I DID NOT intend to install 3.2 on a new box with a copy of the 2.4 data files.
I DID intend to one of the following...
1. Start up a 3.2 box with no data files (described as the 1st strategy in https://docs.mongodb.com/v3.2/tutorial/expand-replica-set/#prepare-the-data-directory) and do an rs.add() on the RS Primary. Once it caught up, I would take a snapshot of it to create the other 2 new boxen. Introduce to the RS, elect a 3.2 primary, and remove the 2.4 boxen.
2. Start up a 2.6 box with the 2.4 data files. Stop mongod, upgrade to 3.0, and start. Stop mongod, upgrade to 3.2, and start. Make 2 copies of this new box for a total of 3 boxen. Now that I have 3 boxen with reletively up to date data (described as the 2nd strategy in https://docs.mongodb.com/v3.2/tutorial/expand-replica-set/#prepare-the-data-directory), add them to the Replica Set. Once caught up, remove the 3 2.4 boxen.

The prerequisites are outlined in the upgrade notes you have already reviewed, which includes:
  • (MongoDB 3.0) All replica set members must be running version 2.6 before you can upgrade them to version 3.0
  • (MongoDB 3.2) All replica set members must be running version 3.0 before you can upgrade them to version 3.2

So now you know the why and the proposed what. I want to know:
1. Is introducing 3.2 servers to the RS of 2.4 servers is okay to do?

This is definitely not OK and will not work.

 
2. Is this how you would do it?

I'd really like to AVOID doing a rolling upgrade to 2.6, then a rolling upgrade to 3.0, then starting at Step 1 with s/v2.4/v3.0/

Can you confirm a few points:
  • What O/S version are you using?
The current v2.4 boxen run Ubuntu 14.04 LTS.
The new v3.2 boxen will run Ubuntu 16.04 LTS. 
  • Are you OK with downtime (if that avoids doing successive rolling upgrades)?
I want to avoid it. This my first job administering MongoDB and I'd like the learning experience of doing a ZDT upgrade. 
  • Is your deployment using authentication?
No 
  • Do you have a large number of authenticated users?
rs-prod:PRIMARY> db.getUsers().length
0 

Stephen Steneker

unread,
Oct 28, 2016, 12:55:16 AM10/28/16
to mongodb-user
On Friday, 28 October 2016 14:35:41 UTC+11, Bruno Bronosky wrote:
Can you clarify what you mean by "no upgrading going to take place"? If you plan to use a newer major server release (2.6+) with older data files, that is implicitly an upgrade.

I DID NOT intend to install 3.2 on a new box with a copy of the 2.4 data files.
I DID intend to one of the following...
1. Start up a 3.2 box with no data files (described as the 1st strategy in https://docs.mongodb.com/v3.2/tutorial/expand-replica-set/#prepare-the-data-directory) and do an rs.add() on the RS Primary. Once it caught up, I would take a snapshot of it to create the other 2 new boxen. Introduce to the RS, elect a 3.2 primary, and remove the 2.4 boxen.

Hi Bruno,

Thank you for clarifying your expectations. You cannot mix 2.4 and 3.2 members in the same replica set, so this first step will not work as described. With or without data files, there are incompatible changes to replication and server internals and this is still a scenario subject to the same prerequisites in the upgrade notes.

Since you aren't using authentication, this does avoid some of the steps (eg. upgrading auth schema in 2.6 and 3.0).


2. Start up a 2.6 box with the 2.4 data files. Stop mongod, upgrade to 3.0, and start. Stop mongod, upgrade to 3.2, and start. Make 2 copies of this new box for a total of 3 boxen. Now that I have 3 boxen with reletively up to date data (described as the 2nd strategy in https://docs.mongodb.com/v3.2/tutorial/expand-replica-set/#prepare-the-data-directory), add them to the Replica Set. Once caught up, remove the 3 2.4 boxen.

Your step 2 could work .. iff you create a second deployment on your new servers using a backup of your data, follow the upgrade path to 3.2, and eventually cut over from the old to new deployment. Swapping between two deployments with minimal downtime will mean you need some strategy on how to handle data that has changed in the interim.

  • Are you OK with downtime (if that avoids doing successive rolling upgrades)?
I want to avoid it. This my first job administering MongoDB and I'd like the learning experience of doing a ZDT upgrade. 

The minimal downtime approach is using rolling upgrades. If you are open to the option of Cloud Manager Automation, it can orchestrate the process for you and is probably the easiest and fastest option to get you from 2.4 to 3.2.

Aside from server upgrades, you also need to plan to update the driver used by your application(s). Drivers that haven't been updated since the 2.4 era will not have support for some newer features of MongoDB (for example, the WiredTiger storage engine which is the default in MongoDB 3.2). I'd recommend reviewing the Driver Compatibility information.

If you have any further questions, please let us know.

Regards,
Stephen
Reply all
Reply to author
Forward
0 new messages