How to specify the MongoDB master replica set in the config file?

[alfr]

Hi,

I've setup my MongoDB with replication. Now I'm trying to run an instance of Sitewhere in one of the VMs that holds the secondary replica set.  In the global config file I just left the default setting of the MongoDB as is, i.e.

                        <!-- MongoDB datastore used for global data model -->

                        <sw:mongo-datastore hostname="localhost" port="27017" databaseName="sitewhere"/>

However, when I ran this SiteWhere it complains with this error:

16:18:58,582 ERROR [MongoUserManagement] MongoUserManagement state transitioned to ERROR.

com.mongodb.CommandFailureException: { "serverUsed" : "localhost:27017" , "ok" : 0.0 , "errmsg" : "not master" , "code" : 10107}

        at com.mongodb.CommandResult.getException(CommandResult.java:76)

        at com.mongodb.CommandResult.throwOnError(CommandResult.java:140)

        at com.mongodb.DBCollectionImpl.createIndex(DBCollectionImpl.java:399)

 

So how do I specify that the db is using replica set and it needs to look for the master?

Thanks.

alf

[Derek Adams]

It sounds like SiteWhere is trying to write to a secondary member, which is not allowed. If you have two SiteWhere instances, they would both have to point to the primary member. You probably want to use an actual IP address rather than "localhost" in the MongoDB settings so that it connects to the correct machine/instance. I'm not 100% sure of your topology, so these are just assumptions.

[alfr]

Hi Derek,

I want to avoid using the actual IP because that would defeat the purpose of clustering.  If the master goes down and a secondary member is elected as the master on a different machine then that would mean modifying the SiteWhere config file.

My understanding is that the MongoDb driver should discover the master regardless which replica it connects to. Is SiteWhere using the Spring MongoDB driver? 

Al

[Derek Adams]

That makes sense. We will try to replicate the problem and see if there is a workaround.

[Duy Anh Pham]

Looking at sitewhere-mongodb project source code.  Sitewhere uses another MongoDB client. 

The code shows that sitewhere only connect to one mongodb hostname & one mongodb port.  To connect to a replica set with this mongodb client, you need to modify source code to connect sitewhere to a list of mongodb servers.

Below is some pseudo code:   (file to edit:  SitewhereMongoClient.java)

String[] hosts;  // array of mongodb hostnames

String[] ports;  // array of mongodb ports

List<ServerAddress> servers = new ArrayList<>();

try {

for (int i=0; i<hosts.length; i++) {

servers.add(new ServerAddress(hosts[i], Integer.parseInt(ports[i])));

}

} catch (Exception e) {

throw new RuntimeException("Invalid Mongodb Configuration");

}

LOGGER.info("Mongodb server list: " + servers);

// Handle authenticated access.

if ((getUsername() != null) && (getPassword() != null)) {

MongoCredential credential =

MongoCredential.createCredential(getUsername(), getDatabaseName(),

getPassword().toCharArray());

this.client =

new MongoClient(servers,

Arrays.asList(credential), builder.build());

}

[Duy Anh Pham]

More info can be found in this link:

http://api.mongodb.com/java/2.10.1/com/mongodb/MongoClient.html

[Derek Adams]

We will need to take a look at how to add support for this in the configuration file and tenant configuration editor. I added a Jira issue to make sure we add support in the near future:

https://sitewhere.atlassian.net/browse/SITEWHERE-326

[Duy Anh Pham]

FYI, this is how I implemented it in my custom sitewhere build:  (simple and fast. backward compatible with old config)

- Modify sitewhere XSD file to use mongodb port as String

- Config in sitewhere-server.xml

<!-- MongoDB datastore used for global data model -->

<sw:mongo-datastore hostname="${datastore.host}" port="${datastore.port}"

databaseName="sitewhere"/>

- Config in sitewhere.properties

// To configure replica set:

// datastore.host=localhost;localhost

// datastore.port=27017;27018

- In SitewhereMongoClient.java

/** Port used to access the Mongo datastore */

private String port = DEFAULT_PORT;

... start() {

...

String[] hosts = getHostname().split(";");

String[] ports = getPort().split(";");

// and add the code in previous reply ....

}

[alfr]

Hi Derek,

Do you know which release do you plan on including this?

Thanks.

alf

[Derek Adams]

Sorry about not including the updates in 1.9.0. It fell through the cracks because we did not assign a release to it. I just assigned it to the 1.10.0 release, which will be available in late January.

Thanks,

Derek

[alfr]

That's great. Thanks.

[alfr]

Hi Derek,

I was looking at 1.10.0 codes and it seems that the following lines are incorrect: I believe they should be reversed?  If it is a replica set then the client should receive more than 1 host.

I actually tried modifying it and it seems to work when I step down the primiary while Sitewhere is runing.

if (isUsingReplicaSet) {

   this.client = new MongoClient(addresses.get(0), Arrays.asList(credential), builder.build());

} else {

   this.client = new MongoClient(addresses, Arrays.asList(credential), builder.build());

}

   }

   // Handle unauthenticated access.

   else {

if (isUsingReplicaSet) {

   this.client = new MongoClient(addresses.get(0), builder.build());

} else {

   this.client = new MongoClient(addresses, builder.build());

}

   }

alfr

[Derek Adams]

Good catch.. the logic is definitely backwards. It's weird that it seemed to work when we were testing. I'll swap the logic and do some more testing.

Thanks,
Derek

[alfr]

Perhaps the first host in the list is already a primary so the client will just see it as a replica set with 1 member.

[Derek Adams]

Yes, that's what I was thinking as well. We checked in an update with the logic corrected.