RPM packaging to support Shard Router and Config Servers

[Reid Morrison]

Thank you for the current 10gen RPM packages, they are a huge time
saver when installing MongoDB either standalone or as a replica set on
Red Hat EL.

In moving to a Shard we have bumped up against a few challenges. For
example, when running the config server on an existing Mongo database
server we have had to write custom init.d scripts. To install the
Mongo Router (mongos) on each of the web servers requires installing
the RPM server package, which also installs and starts a full Mongo
database server.

We would like to make the "out of the box" experience for setting up
Mongo shards easier. The objective of these changes is to allow an end-
user to install and have a functional shard setup consisting of the
routers, config servers and Mongo servers on the same or on different
systems without any conflicts. All that is required is to install the
appropriate RPM packages and then edit the relevant config files.

Mongos:

For mongos, we can create a separate RPM package, something like:
'mongo-10gen-sharding-router' which would depend on 'mongo-10gen'.
This RPM package would install mongos and already functional /etc/
init.d/mongos and /etc/mongos.conf files.

In this way mongos can be installed on any web server without starting
the database server, and can also be installed on the database servers
without any conflicts.

Mongo Config server

For the mongo shard config server, we can create a separate RPM
package, something like: 'mongo-10gen-sharding-config' which would
depend on 'mongo-10gen'. This RPM package would install already
functional /etc/init.d/mongoc and /etc/mongoc.conf files. It would be
very similar to the 'mongo-10gen-server' package, but since it has
different init.d and config files it can be installed on the Mongo
server systems without any conflicts.


We have written and tested, init.d scripts and config files that we
want to make available to other people using shards on RHEL. Is
forking the Git repo the right way to go?

We also want to include any features that 10gen and the end user
community need in these new packages. Suggestions and feedback
welcome.

Thank you,
Reid Morrison

[Richard Kreuter]

Having init scripts for mongos and config servers would be a good
thing.

If you wouldn't mind forking the repo, that'd be a good way for us to
see
your changes.

Thanks,
Richard

[Reid Morrison]

I created a fork, along with a feature branch at:
https://github.com/ClarityServices/mongo/tree/feature%2Frpm/rpm

Updated init.d-mongos:
https://github.com/ClarityServices/mongo/commit/a29bedeb55e292b629462bde628b0015e4f3b37a

Also added the mongos init script and sample config:
https://github.com/ClarityServices/mongo/commit/ed56c993214ac80065788444d3e8a30a9763d63a

To create a mongo config server instance the RPM installer would copy
init.d/mongos to /etc/init.d/mongoc. Similarly for the config file the
RPM would copy mongos.conf to /etc/mongoc.conf. Similarly for the
sysconfig file.

If we modify the RPM spec to create the directory /var/run/mongo with
mongod as the owner then lines 44 through 48 in the init.d scripts can
be removed.

[Reid Morrison]

Also added init.d and config file for Mongo Config server since
configurations are less than one would set for a regular mongod
instance.

https://github.com/ClarityServices/mongo/commit/d45a824cedba957808c89f40282a8c18bb44e298

On Aug 26, 2:25 pm, Reid Morrison <rei...@gmail.com> wrote:
> I created a fork, along with a feature branch at:https://github.com/ClarityServices/mongo/tree/feature%2Frpm/rpm
>

> Updated init.d-mongos:https://github.com/ClarityServices/mongo/commit/a29bedeb55e292b629462...
>
> Also added the mongos init script and sample config:https://github.com/ClarityServices/mongo/commit/ed56c993214ac80065788...

[Erez Zarum]

I would use something like this to load the sysconfig, that way you can for example control the ulimit of all instances without changing one by one init script
so if one does not have a sysconfig for the current instance, he still inherit values from /etc/sysconfig/mongod, this should be put after sourcing /etc/rc.d/init.d/functions
so you can add ULIMIT=16000 to /etc/sysconfig/mongod and remove those variables from the init.d script.

if [ -f "$SYSCONFIG" ]; then
  . "$SYSCONFIG"
elif [ -f /etc/sysconfig/mongod ]; then
  . /etc/sysconfig/mongod
fi

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

[Richard Kreuter]

Just FYI: we're in the middle of 2.0 release testing right now, but
I've created a JIRA issue you can track for this improvement:

https://jira.mongodb.org/browse/SERVER-3748

Regards,
Richard

On Aug 26, 2:48 pm, Reid Morrison <rei...@gmail.com> wrote:
> Also added init.d and config file for Mongo Config server since
> configurations are less than one would set for a regular mongod
> instance.
>

> https://github.com/ClarityServices/mongo/commit/d45a824cedba957808c89...

[Christian Csar]

It would have been nice if this had made it into 2.2. That or better support for deploying sharded clusters generally.

[Sosh]

Yeah seconded.  It looks like we don't have it in 2.4 either.