MongoDB: Change block_compressor for existing collection

[Tobias Kempkensteffen]

Hi,

I have a server running with wiredTiger storage engine, but without compression enabled globally.

Now I want to change the "block_compressor" option for a single collection to use "snappy".

According to the mongoDB documentation this is possible while creating a new collection using "db.createCollection()", but how can I perform this step for an existing one? (Maybe for new replica set members before initial sync)

Since initial sync between non-wiredTiger and wiredTiger members in a replica set is performed on a per-document level (and not by copying the files), this should be possible...

http://docs.mongodb.org/manual/reference/method/db.createCollection/#specify-storage-engine-options

[Tobias Kempkensteffen]

Still having the problem.

Am I the only one with existing collections!? :)

[Kevin Adistambha]

Hi Tobias,

Now I want to change the “block_compressor” option for a single collection to use “snappy”.

It is not possible to change the block_compressor option of an existing collection.

However, you can create a new collection with the required options and replace the old collection:

1. Create a new collection with the parameters you need, as described in the Specify Storage Engine Options page. E.g.: db.createCollection( "coll_snappy", { storageEngine: { wiredTiger: { configString: 'block_compressor=snappy' }}})
2. Copy all data from the existing collection into the new collection. Note: Please refrain from adding new data into the old collection during this stage.
3. Drop the old collection and rename the new collection using db.collection.renameCollection(), so that the new collection replaces the old collection.

Since initial sync between non-wiredTiger and wiredTiger members in a replica set is performed on a per-document level (and not by copying the files), this should be possible…

It is possible to mix different storage engines in a replica set, but it is currently not possible (as of MongoDB 3.2) to set different storage engine options for a collection in a replica set. I.e., if you specify a collection to use the snappy compressor, the collection will be using snappy across all secondaries as well (since collection creation options are replicated to the secondaries).

Best regards,
Kevin

​

[Tobias Kempkensteffen]

Hey Kevin,

thanks for your reply!

I already thought about creating a new (compressed) collection and to simpy copy the data, but the collection is about 400GB in size. This would lead to some days of offline time, which is not possible for us.

My new plan is to add a new replica set member with enabled snappy compression and instead of copying the big collection, I would copy all other collections (which are much smaller) to new collections with disabled compression. Not perfect, since I have to disable compression for every new collection, but may be a way to handle the problem.

What do you think about this way?

Best regards,
Tobias

[Kevin Adistambha]

Hi Tobias,

My new plan is to add a new replica set member with enabled snappy compression and instead of copying the big collection, I would copy all other collections (which are much smaller) to new collections with disabled compression. Not perfect, since I have to disable compression for every new collection, but may be a way to handle the problem.

May I ask why you want to turn off compression for some of the collections? Why not just use Snappy compression (the default in WiredTiger engine) for all the collections instead? Snappy requires relatively low CPU resources, but provides a reasonable level of compression. Compressed data has the advantage of lowering disk usage (since there’s less data to transfer) and more efficient memory use in general. Also, if a document cannot be compressed, Snappy will store it uncompressed. Please see the Snappy page for more information.

Once compression is enabled in a specific mongod (e.g. Snappy by default, none, or zlib using --wiredTigerCollectionBlockCompressor), all new collection created in that particular mongod will use that block compressor.

This behaviour can be changed by creating a new collection using db.createCollection with specific storage engine options. However please be aware that any collection-specific setting will override any mongod setting, and it is not possible to change those settings anymore unless the collection is recreated.

If your use case allows it, I would recommend to use Snappy compression for all collections, since the benefits (lower disk usage, more efficient memory usage) generally outweighs the drawbacks (sightly higher CPU utilization). Another benefit for your case is that you don’t need to micromanage collection-level specific settings.

Best regards,
Kevin

​