Indexes and WiredTiger

[MARK CALLAGHAN]

This post is interesting - http://ilearnasigoalong.blogspot.com/2015/03/wired-tiger-how-to-reduce-your-mongdb.html and I appreciate the effort to write it.

But I have some questions after reading it.
1) Is there an option to do block compression for index pages with WT?
2) When will we get support for clustered indexes on _id?

After reading the blog post and then the manual at http://docs.mongodb.org/manual/reference/configuration-options/#storage.wiredTiger it isn't clear to me whether compression can be done for index pages. Some of the indexes that matter a lot to me are covering, so they have extra columns and use more disk space. I prefer to compress them when they are stored on SSD. Can the manual be more clear about this?

AFAIK, with both the WT and RocksDB storage engines in 3.X the _id index isn't clustered. This makes it easier to cache the _id index but also means we might do more random disk IO to go from the _id index to the other columns. If I am correct will there be an option to make _id clustered in the future?

--

Mark Callaghan
mdca...@gmail.com

[Asya Kamsky]

Hi Mark:

The only option I'm aware of with indexes is prefix compression that John talks about in the post you referenced - but this compression applies both on disk and in memory.

Clustered indexes are on the roadmap - I can't tell you that they will make 3.2 but there is active work going on in this area.

Asya

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAFbpF8McYs%3DUA16dCsOUDScKEwfGEy4VRe6gVANz-shzykH84w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
MongoDB World is back! June 1-2 in NYC. Use code ASYA for 25% off!

[MARK CALLAGHAN]

Does this mean that more space than expected will be used for wide covering indexes? The linkbench schema is an example where there is a secondary index that has most of the columns from the table. It gets range scans and we want to avoid random IO back to the base table. I want to get more than prefix compression for that index.

https://github.com/facebook/linkbench

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAOe6dJAN%2BoKwCXgw5OVAXhQycBTWy6nK8vD0Tq0rkL_QU28-2w%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Mark Callaghan
mdca...@gmail.com

[Asya Kamsky]

On Mon, Mar 23, 2015 at 11:26 AM, MARK CALLAGHAN <mdca...@gmail.com> wrote:
> Does this mean that more space than expected will be used for wide covering indexes?

I'm not sure how to answer that precisely since I'm not sure how much space you would be expecting it to take.

> The linkbench schema is an example where there is a secondary index that has most of the columns from
> the table. It gets range scans and we want to avoid random IO back to the base table. I want to get more
> than prefix compression for that index.
> https://github.com/facebook/linkbench

I believe that linkbench (adapted for MongoDB) is one of the tests that our performance team uses to benchmark new vs old versions, etc. They also use "Socialite" - which takes advantage of covering compound indexes as well. I would expect index prefix compression to benefit compounded indexes even more than simple indexes.

I just did a quick check on some real world data in a replica set where two new secondaries were recently added (so they both have relatively compact indexes since they fully resync'ed) the secondaries are 3.0.1 mmapv1 and wiredTiger, default settings.

Indexes in two collections I compared range in size since they have single field, two field and three field indexes, as well as a couple of geoSpatial indexes. The compression that WT gives makes the indexes anywhere from half the size compared to Mmap to barely over 10%.

The highest level of compression was for a compound index but not longest (I.e. not most fields in index).  As you can guess, selectivity of the leading fields will impact the compression in converse way (lower selectivity, highest compression).

Lowest compression, about 40%, was in one of two _id indexes, I didn't look at contents but I'm guessing it was totally or mostly random.

Majority of indexes were compressed down to about 20-30% of their mmap size, with a few in the 11-18% range.

I'm not sure how much better compression you can get on indexes, and given that you will lose some latency to compressing them on writes it seems unlikely to be a beneficial trade-off.

Asya
P.S. I suppose I can try to test whether the compressed indexes are further compressible but I'm not 100% certain what would be a most realistic test for that.

[MARK CALLAGHAN]

I did a few code reviews on the linkbench/mongodb port a long time ago but I haven't heard anything since then. Is there code to share?

I think Mongo/RocksDB vs Mongo/WT is a better way to confirm whether there is a benefit from doing block compression for secondary index pages. AFAIK both do prefix compression for the secondary index pages, RocksDB does block compression (when configured) and WT does not. I think covering indexes will benefit from block compression but we can let the data decide and revisit this later.

--

You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAOe6dJDogmxBqm0ypgmOnK1Hrb%3DRTodNKbdEjrsXoqStBWvbBw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Mark Callaghan
mdca...@gmail.com

[MarkCallaghan]

A few more details on linkbench (http://github.com/facebook/linkbench), which is based on a workload I support at work. The secondary index, id1_type, has all of the columns in the table so that short range scans on it don't do random IO to lookup missing columns from the clustered index on the PK. So we store all columns twice to avoid too much random IO and trade more space for less read amplification. I don't think prefix compression will help much beyond the first 3 columns in the id1_type index.

    CREATE TABLE `linktable` (

      `id1` bigint(20) unsigned NOT NULL DEFAULT '0',

      `id2` bigint(20) unsigned NOT NULL DEFAULT '0',

      `link_type` bigint(20) unsigned NOT NULL DEFAULT '0',

      `visibility` tinyint(3) NOT NULL DEFAULT '0',

      `data` varchar(255) NOT NULL DEFAULT '',

      `time` bigint(20) unsigned NOT NULL DEFAULT '0',

      `version` int(11) unsigned NOT NULL DEFAULT '0',

      PRIMARY KEY (link_type, `id1`,`id2`),

      KEY `id1_type` (`id1`,`link_type`,`visibility`,`time`,`id2`,`version`,`data`)

    ) ENGINE=InnoDB DEFAULT CHARSET=latin1 PARTITION BY key(id1) PARTITIONS 16;

[Alexander Gorrod]

Hi Mark,

> 1) Is there an option to do block compression for index pages with WT?

There is an undocumented option that allows you to enable block compression for WiredTiger indexes in MongoDB. The command line version of the option is 

--wiredTigerIndexConfigString="block_compressor=snappy"

The equivalent configuration file option can also be used: http://docs.mongodb.org/manual/reference/configuration-options/#storage.wiredTiger.indexConfig

You can specify "zlib" or "snappy" as the block compressor. The option is undocumented because it's not recommended for general usage and not tested by MongoDB.

- Alex

[badi]

Hi Alex, 

Cool ! thank you very much !

Could you be so kind and  share with us more such un-documented options ? I like to play with them and compare results. Also options for mongoimport/export would be veeery usefull and appreciated. ;-)

Thanks \ badi

[Alexander Gorrod]

Hi Badi,

> Could you be so kind and  share with us more such un-documented options ?

I don't have general advice about undocumented options that might be worth trying - we choose the best configurations for a majority of workloads "out of the box". If you have specific questions about features or the behavior of specific workloads I'll help where I can.

- Alex