Sharding key

[Chris Scribner]

I wanted to vet a strategy I've been thinking about for choosing a
sharding key. The use case is user created documents which are looked
up randomly by _id. The workload is more read heavy than write heavy.

I'm considering using the ObjectId format with two extra bytes on the
front of the key:

[ 2 - shard num ][ 4 - time ][ 3 - machine ][ 2 - pid ][ 3 -
increment / random ]

If there are 4 shards, the shard num will be randomly selected from 1
of 4 values: 2^15, 2^14, 2^15 + 2^14, or 2^13. (These numbers are
spaced at 1/2, 1/4, 3/4, and 1/8 along 2^16 (max shard num)).
Prefixing the _id key with these numbers will spread the writes out
into 4 "insertion points".

That would give equal write and read distribution across 4 shards, and
inserts within each shard will be in order to improve the likelihood
the correct part of the _id index is in memory and reduce page
splits / fragmentation.

It gets a little messier when we add shards. When a 5th shard is
added, the data will be re-balanced. We will update the shard num to
include the point 2^14 + 2^13 (3/8 of 2^16), to give us 5 "insertion
points".

Inserts will no longer go to the end of each shard, but at least they
will go to the same place in each shard. I would guess that means a
little more page splitting, but the relevant portion of the _id index
should still be in memory.

Has anyone considered this sort of sharding strategy? Does it seem
like a reasonable approach, or are there some other approaches that
are possibly better?

Thanks,

Chris

[Greg Studer]

I'm not sure this method will help you, particularly if your workload
is read-heavy, and it will also make it very complex to add capacity.
New inserts may go to similar locations in memory, but your reads will
still be effectively random, and there are more reads.

Sharding by user, as one example, might be better, since your working
set on each shard would effectively be data of the active users at
that time, and spread between shards.

[Chris Scribner]

I understand that it's a hard question to answer given limited
knowledge of the software.

In the collection in question, users are likely to use documents
generated by others, not just themselves. So, I wouldn't expect reads
to be centered around a particular user. For the collection in
question, I think we can say the reads are mostly random, with more
popular documents getting read more often, and some documents left to
wallow in obscurity. (Something a LRU cache should handle decently
well).

Chris

[Chris Scribner]

In response to "very complex to add capacity", all we would have to do
is increase / decrease a shard counter in the application
configuration. The rest is just based on a formula. I don't think it
ends up being all that complicated.

On Oct 21, 6:50 pm, Greg Studer <g...@10gen.com> wrote:

[Greg Studer]

No, the implementation wouldn't be that difficult - it's just that you
lose the advantages of separating your application code from the
backend - cluster maintenance is no longer transparent to the app. I
don't fully understand the software, but if it turns out that insert
speed isn't a bottleneck then keeping it simple on the app side can
save you time until you need the extra speed. But, again, you know
more about the insert rate than I do.

You can also do similar things via pre-splitting if you need this
level of insert control, for example by using a shard key of { day,
rand# } and pre-splitting 1/Nth of the inserts each day between N
shards (via a cron if you like). Changing "day" to "hour" would let
you tune the proportion of actively used index.