Chunks Migration violates Shard Key?

[Nikolay Dmitriev]

Hi,
I have a question related to a chunks migration in a MongoDb Balanced Sharded Cluster.

According to Sharded Cluster Balancer manual page, on reaching some thresholds, balancer starts actual chunks migration between shards to balance the chunks distribution.
I don't understand and I could not find a clear answer - how does this process corellate with a shard key range?
How can a chunk with its own range of data based on a shard key be migrated to another shard that simply cannot hold that data because it has different shard key value boundaries?
I can assume this when a hashed sharding strategy is used - as I got it in this case mongodb operates hashed shard key value boundaries for shards on its own and probably can change them (???).
But I cannot understand this for a ranged sharding strategy.

E.g.,, I have 2 shards with a collection sharded on an X key:
1st holds data for X values from 1 to 1mlrd
2nd holds data for X values from 1mlrd to 2mlrd

How can a chunk from the 1st shard migrate to the 2nd when its data's shard keys doesn't fit into the 2nd shard shard key's value range?
Am I getting smth wrong about all this "chunk migration"?

Please, help. I'm stuck )))

[Weishan Ang]

Hi,

I think you are a little confused.

shard holds chunks that specify the shard range.

Using your example:

If shard1 has 250 chunks and shard2 has 5 chunks, during balancing, it will balance the chunk between them and ensure that they are holding similar amount of chunks. The shard range that is "hold" by the chunk will get moved as well.

[Nikolay Dmitriev]

Hi, thanks for your answer!
I think, I got it.

So, using my example, mongodb will automatically adjust the shards' range, so that chunks migration is possible and everything is correct

вторник, 8 августа 2017 г., 18:26:55 UTC+3 пользователь Weishan Ang написал:

[Kevin Adistambha]

Hi Nikolay

I believe the confusion came from this sentence:

on reaching some thresholds, balancer starts actual chunks migration between shards to balance the chunks distribution.

The page in question was talking about the balancer’s method of balancing the number of chunks across the sharded cluster. What you’re missing is the information regarding the data partitioning with chunks. To paraphrase:

MongoDB uses the shard key associated to the collection to partition the data into chunks. A chunk consists of a subset of sharded data. Each chunk has a inclusive lower and exclusive upper range based on the shard key.

So, using my example, mongodb will automatically adjust the shards’ range

The shard key range is not “adjusted” (i.e. not modified), but instead a chunk that contains a continuous range of the shard key is split into two (or more) chunks. The Chunk Splits section describes this process in more details.

I believe the Data Partitioning with Chunks page should have the answer to all your questions regarding chunks and its balancing process.

Best regards,
Kevin

​

[Nikolay Dmitriev]

Thanks, Kevin

четверг, 31 августа 2017 г., 6:57:05 UTC+3 пользователь Kevin Adistambha написал: