Loadbalancing MongoDB

[AndrewDoc]

Hi, I'm a computer science student and i must to do a project about mongoDB. I have some questions.

How can i download mongoDB source code for analyze it? 

In particolar where i can find the part of code that implements loadbalancing mechanism? 

Thanks

[Wan Bachtiar]

How can i download mongoDB source code for analyze it?

Hi Andrew,

You can find the source code (master) for MongoDB on Github: MongoDB. For the latest stable release, which is currently at v3.2.6, you can check out the tag tree r3.2.6.

In particolar where i can find the part of code that implements loadbalancing mechanism?

By ‘load-balancing’, I assumed you mean the sharding balancing. The cluster balancer is responsible for redistributing the chunks of a sharded collection evenly among the shards for every sharded collection. You could review the code for for the balancer (current stable v3.2.6) on src/mongo/s/balance.cpp

You may also be interested in:

• Maintaining a balanced data distribution.
• Manage Sharded Cluster Balancer.

Best regards,

Wan.

​

[AndrewDoc]

Hi, 

thanks for your response. I have another question.

In mongoDB I have understood that claster balancer distributes chunks in a shard and so i think that it's a write balancer. 

Is there a read balancer too? For me, read balancer redistributes the chunks in a shard in order to balancing the number of reading in every shards. 

Thanks 

Best regards,

Andrew

​

[Wan Bachtiar]

so i think that it’s a write balancer.
Is there a read balancer too? For me, read balancer redistributes the chunks in a shard in order to balancing the number of reading in every shards.

Hi Andrew,

In a sharded cluster, only the distribution of chunks are balanced between shards.

Normally you would connect to mongos in a sharded cluster deployment - mongos is a routing service that determines the location of chunks in the cluster based on the shard key. The location of these chunks determines which shard will receive the read and write operations. See shard-key: write-scaling, shard-key: querying and shard-key: considerations for more information.

I would recommend to review the following to find our more about MongoDB sharding:

• Sharding introduction
• Sharding concepts
• Sharded cluster tutorials

Worth noting that there are free online course at MongoDB University - M102 that covers sharding amongst other topics.

Regards,

Wan.

​

​

[AndrewDoc]

Hi,

   thanks for your response. I have another question.

   I have saw that there are 2 different ways to shard documents in the cluster: hashed or range based. 

   I don't understand where the users can do this choice and in which class there is this method because i searched it but i don't found it.

  

  Thanks, regards

   AndrewDoc

​

​

[Wan Bachtiar]

I don’t understand where the users can do this choice

Hi Andrew,

In order to shard a collection, you have to define a shard key using the sh.shardCollection() method. The usage pattern is such below:

sh.shardCollection( <database.collection>, key )

Where the key parameter consists of a document containing a field and the index type. This is where a user can decide the use of hashed sharding or ranged sharding strategy depending on their use case.

which class there is this method because i searched it but i don’t found it

Based on the latest tag branch v3.3.9, you could check out the ShardCollectionCmd() class. See also parsedShardKeyPattern()

To learn more about sharding, I would recommend to spin up a MongoDB sharded cluster and perform tests for better observation on sharding behaviours.

Regards,

Wan.

​

​

​

[AndrewDoc]

Hi, thanks for your response.

I will search now and if i'll have some questions i'll write it.

Regards,

AndrewDoc

[Pomo]

Hi Wan,

I've a question for you :) 

I saw that documents are assigned to shard in according to shard key. I don't understand where this function is made, in particular which part of code implements this. 

I'm interested to know which part of code implements the initial upload of documents in different shard in according to shard key and shard cluster.

Thank you so much

Pomo

[Wan Bachtiar]

I saw that documents are assigned to shard in according to shard key. I don’t understand where this function is made, in particular which part of code implements this.

Hi Pomo,

Sharding is not a simple operation that could be fitted into a single function or even a module.

Sharding cluster consists of multiple components: mongos, shard, and config servers. The very brief mechanics is, mongos routes queries and write operations to the appropriate chunk in a sharded cluster based on metadata stored in the config servers. See also config database. Having a good understanding for each of the sharding components will help you to navigate the code base better.

If you are interested in how documents/data are distributed, please read Data partitioning with Chunks and mongos: routing and results process

For a write operation, you could start tracing from write_ops/write_op.h (TargetedWrite and ShardEndpoint).

Regards,

Wan.

​

​

​

[Pomo]

Thank you, Wan.

In which class there is the method that contains the piece of code that search the shard (and the chunk) with a specific document inside? Because, when we have to do some operations in a document, we must found this document in a cluster

Thanks,

Pomo