Implementation of SERVER-939

[Sergei Tulentsev]

Hey guys, 

Today there was a couple of new comments to that feature request: https://jira.mongodb.org/browse/SERVER-939

And since I'm kinda also needing this, I thought to myself: why not try to do it myself?

Could you guys give me some pointers to where I should start digging from? Any advice? :)

--
Best regards,
Sergei Tulentsev

[Jenna]

Hello Sergei,
As Eliot points out in SERVER-939, in MongoDB version 2.2 (the next
stable release) it will be possible to shard a collection using a
"noBalance" flag, which will ensure that the collection's chunks will
not be automatically migrated. This flag makes it possible to
manually distribute entire collections around a sharded cluster (in
other words, to treat collections as a single chunk).

The process will involve enabling sharding on the collection with the
noBalance flag. Moving the collection from one shard to another will
merely involve moving the collection's chunks:
http://www.mongodb.org/display/DOCS/Moving+Chunks
http://www.mongodb.org/display/DOCS/Configuring+Sharding#ConfiguringSharding-ShardingaCollection

Please let me know if you have any questions or if this solution is
not what you had in mind.

On Aug 1, 11:51 am, Sergei Tulentsev <sergei.tulent...@gmail.com>
wrote:

[Jenna]

Here is the JIRA ticket for the noBalance flag:
https://jira.mongodb.org/browse/SERVER-4621

On Aug 1, 3:53 pm, Jenna <jenna.deboisbl...@10gen.com> wrote:
> Hello Sergei,
> As Eliot points out in SERVER-939, in MongoDB version 2.2 (the next
> stable release) it will be possible to shard a collection using a
> "noBalance" flag, which will ensure that the collection's chunks will
> not be automatically migrated.  This flag makes it possible to
> manually distribute entire collections around a sharded cluster (in
> other words, to treat collections as a single chunk).
>
> The process will involve enabling sharding on the collection with the
> noBalance flag. Moving the collection from one shard to another will

> merely involve moving the collection's chunks:http://www.mongodb.org/display/DOCS/Moving+Chunkshttp://www.mongodb.org/display/DOCS/Configuring+Sharding#ConfiguringS...

[Sergei Tulentsev]

Jenna, thanks. I read that ticket before posting this. It's not good enough for me. In my app there is a lot of small-to-medium collections and they are dynamically created. I reeeaally don't want to complicate my code and have it deal with pre-creating, sharding, disabling balancer and actually moving the collection. Not to mention potential problems when lots of collections are sharded, but not really sharded.

--
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com
To unsubscribe from this group, send email to
mongodb-user...@googlegroups.com
See also the IRC channel -- freenode.net#mongodb

[Jenna]

Hello Sergei,
I'm not sure if this will help you, but another, easier solution might
involve the new tagging feature that is also available in 2.2.
Kristina has a great blog post that illustrates how tags can be used
to pin collections to particular shards (so the process isn't manual
like the noBalance flag solution):
http://www.kchodorow.com/blog/2012/07/25/controlling-collection-distribution/

Approximately how big are your collections? It's possible to adjust
the default chunk size (64MB by default), which may be useful if you'd
like to shard your collections but ensure that the collection's
documents stay together.

On Aug 1, 4:04 pm, Sergei Tulentsev <sergei.tulent...@gmail.com>
wrote:

> Jenna, thanks. I read that ticket before posting this. It's not good enough
> for me. In my app there is a lot of small-to-medium collections and they
> are dynamically created. I reeeaally don't want to complicate my code and
> have it deal with pre-creating, sharding, disabling balancer and actually
> moving the collection. Not to mention potential problems when lots of
> collections are sharded, but not really sharded.
>
>
>
>
>
>
>
>
>
> On Wed, Aug 1, 2012 at 11:53 PM, Jenna <jenna.deboisbl...@10gen.com> wrote:
> > Hello Sergei,
> > As Eliot points out in SERVER-939, in MongoDB version 2.2 (the next
> > stable release) it will be possible to shard a collection using a
> > "noBalance" flag, which will ensure that the collection's chunks will
> > not be automatically migrated.  This flag makes it possible to
> > manually distribute entire collections around a sharded cluster (in
> > other words, to treat collections as a single chunk).
>
> > The process will involve enabling sharding on the collection with the
> > noBalance flag. Moving the collection from one shard to another will
> > merely involve moving the collection's chunks:
> >http://www.mongodb.org/display/DOCS/Moving+Chunks
>

> >http://www.mongodb.org/display/DOCS/Configuring+Sharding#ConfiguringS...

[Sergei Tulentsev]

Here's my typical collection (I have many of them).

{

"ns" : "mydb.mycollection",

"count" : 9724727,

"size" : 1722230412,

"avgObjSize" : 177.09807298446526,

"storageSize" : 1952456704,

"numExtents" : 24,

"nindexes" : 1,

"lastExtentSize" : 335224832,

"paddingFactor" : 1,

"flags" : 1,

"totalIndexSize" : 474363344,

"indexSizes" : {

"_id_" : 474363344

},

"ok" : 1

[Jenna]

Hello Sergei,
Since your collections are large, my idea to increase the default
chunk size probably isn't advisable since balancing large collections
would be expensive. I should have asked at the beginning- could you
tell us why you don't want to shard your collections in the usual
fashion, i.e. by splitting collections up into chunks?

On Aug 1, 5:15 pm, Sergei Tulentsev <sergei.tulent...@gmail.com>
wrote:

> Here's my typical collection (I have many of them).
>
> {
> "ns" : "mydb.mycollection",
> "count" : 9724727,
>  "size" : 1722230412,
> "avgObjSize" : 177.09807298446526,
>  "storageSize" : 1952456704,
> "numExtents" : 24,
> "nindexes" : 1,
>  "lastExtentSize" : 335224832,
> "paddingFactor" : 1,
> "flags" : 1,
>  "totalIndexSize" : 474363344,
> "indexSizes" : {
> "_id_" : 474363344
>  },
> "ok" : 1
>
>
>
>
>
>
>
>
>
> }
> On Thu, Aug 2, 2012 at 1:08 AM, Jenna <jenna.deboisbl...@10gen.com> wrote:
> > Hello Sergei,
> > I'm not sure if this will help you, but another, easier solution might
> > involve the new tagging feature that is also available in 2.2.
> > Kristina has a great blog post that illustrates how tags can be used
> > to pin collections to particular shards (so the process isn't manual
> > like the noBalance flag solution):
>

> >http://www.kchodorow.com/blog/2012/07/25/controlling-collection-distr...