How to choose a shard key for Gridfs?
gen liu
sample.file.chunks (sample.file.files doesn't need to be shard).
But I put large file to cluster, I got this message:
alancer move failed: { chunkTooBig: true, estimatedChunkSize:
92511328, errmsg: "chunk too big to move", ok: 0.0 } from: set1 to:
set3 chunk: { _id: "sample.file.chunks-
files_id_ObjectId('4f2ae63b15c5c85a39253522')", lastmod: Timestamp
16000|1, ns: "sample.file.chunks", min: { files_id:
ObjectId('4f2ae63b15c5c85a39253522') }, max: { files_id:
ObjectId('4f2b00b715c5f0b514269785') }, shard: "set1" }
Sat Mar 3 15:21:19 [Balancer] forcing a split because migrate failed
for size reasons
Sat Mar 3 15:21:19 [Balancer] want to split chunk, but can't find
split point chunk ns:sample.file.chunks at:
set1:set1/192.168.1.251:37017,192.168.1.250:37018 lastmod: 16|1 min:
{ files_id: ObjectId('4f2ae63b15c5c85a39253522') } max: { files_id:
ObjectId('4f2b00b715c5f0b514269785') } got: <empty>
As you see, the chunk is too big to move.
One day later, mongos print this message again.
Then how to choose a shard key for GirdFS to avoid "chunk too big to
move"?
gen liu
Scott Hernandez
to increase the chunksize to that to avoid these warning messages.
http://www.mongodb.org/display/DOCS/Choosing+a+Shard+Key#ChoosingaShardKey-GridFS
In the future (next version) you will be able to shard on
[files_id,n], instead of just files_id which should help a good deal
if you have varying file sizes.
https://jira.mongodb.org/browse/SERVER-3746
> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
gen liu
Support sharding GridFS on {"files_id"::1, n:1} should assure that one
files chunks reside on the same shard, right?
else the md5 command will report error.
On 3月3日, 下午11时06分, Scott Hernandez <scotthernan...@gmail.com> wrote:
> How big is your largest file which you store in gridfs? You may want
> to increase the chunksize to that to avoid these warning messages.
>
> In the future (next version) you will be able to shard on
> [files_id,n], instead of just files_id which should help a good deal
gen liu
> to increase the chunksize to that to avoid these warning messages.
>
> In the future (next version) you will be able to shard on
> [files_id,n], instead of just files_id which should help a good deal
Scott Hernandez
This is on the sharding side of things and the default is 200/64 MB depending on the version (pre 2.0).
If the shard key range chunk can not be split then you will probably get larger diferences in storage size even though the number of chunks are balanced.