> Any ideas of what I should be looking for as our bottlekneck?

> On Thu, Oct 11, 2012 at 1:52 PM, Merl Corry <merl...@gmail.com<javascript:>
> > wrote:

>> What are you connecting to from your application?  Mongos

>> When you say, "our cluster loads an unreal amount of data", what exactly
>> do you mean?
>> We have a 3 node sharded cluster.  Each node has 96GB of Ram. When we
>> first start our data loader, we have seen numbers in the range of 4 - 5
>> million documents loaded in less than an hour.  Then it slows down to
>> something in the range of 1 - 2 once Mongo starts to cache.

>> Where does the data come from?  Is it evenly spread across your cluster?
>> The data comes from our systems reporting their transaction log files to
>> the cluster.  All data comes into the first node?

>> When you say, "ram fills up, the chunks starts building up" - RAM on
>> which machine?
>> RAM on the Primary Shard (The 2nd Node).  The chunks will begin to become
>> unbalanced and the primary shard will no distribute the chunks.

>> So in summary:

>> Node 1: Mongod, Mongos, App which loads the data
>> Node 2: Mongod, Mongos, Primary Shard
>> Node 3: Mongod, Mongos, just a Node

>> Each Node: 96GB Ram, 24 Hyperthreaded Cores,  6 - 300GB SAS Drives 16MB
>> Cache 10K RPM (Not High Performance Drives) Raid-10 EXT-4, 10000 mb Network
>> connections

>> On Thu, Oct 11, 2012 at 1:25 PM, Sam Helman <sam.h...@10gen.com<javascript:>
>> > wrote:

>>> To isolate what might be causing your issues:

>>> What are you connecting to from your application?  Is it a mongos, or is
>>> it a direct connection to a mongod?
>>> When you say, "our cluster loads an unreal amount of data", what exactly
>>> do you mean?  Where does the data come from?  Is it evenly spread across
>>> your cluster?
>>> When you say, "ram fills up, the chunks starts building up" - RAM on
>>> which machine?

>>> On Thursday, October 11, 2012 10:42:34 AM UTC-4, Merl wrote:

>>>> It is  2 random 8 character strings as a compound as a compound shard
>>>> key.

>>>> On Thu, Oct 11, 2012 at 10:27 AM, Sam Helman <sam.h...@10gen.com>wrote:

>>>>> I do not know of any issues with Morphia, what is your current shard
>>>>> key?

>>>>> On Thursday, October 11, 2012 9:31:41 AM UTC-4, Merl wrote:

>>>>>> I am curious.  Is morphia still recommended to be used with Mongo?
>>>>>>  We are having major sharding issues with Mongo.  At first it
>>>>>> was believed that our issue was with our sharding key.  We changed the key
>>>>>> to something more random, same problem.  We have upgraded to every new
>>>>>> version of Mongo and still get the same result.  Our cluster loads an
>>>>>> unreal amount of data until ram fills up.  Once ram fills up, the chunks
>>>>>> starts building up on one shard and the cluster becomes unbalanced.  Then
>>>>>> eventually the loading of data backlogs and the cluster becomes completely
>>>>>> unresponsive.  The only thing that has not changed in our entire process is
>>>>>> morphia.  We have used morphia from day one to load data into the system.
>>>>>>  Should we be using something else?

>>>>>  --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "mongodb-user" group.
>>>>> To post to this group, send email to mongod...@googlegroups.com

>>>>> To unsubscribe from this group, send email to
>>>>> mongodb-user...@**googlegroups.com

>>>>> See also the IRC channel -- freenode.net#mongodb

>>>>  --
>>> You received this message because you are subscribed to the Google
>>> Groups "mongodb-user" group.
>>> To post to this group, send email to mongod...@googlegroups.com<javascript:>
>>> To unsubscribe from this group, send email to
>>> mongodb-user...@googlegroups.com <javascript:>
>>> See also the IRC channel -- freenode.net#mongodb