sharding mongos high cpu load

[Oli]

http://npaste.de/ia/

anyone seen somethin similiar before?
running mongodb1.6.2 in a "sharded" environment with currently only
one shard

things i dont understand:

Mon Sep 6 16:21:55 [conn63] ns: postings.postings.facebook.
4c7f9850e9fcaa52d6000008 ClusteredCursor::query ShardConnection had to
change attempt: 0
Mon Sep 6 16:21:55 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:21:55 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:21:56 [conn90] ns: postings.postings.blog.
4c7f9a7ce9fcaa52d600000a ClusteredCursor::query ShardConnection had to
change attempt: 0
Mon Sep 6 16:21:56 [conn91] ns: postings.postings.blog.
4c7f958be9fcaa52d6000001 ClusteredCursor::query ShardConnection had to
change attempt: 0
Mon Sep 6 16:21:56 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:21:56 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:21:56 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:21:57 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:21:58 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:21:58 [conn82] DROP:
postings.tmp.mr.mapreduce_1283782918_321
Mon Sep 6 16:21:58 [conn68] DROP:
postings.tmp.mr.mapreduce_1283782918_320
Mon Sep 6 16:21:58 [conn82] DROP:
postings.tmp.mr.mapreduce_1283782918_341
Mon Sep 6 16:21:58 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:21:58 [conn68] DROP:
postings.tmp.mr.mapreduce_1283782918_344
Mon Sep 6 16:22:00 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:22:00 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:22:01 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:22:02 [conn61] DROP:
postings.tmp.mr.mapreduce_1283782921_389
Mon Sep 6 16:22:12 [conn61] DROP:
postings.tmp.mr.mapreduce_1283782931_409
Mon Sep 6 16:22:22 [conn61] DROP:
postings.tmp.mr.mapreduce_1283782940_411
Mon Sep 6 16:22:25 [conn74] ns: postings.postings.blog.
4c7f958be9fcaa52d6000001 ClusteredCursor::query ShardConnection had to
change attempt: 0
Mon Sep 6 16:22:26 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:22:27 [conn91] ns: postings.postings.twitter.
4c7f958be9fcaa52d6000002 ClusteredCursor::query ShardConnection had to
change attempt: 0
Mon Sep 6 16:22:28 [conn85] ns: postings.postings.twitter.
4c7f958be9fcaa52d6000002 ClusteredCursor::query ShardConnection had to
change attempt: 0
Mon Sep 6 16:22:29 [conn61] DROP:
postings.tmp.mr.mapreduce_1283782949_429
Mon Sep 6 16:22:29 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:22:30 [conn87] ns: postings.postings.twitter.
4c7f958be9fcaa52d6000002 ClusteredCursor::query ShardConnection had to
change attempt: 0
Mon Sep 6 16:22:32 [conn58] ns: postings.postings.blog.
4c7f958be9fcaa52d6000001 ClusteredCursor::query ShardConnection had to
change attempt: 0
Mon Sep 6 16:22:32 [conn66] ns: postings.postings.twitter.
4c7f958be9fcaa52d6000002 ClusteredCursor::query ShardConnection had to
change attempt: 0
Mon Sep 6 16:22:34 [conn93] ns: postings.postings.blog.
4c7f958be9fcaa52d6000001 ClusteredCursor::query ShardConnection had to
change attempt: 0
Mon Sep 6 16:22:42 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:22:42 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:22:42 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:22:43 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:22:43 Assertion failure _grab client/parallel.cpp 461
Mon Sep 6 16:22:47 [conn65] DROP:
postings.tmp.mr.mapreduce_1283782942_413
Mon Sep 6 16:22:47 [conn65] DROP:
postings.tmp.mr.mapreduce_1283782967_492
Mon Sep 6 16:23:15 [conn75] ns: postings.postings.twitter.
4c7f958be9fcaa52d6000002 ClusteredCursor::query ShardConnection had to
change attempt: 0

Mon Sep 6 16:47:34 [conn54] DBException in process: can't do update
with query that doesn't have the shard key
Mon Sep 6 16:55:08 [conn54] DBException in process: can't do update
with query that doesn't have the shard key
Mon Sep 6 16:55:08 [conn54] DBException in process: can't do update
with query that doesn't have the shard key

I hope somone could help me

[Eliot Horowitz]

Are you doing a number of map/reduces?

There is a concurrency issuing with many map/reduces in mongos.

Could that be happening here?

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

[Oli]

yeah you're right im doing a few mapreduces parallel
what do you think is a too large number of concurrent mapreduces?
will this issue be resolved in 1.6.3?

thanks

> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsubscribe@google groups.com>
> > .

[Eliot Horowitz]

We're trying to resolve for 1.6.3.

If you can provide a reproducible case would help a lot.

To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.

[hmeiser]

Hi,

we observed also a 100% cpu-load of the mongos process while executing
a
map/reduce with this mongos process. the mongos is installed on the
client
and not on the sharded nodes.

ciao.hans.

> > > > mongodb-user...@googlegroups.com<mongodb-user%2Bunsu...@googlegroups.com>

> > <mongodb-user%2Bunsubscribe@google groups.com>
> > > > .
> > > > For more options, visit this group at
> > > >http://groups.google.com/group/mongodb-user?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "mongodb-user" group.
> > To post to this group, send email to mongod...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsu...@googlegroups.com>

[Oli]

Could you tell me something about the DBException errors?
I got a lot of them like this:

DBException in process: can't do update with query that doesn't have
the shard key

On 6 Sep., 18:20, Eliot Horowitz <eliothorow...@gmail.com> wrote:

[Eliot Horowitz]

If you do an update with a query that doesn't have the _id or shard key, it has to be a multi update so we can safely send to all servers.

To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.