Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
Message from discussion slave '' is not initialized or invalid

Received: by 10.213.2.72 with SMTP id 8mr1032033ebi.0.1322676977307;
        Wed, 30 Nov 2011 10:16:17 -0800 (PST)
X-BeenThere: nosql-databases@googlegroups.com
Received: by 10.14.15.103 with SMTP id e79ls1036086eee.6.gmail; Wed, 30 Nov
 2011 10:16:16 -0800 (PST)
Received: by 10.14.9.12 with SMTP id 12mr261801ees.0.1322676976767;
        Wed, 30 Nov 2011 10:16:16 -0800 (PST)
Received: by 10.14.9.12 with SMTP id 12mr261800ees.0.1322676976731;
        Wed, 30 Nov 2011 10:16:16 -0800 (PST)
Return-Path: <the.mindstorm.mailinglist+caf_=nosql-databases=googlegroups....@gmail.com>
Received: from mail-ey0-f173.google.com (mail-ey0-f173.google.com [209.85.215.173])
        by gmr-mx.google.com with ESMTPS id n13si1745093eef.2.2011.11.30.10.16.16
        (version=TLSv1/SSLv3 cipher=OTHER);
        Wed, 30 Nov 2011 10:16:16 -0800 (PST)
Received-SPF: pass (google.com: domain of the.mindstorm.mailinglist+caf_=nosql-databases=googlegroups....@gmail.com designates 209.85.215.173 as permitted sender) client-ip=209.85.215.173;
Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of the.mindstorm.mailinglist+caf_=nosql-databases=googlegroups....@gmail.com designates 209.85.215.173 as permitted sender) smtp.mail=the.mindstorm.mailinglist+caf_=nosql-databases=googlegroups....@gmail.com; dkim=pass (test mode) header.i=@googlegroups.com
Received: by eaai10 with SMTP id i10so1173231eaa.4
        for <nosql-databases@googlegroups.com>; Wed, 30 Nov 2011 10:16:16 -0800 (PST)
Received: by 10.213.22.2 with SMTP id l2mr580053ebb.83.1322676976220;
        Wed, 30 Nov 2011 10:16:16 -0800 (PST)
X-Forwarded-To: nosql-databases@googlegroups.com
X-Forwarded-For: the.mindstorm.mailingl...@gmail.com nosql-databases@googlegroups.com
Delivered-To: the.mindstorm.mailingl...@gmail.com
Received: by 10.213.3.4 with SMTP id 4cs14851ebl;
        Wed, 30 Nov 2011 10:16:14 -0800 (PST)
Return-Path: <mongodb-user+bncCNzm5pKBCBDh5dn2BBoE8aZIPw@googlegroups.com>
Received-SPF: pass (google.com: domain of mongodb-user+bncCNzm5pKBCBDh5dn2BBoE8aZIPw@googlegroups.com designates 10.236.161.165 as permitted sender) client-ip=10.236.161.165;
Received: from mr.google.com ([10.236.161.165])
        by 10.236.161.165 with SMTP id w25mr5724451yhk.15.1322676972429 (num_hops = 1);
        Wed, 30 Nov 2011 10:16:12 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=googlegroups.com; s=beta;
        h=x-beenthere:received-spf:date:from:reply-to:to:message-id
         :in-reply-to:references:subject:mime-version:x-original-sender
         :x-original-authentication-results:precedence:mailing-list:list-id
         :x-google-group-id:list-post:list-help:list-archive:sender
         :list-subscribe:list-unsubscribe:content-type;
        bh=t3xe9Dmu6iyta5F30UQy7P9MqQZql6dDrMFpizDZwVQ=;
        b=HFE8IL90c9uwl97FsnWqBPQBMcEtBWVcL1JaTX+xSttMNxgq5OJ4uJIYBy1TpuGlOz
         bSgfLtEASW40uOEJeUiMq2PnEFRN0gjTB1OrBxegrTIvewiV8xcUamBkYIZpi+C7+eox
         Spe8FkP451HmXmxgjzs5Vy2sOeTePRflFp4JQ=
Received: by 10.236.161.165 with SMTP id w25mr1656684yhk.15.1322676961904;
        Wed, 30 Nov 2011 10:16:01 -0800 (PST)
X-BeenThere: mongodb-user@googlegroups.com
Received: by 10.101.2.12 with SMTP id e12ls8916057ani.3.gmail; Wed, 30 Nov
 2011 10:16:01 -0800 (PST)
Received: by 10.236.187.7 with SMTP id x7mr5815619yhm.7.1322676961224;
        Wed, 30 Nov 2011 10:16:01 -0800 (PST)
Received: by 10.236.187.7 with SMTP id x7mr5815612yhm.7.1322676961209;
        Wed, 30 Nov 2011 10:16:01 -0800 (PST)
Received: from mail-qw0-f56.google.com (mail-qw0-f56.google.com [209.85.216.56])
        by gmr-mx.google.com with ESMTPS id b15si633150ybi.1.2011.11.30.10.16.01
        (version=TLSv1/SSLv3 cipher=OTHER);
        Wed, 30 Nov 2011 10:16:01 -0800 (PST)
Received-SPF: pass (google.com: domain of dchenbec...@gmail.com designates 209.85.216.56 as permitted sender) client-ip=209.85.216.56;
Received: by qap15 with SMTP id 15so20526qap.21
        for <mongodb-user@googlegroups.com>; Wed, 30 Nov 2011 10:16:01 -0800 (PST)
Received: by 10.229.250.209 with SMTP id mp17mr76084qcb.15.1322676961018;
        Wed, 30 Nov 2011 10:16:01 -0800 (PST)
Date: Wed, 30 Nov 2011 10:16:00 -0800 (PST)
From: Derek Chen-Becker <dchenbec...@gmail.com>
Reply-To: mongodb-user@googlegroups.com
To: mongodb-user@googlegroups.com
Message-ID: <16546394.869.1322676960708.JavaMail.geo-discussion-forums@vbwb6>
In-Reply-To: <056ac208-8a6d-4b86-9258-0c1704d44...@u6g2000vbg.googlegroups.com>
References: <32236370.177.1322513750294.JavaMail.geo-discussion-forums@yqmm19> <29c2bbb0-3566-4633-8832-9a3adaac9...@o37g2000prn.googlegroups.com>
 <056ac208-8a6d-4b86-9258-0c1704d44...@u6g2000vbg.googlegroups.com>
Subject: [mongodb-user] Re: slave '' is not initialized or invalid
MIME-Version: 1.0
X-Original-Sender: dchenbec...@gmail.com
X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com:
 domain of dchenbec...@gmail.com designates 209.85.216.56 as permitted sender) smtp.mail=dchenbec...@gmail.com
Precedence: list
Mailing-list: list mongodb-user@googlegroups.com; contact mongodb-user+owners@googlegroups.com
List-ID: <mongodb-user.googlegroups.com>
List-Post: <http://groups.google.com/group/mongodb-user/post?hl=en_US>, <mailto:mongodb-user@googlegroups.com>
List-Help: <http://groups.google.com/support/?hl=en_US>, <mailto:mongodb-user+help@googlegroups.com>
List-Archive: <http://groups.google.com/group/mongodb-user?hl=en_US>
Sender: mongodb-user@googlegroups.com
List-Subscribe: <http://groups.google.com/group/mongodb-user/subscribe?hl=en_US>,
 <mailto:mongodb-user+subscribe@googlegroups.com>
List-Unsubscribe: <http://groups.google.com/group/mongodb-user/subscribe?hl=en_US>,
 <mailto:mongodb-user+unsubscribe@googlegroups.com>
Content-Type: multipart/alternative; 
	boundary="----=_Part_868_28933621.1322676960706"

------=_Part_868_28933621.1322676960706
Content-Type: text/plain; charset=ISO-8859-1

Well, this is looking very serious. I ended up having to restart all three 
config servers as well as mongos processes on all app servers to get things 
to the point where the app servers would all report queries consistently. 
Further investigation shows that during the period between where one of our 
shard servers (the primary for its replset) started reporting

Assertion: 10422:write with bad shard config and no server id!

and when I restarted configs/mongos, we are definitely missing data and 
lots of it. Fortunately, we have raw data sent to a completely separate, 
non-sharded replset, so we can reconstruct, but it's going to take days to 
do that with no guarantee that this won't happen again. What can we do to 
help diagnose the issues leading up to this state and prevent it from 
happening in the first place. FYI, everything in the system is Mongo 2.0.1.

Derek

-- 
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To view this discussion on the web visit https://groups.google.com/d/msg/mongodb-user/-/upajgYqL_QMJ.
To post to this group, send email to mongodb-user@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.


------=_Part_868_28933621.1322676960706
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Well, this is looking very serious. I ended up having to restart all three =
config servers as well as mongos processes on all app servers to get things=
 to the point where the app servers would all report queries consistently. =
Further investigation shows that during the period between where one of our=
 shard servers (the primary for its replset) started reporting<div><br></di=
v><div>Assertion: 10422:write with bad shard config and no server id!<br></=
div><div><br></div><div>and when I restarted configs/mongos, we are definit=
ely missing data and lots of it. Fortunately, we have raw data sent to a co=
mpletely separate, non-sharded replset, so we can reconstruct, but it's goi=
ng to take days to do that with no guarantee that this won't happen again. =
What can we do to help diagnose the issues leading up to this state and pre=
vent it from happening in the first place. FYI, everything in the system is=
 Mongo 2.0.1.</div><div><br></div><div>Derek</div>

<p></p>

-- <br />
You received this message because you are subscribed to the Google Groups "=
mongodb-user" group.<br />
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/d/msg/mongodb-user/-/upajgYqL_QMJ">https://groups.google.com/d/msg/mongo=
db-user/-/upajgYqL_QMJ</a>.<br />=20
To post to this group, send email to mongodb-user@googlegroups.com.<br />
To unsubscribe from this group, send email to mongodb-user+unsubscribe@goog=
legroups.com.<br />

For more options, visit this group at http://groups.google.com/group/mongod=
b-user?hl=3Den.<br />



------=_Part_868_28933621.1322676960706--